PercolatorInfile Class Reference

Class for storing Percolator tab-delimited input files. More...

#include <OpenMS/FORMAT/PercolatorInfile.h>

Public Types
using	PinFeatureMetaValueMap = std::map< std::pair< size_t, size_t >, std::set< std::string > >
	PIN meta values added to each hit, keyed by (peptide ID index, hit index).

Static Public Member Functions
static void	store (const std::string &pin_file, const PeptideIdentificationList &peptide_ids, const StringList &feature_set, const std::string &enz, int min_charge, int max_charge)
	Writes the PSMs of peptide_ids as a Percolator input file.
static PeptideIdentificationList	load (const std::string &pin_file, bool higher_score_better, const std::string &score_name, const StringList &extra_scores, StringList &filenames, std::string decoy_prefix="", double threshold=0.01, bool SageAnnotation=false)
	Loads peptide identifications from a Percolator input file.
static std::string	getScanIdentifier (const PeptideIdentification &pid, size_t index)
static std::string	getFileIdentifier (const PeptideIdentification &pid)
	The spectrum file of a PSM: its 'file_origin' (empty if not set), followed by '|' and its 'id_merge_index' if that is set.
static std::string	getFileIdentifier (const PeptideIdentification &pid, UInt file_origin_index, UInt id_merge_index_index)
	getFileIdentifier() with the registry indices of 'file_origin' and 'id_merge_index' looked up once (MetaInfoRegistry::getIndex(); UInt(-1): not registered, so no PSM carries it): for loops over many PSMs, where the lookups by name take the registry's lock per call. Look the indices up after the PSMs are loaded: a name registered later (setMetaValue() by name) is not seen through an index cached as UInt(-1).
static StringList	getStandardFeatureSet (int min_charge, int max_charge)
	Returns the standard Percolator feature columns every .pin file should declare.
static StringList	getFeatureSet (const ProteinIdentification::SearchParameters &search_parameters, int min_charge, int max_charge)
	The feature columns of the .pin file of the PSMs of a search run: getStandardFeatureSet(), the search engine's own features, then Peptide and Proteins.
static StringList	getPinColumnNames (int min_charge, int max_charge)
	The names of all PIN columns (meta values) that stampPinFeaturesOnHits sets on a hit.
static std::set< std::pair< size_t, size_t > >	stampPinFeaturesOnHits (PeptideIdentificationList &peptide_ids, const std::string &enz, int min_charge, int max_charge)
	Compute and stamp PIN-equivalent meta values on every PeptideHit.
static std::set< std::pair< size_t, size_t > >	stampPinFeaturesOnHits (PeptideIdentificationList &peptide_ids, const std::string &enz, int min_charge, int max_charge, PinFeatureMetaValueMap &added_meta_values)
	Compute and stamp PIN-equivalent meta values, recording newly added keys.
static double	getFeatureValue (const DataValue &value, const std::string &feature)
	Numeric value of a PIN feature, as the percolator executable reads it.
static Size	buildFeatureMatrix (const PeptideIdentificationList &peptide_ids, const StringList &features, const std::string &enz, int min_charge, int max_charge, RescoreInput &input, std::vector< std::pair< size_t, size_t > > &rows)
	The feature matrix of in-process rescoring: the PIN columns of the PSMs as Percolator reads them from a .pin file, computed straight into dense rows, without meta values on the hits.
static void	refreshPinMetaValues (PeptideIdentificationList &peptide_ids, const std::string &enz, int min_charge, int max_charge)
	Sets the PIN columns that the hits already carry as meta values to their PIN values, adding none.

Protected Types
enum	PinColumn_ : int {   PIN_SPEC_ID , PIN_SCAN_NR , PIN_LABEL , PIN_CALC_MASS ,   PIN_EXP_MASS , PIN_DELTAMASS , PIN_RETENTIONTIME , PIN_MASS ,   PIN_SCORE , PIN_PEPLEN , PIN_ENZ_N , PIN_ENZ_C ,   PIN_ENZ_INT , PIN_DM , PIN_ABSDM , PIN_PEPTIDE ,   PIN_PROTEINS , PIN_CHARGE }
	Fixed PIN columns as computed by pinHitColumns_ (the charge{N} columns follow from PIN_CHARGE on) More...
enum	PinLookup_ : int { LOOKUP_CALC_MASS , LOOKUP_ISOTOPE_ERROR_LEGACY , LOOKUP_ISOTOPE_ERROR }
	Meta values of a hit that pinHitColumns_ reads. More...

Static Protected Member Functions
static void	warnWithoutScanNumber_ (const std::vector< std::string > &without_scan, const std::vector< char > &stamped)
template<bool with_strings, typename Exists , typename Get , typename Put >
static double	pinHitColumns_ (const PeptideHit &hit, bool decoy, const std::string &spec_id, Int scan_number, double exp_mass, double retention_time, const PinEnzyme_ &enz, int min_charge, int max_charge, Exists &&exists, Get &&get, Put &&put)
	The PIN columns of one hit: the formulas of the .pin file, shared by stamping and buildFeatureMatrix.
static TextFile	preparePin_ (const PeptideIdentificationList &peptide_ids, const StringList &feature_set, const std::string &enz, int min_charge, int max_charge)
static bool	isEnz_ (const char &n, const char &c, const std::string &enz)
static Size	countEnzymatic_ (const std::string &peptide, const std::string &enz)

Detailed Description

Class for storing Percolator tab-delimited input files.

The enzyme name of the enzN/enzC/enzInt features (enz of the functions below) is one of Percolator's lower-case names (trypsin, trypsinp, chymotrypsin, thermolysin, proteinasek, pepsin, elastase, lys-n, lys-c, arg-c, asp-n, glu-c); any other name makes every site enzymatic. The suffix "+termini" (e.g. "trypsinp+termini") makes a protein terminus an enzymatic site for enzN/enzC, as Percolator counts it; without it OpenMS' protein-terminus marks are compared as residues (enzN = 0 at a protein N-terminus).

Member Typedef Documentation

PinFeatureMetaValueMap

using PinFeatureMetaValueMap = std::map<std::pair<size_t, size_t>, std::set<std::string> >

PIN meta values added to each hit, keyed by (peptide ID index, hit index).

Member Enumeration Documentation

PinColumn_

enum PinColumn_ : int

protected

Fixed PIN columns as computed by pinHitColumns_ (the charge{N} columns follow from PIN_CHARGE on)

Enumerator
PIN_SPEC_ID
PIN_SCAN_NR
PIN_LABEL
PIN_CALC_MASS
PIN_EXP_MASS
PIN_DELTAMASS
PIN_RETENTIONTIME
PIN_MASS
PIN_SCORE
PIN_PEPLEN
PIN_ENZ_N
PIN_ENZ_C
PIN_ENZ_INT
PIN_DM
PIN_ABSDM
PIN_PEPTIDE
PIN_PROTEINS
PIN_CHARGE

PinLookup_

enum PinLookup_ : int

protected

Meta values of a hit that pinHitColumns_ reads.

Enumerator
LOOKUP_CALC_MASS
LOOKUP_ISOTOPE_ERROR_LEGACY
LOOKUP_ISOTOPE_ERROR

Member Function Documentation

buildFeatureMatrix()

static Size buildFeatureMatrix ( const PeptideIdentificationList &  peptide_ids, const StringList &  features, const std::string &  enz, int  min_charge, int  max_charge, RescoreInput &  input, std::vector< std::pair< size_t, size_t > > &  rows  )

static

The feature matrix of in-process rescoring: the PIN columns of the PSMs as Percolator reads them from a .pin file, computed straight into dense rows, without meta values on the hits.

Row for row and value for value what in-process rescoring formerly read back after stampPinFeaturesOnHits(): the PIN columns are computed by the same functions as there (same arithmetic, so the same doubles), the other features are read from the hits' meta values with getFeatureValue. The rows are the hits that stampPinFeaturesOnHits() does not skip (they have peptide evidences and a known target/decoy type) and that carry every feature, in the order of the identifications and their hits (none without features). Filled per row:

• input.feature_names = features, input.feature_matrix (dense, row-major), input.features stays empty
• input.is_decoy (Label -1), input.scan_numbers (ScanNr), input.exp_masses (ExpMass, the precursor m/z), input.calc_masses (CalcMass), input.spec_file_numbers (the spectrum files, see getFileIdentifier, in order of appearance, as Percolator numbers the FileName column)
• rows: the hit of each row, (index of its PeptideIdentification, index of the hit): its peptide, proteins and spectrum

The spectra are processed in parallel. Errors are those that stampPinFeaturesOnHits() and then the reading of the stamped feature rows raised, in that order: the first error of the stamping by spectrum, else the first error of the rows. Hits without evidence or target/decoy type are reported with a warning, as by stamping. The meta value names of the PIN columns are registered (as by stamping the first hit), so meta values that later code adds keep the order they had when rescoring stamped (MetaInfo orders by name index).

Parameters

[in]	peptide_ids	The PSMs (not modified)
[in]	features	The numeric features (columns): PIN columns (e.g. of getStandardFeatureSet; not SpecId, Peptide, Proteins) or meta values of the hits
[in]	enz	Enzyme name (as for store)
[in]	min_charge	Lower bound for the charge{N} one-hot features
[in]	max_charge	Upper bound for the charge{N} one-hot features
[out]	input	The feature matrix and the bookkeeping columns
[out]	rows	The hit of each row

Returns

The number of skipped hits (no evidence or unknown target/decoy type)

Exceptions

Exception::InvalidValue

for a feature without a finite numeric value (see getFeatureValue), an unknown target/decoy value, or a string column (SpecId, Peptide, Proteins) among features

countEnzymatic_()

static Size countEnzymatic_ ( const std::string &  peptide, const std::string &  enz  )

staticprotected

getFeatureSet()

static StringList getFeatureSet ( const ProteinIdentification::SearchParameters &  search_parameters, int  min_charge, int  max_charge  )

static

The feature columns of the .pin file of the PSMs of a search run: getStandardFeatureSet(), the search engine's own features, then Peptide and Proteins.

The search engine's features are the meta value 'extra_features' of search_parameters (a comma-separated list of meta values of the hits, as the search engines and PSMFeatureExtractor annotate them); without it, the main score ("score"). PercolatorAdapter, ProSE and OpenNuXL rescore their PSMs on this feature set (store(), Percolator::rescorePSMs()).

getFeatureValue()

static double getFeatureValue ( const DataValue &  value, const std::string &  feature  )

static

Numeric value of a PIN feature, as the percolator executable reads it.

The .pin writer (store) prints every feature as text and the percolator executable parses that text as a number. So an integer or floating-point meta value counts as it is, and a string meta value counts as the number it spells: adapters that read search engine scores from text keep them as strings (e.g. SageAdapter, via load). In-process rescoring has to read feature values through this function to train on the same numbers as the executable; DataValue's conversion to double does not parse a string, it yields an unrelated number.

Parameters

[in]	value	Meta value of the feature
[in]	feature	Name of the feature (for the error message)

Returns

The numeric value

Exceptions

Exception::InvalidValue

if value is empty, a list, a string that is not a number, or not finite (NaN, infinity); the executable rejects such a feature as well

getFileIdentifier() [1/2]

static std::string getFileIdentifier ( const PeptideIdentification &  pid )

static

The spectrum file of a PSM: its 'file_origin' (empty if not set), followed by '|' and its 'id_merge_index' if that is set.

The SpecId of a PSM in the .pin file starts with it. When the PSMs of a .pin file come from more than one spectrum file, store also writes it to a 'FileName' column. Percolator numbers the spectrum files from that column and identifies a spectrum by file, ScanNr and ExpMass (e.g. for target-decoy competition), so spectra of different files with the same scan number stay apart.

getFileIdentifier() [2/2]

static std::string getFileIdentifier ( const PeptideIdentification &  pid, UInt  file_origin_index, UInt  id_merge_index_index  )

static

getFileIdentifier() with the registry indices of 'file_origin' and 'id_merge_index' looked up once (MetaInfoRegistry::getIndex(); UInt(-1): not registered, so no PSM carries it): for loops over many PSMs, where the lookups by name take the registry's lock per call. Look the indices up after the PSMs are loaded: a name registered later (setMetaValue() by name) is not seen through an index cached as UInt(-1).

getPinColumnNames()

static StringList getPinColumnNames ( int  min_charge, int  max_charge  )

static

The names of all PIN columns (meta values) that stampPinFeaturesOnHits sets on a hit.

SpecId, ScanNr, Label, CalcMass, ExpMass, deltamass, retentiontime, mass, score, peplen, enzN, enzC, enzInt, dm, absdm, Peptide, Proteins, then charge{min_charge..max_charge}.

getScanIdentifier()

static std::string getScanIdentifier ( const PeptideIdentification &  pid, size_t  index  )

static

getStandardFeatureSet()

static StringList getStandardFeatureSet ( int  min_charge, int  max_charge  )

static

Returns the standard Percolator feature columns every .pin file should declare.

The list contains the three mandatory header columns (SpecId, Label, ScanNr) followed by the standard per-PSM features that preparePin_ computes and sets on every hit: ExpMass, CalcMass, mass, peplen, charge{min..max}, enzN, enzC, enzInt, dm, absdm. Callers should append their search-engine-specific extra_features (and finally Peptide, Proteins) to this list before calling store, as getFeatureSet() does. This is the single source of truth used by PercolatorAdapter and any other tool that emits .pin for external percolator consumption.

isEnz_()

static bool isEnz_ ( const char &  n, const char &  c, const std::string &  enz  )

staticprotected

load()

static PeptideIdentificationList load ( const std::string &  pin_file, bool  higher_score_better, const std::string &  score_name, const StringList &  extra_scores, StringList &  filenames, std::string  decoy_prefix = "", double  threshold = 0.01, bool  SageAnnotation = false  )

static

Loads peptide identifications from a Percolator input file.

This function reads a Percolator input file (pin_file) and returns a vector of PeptideIdentification objects. It extracts relevantinformation such as peptide sequences, scores, charges, annotations, and protein accessions, applying specified thresholds and handling decoy targets as needed. Note: If a filename column is encountered the set of filenames is filled in the order of appearance and PeptideIdentifications annotated with the id_merge_index meta value to link them to the filename (similar to a merged idXML file).

Parameters

[in]	pin_file	he path to the Percolator input file with a .pin extension.
[in]	higher_score_better	A boolean flag indicating whether higher scores are considered better (true) or lower scores are better (false).
[in]	score_name	The name of the primary score to be used for ranking peptide hits.
[out]	extra_scores	A list of additional score names that should be extracted and stored in each PeptideHit.
[out]	filenames	Will be populated with the unique raw file names extracted from the input data.
[in]	decoy_prefix	The prefix used to identify decoy protein accessions. Proteins with accessions starting with this prefix are marked as decoys. Otherwise, it assumes that the pin file already contains the correctly annotated decoy status.
[in]	threshold	A double value representing the threshold for the spectrum_q value. Only spectra with spectrum_q below this threshold are processed. Implemented to allow prefiltering of Sage results.
[in]	SageAnnotation	A boolean value used to determine if the pin file is coming from Sage or not

Returns

A std::vector of PeptideIdentification objects containing the peptide identifications.

Exceptions

`Exception::ParseError`

if any line in the input file does not have the expected number of columns. TODO: implement something similar to PepXMLFile().setPreferredFixedModifications(getModifications_(fixed_modifications_names));

pinHitColumns_()

template<bool with_strings, typename Exists , typename Get , typename Put >

static double pinHitColumns_ ( const PeptideHit &  hit, bool  decoy, const std::string &  spec_id, Int  scan_number, double  exp_mass, double  retention_time, const PinEnzyme_ &  enz, int  min_charge, int  max_charge, Exists &&  exists, Get &&  get, Put &&  put  )

staticprotected

The PIN columns of one hit: the formulas of the .pin file, shared by stamping and buildFeatureMatrix.

Calls put(column, value) for the columns in stamping order (an int, double or bool; the string columns SpecId, Peptide and Proteins only with_strings). CalcMass is put only if the hit has none (exists( LOOKUP_CALC_MASS) false); otherwise its value is read (get) and returned as well. Defined (and so usable) in PercolatorInfile.cpp only.

Returns

The calculated mass of the hit (CalcMass)

preparePin_()

static TextFile preparePin_ ( const PeptideIdentificationList &  peptide_ids, const StringList &  feature_set, const std::string &  enz, int  min_charge, int  max_charge  )

staticprotected

refreshPinMetaValues()

static void refreshPinMetaValues ( PeptideIdentificationList &  peptide_ids, const std::string &  enz, int  min_charge, int  max_charge  )

static

Sets the PIN columns that the hits already carry as meta values to their PIN values, adding none.

stampPinFeaturesOnHits overwrites a meta value that a hit already has under the name of a PIN column (except CalcMass, which it reads) and the cleanup after in-process rescoring kept it: such values reached the output. In-process rescoring keeps that effect with this function: only hits that carry such a value are computed and changed (none if no PIN column name is registered yet). An error (e.g. an invalid CalcMass) stops its spectrum, as stamping; the first by spectrum is raised. rescorePSMs() calls it after buildFeatureMatrix(), which raises the errors of the computation in stamping order first.

Parameters

[in,out]	peptide_ids	The PSMs
[in]	enz	Enzyme name (as for store)
[in]	min_charge	Lower bound for the charge{N} one-hot features
[in]	max_charge	Upper bound for the charge{N} one-hot features

Exceptions

Exception::InvalidValue

as stampPinFeaturesOnHits for a hit that carries such a value

stampPinFeaturesOnHits() [1/2]

static std::set< std::pair< size_t, size_t > > stampPinFeaturesOnHits ( PeptideIdentificationList &  peptide_ids, const std::string &  enz, int  min_charge, int  max_charge  )

static

Compute and stamp PIN-equivalent meta values on every PeptideHit.

Runs the same per-hit computation that preparePin_ applies when writing a .pin file — but mutates the PeptideIdentifications in place instead of writing to a text file. After this call, each kept hit carries the full set of PIN meta values: SpecId, ScanNr, Label, CalcMass, ExpMass, deltamass, retentiontime, mass, score, peplen, charge1..chargeN, enzN, enzC, enzInt, dm, absdm, Peptide, Proteins.

ExpMass and mass are the observed precursor m/z, the same for all hits of a spectrum (Percolator identifies a spectrum by ScanNr and ExpMass). A precursor isotope error ('isotope_error', observed minus theoretical in 13C spacings; or the legacy MS-GF+ 'IsotopeError') is removed from the mass difference only: dm, absdm and deltamass.

ScanNr tells the spectra of a spectrum file apart: the scan number of the native ID (e.g. Thermo); for Bruker TDF DDA ("frame=F scan=S precursor=P") the precursor P; for native IDs that number spectra only within a part of the run (Bruker TDF "frame=F ... scan=S" without a precursor, Waters, SCIEX) a running number per spectrum file, in the order of the PSMs.

In-process rescoring does not stamp: buildFeatureMatrix computes the same values with the same functions straight into feature rows.

Hits with empty PeptideEvidences or UNKNOWN target/decoy status are left untouched; their (pid_index, hit_index) pairs are returned so callers know to skip them.

Parameters

peptide_ids	Mutated in place; each kept hit gets new meta values.
enz	Enzyme name (same values accepted as for store).
min_charge	Lower bound for the charge{N} one-hot features.
max_charge	Upper bound for the charge{N} one-hot features.

Returns

Indices of skipped hits as (pid_index, hit_index) pairs.

stampPinFeaturesOnHits() [2/2]

static std::set< std::pair< size_t, size_t > > stampPinFeaturesOnHits ( PeptideIdentificationList &  peptide_ids, const std::string &  enz, int  min_charge, int  max_charge, PinFeatureMetaValueMap &  added_meta_values  )

static

Compute and stamp PIN-equivalent meta values, recording newly added keys.

This overload additionally records, for every kept hit, the meta-value keys that did not exist before stamping. Callers that temporarily stamp hits can use this record to remove only adapter-generated values afterwards while preserving input metadata with the same names.

Parameters

[in,out]	peptide_ids	Mutated in place; each kept hit gets PIN meta values.
[in]	enz	Enzyme name (same values accepted as for store).
[in]	min_charge	Lower bound for the charge{N} one-hot features.
[in]	max_charge	Upper bound for the charge{N} one-hot features.
[out]	added_meta_values	Newly added keys per (peptide ID, hit) index.

Returns

Indices of skipped hits as (pid_index, hit_index) pairs.

store()

static void store ( const std::string &  pin_file, const PeptideIdentificationList &  peptide_ids, const StringList &  feature_set, const std::string &  enz, int  min_charge, int  max_charge  )

static

Writes the PSMs of peptide_ids as a Percolator input file.

enz names the enzyme of the enzN, enzC and enzInt features: a name of PercolatorAdapter's 'enzyme' option (no_enzyme, elastase, pepsin, proteinasek, thermolysin, chymotrypsin, lys-n, lys-c, arg-c, asp-n, glu-c, trypsin, trypsinp) or of ProteaseDB (e.g. 'Trypsin/P', 'Lys-C', 'PepsinA'), which is mapped to it; another enzyme of ProteaseDB (e.g. 'Lys-C/P', 'V8-E') gives the features by its own cleavage rule. Any other enzyme counts every site as enzymatic. A protein terminus ('[', ']') counts as a cleavage site, and so does the position after an initial Met (initial-Met clipping: flank 'M', evidence start 1). The modifier "+keep_termini" after the name (e.g. "trypsinp+keep_termini") compares the terminus marks as residues and leaves out the initial-Met rule (the features of OpenMS 3.5); "+termini" states the default.

warnWithoutScanNumber_()

static void warnWithoutScanNumber_ ( const std::vector< std::string > &  without_scan, const std::vector< char > &  stamped  )

staticprotected

Warns once about the spectra of without_scan (their reference; empty: none) whose scan number could not be read, of the stamped spectra (1: with stamped PSMs)