|
| | Percolator () |
| |
| | ~Percolator () override |
| |
| | Percolator (const Percolator &)=delete |
| |
| Percolator & | operator= (const Percolator &)=delete |
| |
| void | rescore (std::vector< PeptideIdentification > &peptide_ids, const StringList &feature_names={}) |
| | Rescore PSMs in place. Domain-specific convenience wrapper over rescore(RescoreInput).
|
| |
| RescoreOutput | rescore (const RescoreInput &input) |
| | Rescore a feature matrix domain-agnostically.
|
| |
| Size | rescorePSMs (PeptideIdentificationList &peptide_ids, const StringList &feature_set, const std::string &enzyme, int min_charge, int max_charge, const std::string &score_type="q-value", bool best_hit_only=false) |
| | Rescore search engine PSMs in place on the standard Percolator (.pin) feature set.
|
| |
| Size | rescorePSMsWithScorer (PeptideIdentificationList &peptide_ids, const StringList &feature_set, const std::string &enzyme, int min_charge, int max_charge, const std::string &score_type, const PSMScorer &scorer, bool best_hit_only=false) |
| | As rescorePSMs(), but the feature rows are scored by scorer instead of rescore(input).
|
| |
| PercolatorModel | trainPSMs (PeptideIdentificationList &peptide_ids, const StringList &feature_set, const std::string &enzyme, int min_charge, int max_charge) |
| | Train a model on search engine PSMs (the .pin features of rescorePSMs()), for scorePSMs() on other PSMs.
|
| |
| Size | scorePSMs (PeptideIdentificationList &peptide_ids, const PercolatorModel &model, PeptideIdentificationList &reference_ids, const StringList &feature_set, const std::string &enzyme, int min_charge, int max_charge, const std::string &score_type="q-value") |
| | Rescore PSMs with a model trained on other PSMs (trainPSMs()), on the scale of those (reference_ids, e.g. the training PSMs).
|
| |
| const std::vector< std::vector< double > > & | getSvmWeights () const |
| | SVM weights trained in the last rescore()/train() call.
|
| |
| void | storeWeights (const std::string &filename) const |
| | Writes the models of the last rescore()/train() call as the percolator executable writes them with its option -w (–weights).
|
| |
| const CappedTrainingInfo & | getCappedTrainingInfo () const |
| | The folds, sample and training sets of the last rescore() if it trained out-of-fold on a capped sample (subset_mode 'out_of_fold' or 'out_of_fold_avg', more rows than subset_max_train); otherwise empty. For tests and diagnostics.
|
| |
| double | getPi0 () const |
| | Pi0 (null fraction) from the last rescore()/score() call.
|
| |
| bool | usedDefaultDirection () const |
| | Whether the last rescore()/train() call replaced the learned model by the default direction.
|
| |
| PercolatorModel | train (const RescoreInput &input) |
| | Train a Percolator model on feature rows. No scoring.
|
| |
| RescoreOutput | score (const RescoreInput &input, const PercolatorModel &model) |
| | Score feature rows using a pre-trained model. No training.
|
| |
| RescoreOutput | score (const RescoreInput &input, const PercolatorModel &model, const RescoreInput &reference) |
| | Score feature rows with a pre-trained model on the scale of a reference set (e.g. its training set).
|
| |
| std::shared_ptr< const PercolatorReset::Output > | getResetOutput () const |
| | Result of the last rescorePSMs() call with parameter 'algorithm' = 'reset' (null otherwise).
|
| |
 Public Member Functions inherited from DefaultParamHandler |
| | DefaultParamHandler (const std::string &name) |
| | Constructor with name that is displayed in error messages.
|
| |
| | DefaultParamHandler (const DefaultParamHandler &rhs) |
| | Copy constructor.
|
| |
| virtual | ~DefaultParamHandler () |
| | Destructor.
|
| |
| DefaultParamHandler & | operator= (const DefaultParamHandler &rhs) |
| | Assignment operator.
|
| |
| virtual bool | operator== (const DefaultParamHandler &rhs) const |
| | Equality operator.
|
| |
| void | setParameters (const Param ¶m) |
| | Sets the parameters.
|
| |
| const Param & | getParameters () const |
| | Non-mutable access to the parameters.
|
| |
| const Param & | getDefaults () const |
| | Non-mutable access to the default parameters.
|
| |
| const std::string & | getName () const |
| | Non-mutable access to the name.
|
| |
| void | setName (const std::string &name) |
| | Mutable access to the name.
|
| |
| const std::vector< std::string > & | getSubsections () const |
| | Non-mutable access to the registered subsections.
|
| |
|
| static void | buildPSMRescoreInput (const PeptideIdentificationList &peptide_ids, const StringList &feature_set, const std::string &enzyme, int min_charge, int max_charge, RescoreInput &input, PSMRowLocations &rows, bool calc_mass_feature=false) |
| | The feature matrix of PSM rescoring: what rescorePSMs() trains and scores on, without rescoring.
|
| |
| static void | markRescoredRuns (std::vector< ProteinIdentification > &runs, const std::string &tool, const std::vector< std::pair< std::string, DataValue > > &settings, const std::string &version=VENDORED_VERSION) |
| | Marks search runs as post-processed by Percolator, so that downstream tools see that their PSMs were rescored.
|
| |
| static void | fillPINCompatibleFields (const std::vector< PeptideIdentification > &peptide_ids, bool flatten_hits, RescoreInput &input) |
| | Fill PIN-compatible optional fields on a RescoreInput.
|
| |
| static void | saveModel (const PercolatorModel &model, const std::string &filename) |
| | Serialize a PercolatorModel to a plain-text file.
|
| |
| static PercolatorModel | loadModel (const std::string &filename) |
| | Deserialize a PercolatorModel written by saveModel().
|
| |
| Static Public Member Functions inherited from DefaultParamHandler |
| static void | writeParametersToMetaValues (const Param &write_this, MetaInfoInterface &write_here, const std::string &key_prefix="") |
| | Writes all parameters to meta values.
|
| |
In-process Percolator: semi-supervised target/decoy rescoring with q-values + PEPs.
Wraps a vendored subset of Percolator (training and posterior estimation). The public API is grouped into two sets of entry points.
Combined training and scoring (original Percolator semantics; per-fold score normalization on the merged set):
- rescore(std::vector<PeptideIdentification>&, ...) — overload for PSM-shaped data; writes scores back to each PeptideHit as meta values.
- rescore(const RescoreInput&) — domain-agnostic. Accepts a feature matrix, target/decoy labels, optional CV grouping keys, and feature names. Applicable to PSM rescoring, transition rescoring, peak-group rescoring, or any other setting where a semi-supervised target/decoy classifier is appropriate. The vendored implementation requires PSM-shaped records internally; the wrapper attaches synthetic row identifiers to satisfy that requirement and these never surface in the public API.
Separate training and scoring (for train-on-A / predict-on-B workflows; uses fold-averaged weights):
rescore() and train()+score() produce scores on the same scale but are not bitwise equal on the same input; see score() for the rationale.
Preconditions
- At least ~100 decoys and enough discriminable targets to pass SanityCheck, else InvalidValue is thrown.
- Features are continuous-valued doubles. Categorical features must be one-hot-encoded by the caller.
- If the input contains rows that share structure (e.g., multiple transitions per precursor), cv_group_keys must partition them so related rows go into the same CV fold. Otherwise q-values will be optimistic.
Thread safety
A single instance is not concurrent-safe; construct one instance per worker. The vendored Percolator code additionally relies on several process-wide statics (FeatureNames::numFeatures, SanityCheck::initDefaultDir*, etc.) that are reset at the start of each rescore() / train() / score() call. Concurrent calls across different instances* therefore also race on these globals; serialize at the call site if parallelism is required.
Reproducibility
Results are reproducible given the same seed, thread count, and input ordering. Changing the thread count can perturb results because of FeatureMemoryPool allocation ordering.
- See also
- PSM rescoring example: PercolatorAdapter / ProSE.
-
Transition rescoring example: OpenSwath layer (to be added when needed).
The feature matrix of PSM rescoring: what rescorePSMs() trains and scores on, without rescoring.
Computes the feature vectors that the percolator executable reads from the .pin file of peptide_ids (PercolatorInfile::store with feature_set), row for row and bit for bit, straight into a RescoreInput (PercolatorInfile::buildFeatureMatrix(); no meta values are added to the hits):
- feature_names: the numeric columns of
feature_set. The bookkeeping columns (SpecId, Label, ScanNr, ExpMass, Peptide, Proteins) are not training features.
- feature_matrix: the rows, dense (row-major); features stays empty
- is_decoy (Label), scan_numbers (ScanNr), exp_masses (ExpMass), calc_masses (CalcMass) and spec_file_numbers (the spectrum files in order of appearance), as the executable reads them from the .pin file, so CV folds and target-decoy competition match the external binary
rows: the hit of each row, i.e. its peptide, proteins and spectrum
This matrix is the input of every rescoring of rescorePSMs(): Percolator's training and scoring copy its rows into their PSMs, and an alternative algorithm on PSMs reads the same RescoreInput and rows. Hits without peptide evidences, without target/decoy information or with a missing feature get no row.
CalcMass is not a training feature (the executable reads it as metadata; it goes into calc_masses), unless calc_mass_feature (parameter 'calc_mass_feature', the in-process training before OpenMS #10717).
- Exceptions
-
Fill PIN-compatible optional fields on a RescoreInput.
Populates input.scan_numbers, input.spec_file_numbers, input.exp_masses, and input.calc_masses from the given PeptideIdentifications, using the same derivation that PercolatorInfile::store would apply when writing a .pin file:
- scan: parsed via SpectrumLookup::extractScanNumber from the PeptideIdentification's spectrum_reference (or
spectrum_id meta value, or fallback to 1-based index).
- spec_file: numbers the spectrum files (PercolatorInfile::getFileIdentifier:
file_origin and id_merge_index, the PIN SpecId prefix) in order of appearance, as Percolator numbers the FileName column of a .pin file. Zero when single-file / unset.
- exp_mass:
pid.getMZ() (kept as m/z — Percolator doesn't convert to neutral for the sort hash).
- calc_mass: from
hit.metaValueExists("CalcMass") if present, else hit.getSequence().getMZ(hit.getCharge()) (m/z, matching the PercolatorInfile::store fallback).
This helper must be called before rescore(input) when the in-process output is required to match running the external percolator binary through the .pin / .pout pipeline on the same inputs. Without it, the row index is used in place of the scan number, producing a different CV fold split and consequently different trained weights and final scores.
The PeptideIdentifications vector passed in must parallel input.features exactly: same ordering, one row per hit per PeptideIdentification.
- Parameters
-
| peptide_ids | Source of PIN-equivalent metadata. |
| flatten_hits | If true, iterate all hits per PeptideIdentification (matches high-level rescore row ordering). If false, use only the first hit per pid. |
| input | Output: the four PIN-compat fields are written here. |
| const std::vector< std::vector< double > > & getSvmWeights |
( |
| ) |
const |
SVM weights trained in the last rescore()/train() call.
A single fold-averaged weight vector in raw feature space, with the bias appended as the final element. Identical to PercolatorModel::weights, wrapped in a one-element outer vector to preserve the signature expected by older call sites that consumed per-fold weights. Intended for diagnostics and for writing out a Percolator-compatible .weights file.
Updated by rescore() and train(). score() does not modify this buffer; after a train()+score() sequence the contents reflect the train() call. Loading a model via loadModel() and then calling score() leaves the buffer unchanged (empty if this instance has never trained).
After rescore() with an out-of-fold capped training (parameter subset_mode), the models that scored the rows instead: the three fold models, and with 'out_of_fold_avg' their average (for the unsampled rows) fourth.
- Returns
- Outer size = 1 (averaged; 3 or 4 after an out-of-fold capped rescore()); inner size = num_features + 1. Empty until rescore()/train() has been called on this instance.
| Size rescorePSMs |
( |
PeptideIdentificationList & |
peptide_ids, |
|
|
const StringList & |
feature_set, |
|
|
const std::string & |
enzyme, |
|
|
int |
min_charge, |
|
|
int |
max_charge, |
|
|
const std::string & |
score_type = "q-value", |
|
|
bool |
best_hit_only = false |
|
) |
| |
Rescore search engine PSMs in place on the standard Percolator (.pin) feature set.
This is the in-process equivalent of writing a .pin file, running the percolator executable and reading its results back, without the file and without PIN meta values on the hits. PercolatorAdapter's in-process backend and ProSE ('Search:rescore') use it. Steps:
- build the feature matrix (buildPSMRescoreInput()): the feature vectors the executable would read from the .pin file (PercolatorInfile::store) of
peptide_ids, row for row, computed straight into a dense RescoreInput, with the hit of every row
- call rescore(const RescoreInput&) with the parameters of this instance
- write the results back onto the hits:
- the original main score is kept as a meta value named after the old score type
- MS:1001492 holds the SVM score, MS:1001491 the q-value, MS:1001493 the PEP
- the main score becomes the one selected by
score_type, and every PeptideIdentification gets the matching score type and orientation
No PIN column is added to the hits. A meta value that a hit already carries under the name of a PIN column (other than CalcMass, which is read as the hit's calculated mass) is set to its PIN value, as stamping the PIN features did (PercolatorInfile::refreshPinMetaValues).
Hits without peptide evidences, without target/decoy information or with a missing feature are not rescored (see buildPSMRescoreInput()). Like PSMs missing from the executable's output, they get q-value = PEP = 1 and SVM score = -100 (meta values and main score), so that they rank last under the new score type; their original score is kept as a meta value as well.
- Parameters
-
| [in,out] | peptide_ids | Target and decoy PSMs (annotated with 'target_decoy' and peptide evidences) |
| [in] | feature_set | PIN feature columns, e.g. PercolatorInfile::getStandardFeatureSet() plus the search engine's extra features plus "Peptide" and "Proteins" |
| [in] | enzyme | Enzyme name used for the enzN/enzC/enzInt features (as for PercolatorInfile::store) |
| [in] | min_charge | Lowest charge of the charge{N} one-hot features |
| [in] | max_charge | Highest charge of the charge{N} one-hot features |
| [in] | score_type | New main score: "q-value", "pep" or "svm" |
| [in] | best_hit_only | For callers that keep only the best hit of each identification: the meta values above (the original score, the Percolator scores) are written only to the hit that PeptideIdentification::sort() puts first under the new score; the others get only their new main score. (Identifications with a NaN score: all hits, as without.) |
- Returns
- The number of rescored hits
- Exceptions
-
| static void saveModel |
( |
const PercolatorModel & |
model, |
|
|
const std::string & |
filename |
|
) |
| |
|
static |
Serialize a PercolatorModel to a plain-text file.
Writes a comment header line, then the header keys format_version, normalizer, seed, n_features, and bias (one per line, in key: value form), followed by one feature_name<TAB>weight data row per feature. The bias is stored in the header rather than as a data row, so feature names are opaque strings with no reserved values. The format is intended to be human-readable and diff-friendly; it is not interoperable with the external percolator binary's multi-column .weights format.
Weights are written at std::numeric_limits<double>::max_digits10 precision so that loadModel() round-trips losslessly.
- Exceptions
-
Score feature rows using a pre-trained model. No training.
Applies model.weights to input.features unchanged: raw weights × raw features + bias. Callers must not pre-normalize the features. The normalization transform is already folded into the raw weights by Normalizer::unnormalizeweight() at training time; reapplying it would double-count the transform.
The post-scoring pipeline (q-values, optional TDC, PEPs, SVM score rescaling) operates on the input's own target/decoy distribution. q-values and PEPs are therefore always evaluated against the scoring dataset, not the training dataset. As train() and rescore(), score() starts Percolator's random number generator (used for the bootstrap estimate of pi0, see use_pi0) from the seed parameter, so the same input and model give the same q-values and PEPs, whatever ran before.
The returned RescoreOutput.scores are not raw dot products. Internally raw_score(i) = Σ input.features[i][j] * model.weights[j]
- model.weights[n_features]
Scores::normalizeScores then rescales the entire score vector so that the score at test_fdr maps to 0 and the median decoy score maps to -1: scores[i] = (raw_score(i) - fdrScore) / (fdrScore - medianDecoyScore) The rescore() CV-merge path rescales per fold instead of once globally, so scores from rescore(X) and from score(X, train(X)) share a scale but are not bitwise equal. If raw dot-product values are required, compute them directly from model.weights.
Target-Decoy Competition (enabled by default via post_processing_tdc) deduplicates rows by (scan, expMass). Rows that lose competition are returned with the defaults score = 0.0, q_value = 1.0, pep = 1.0. See the rescore(RescoreInput) note for mitigation.
- Parameters
-
| input | Scoring data. Target/decoy labels required for q-value and PEP computation. feature_names must be populated. |
| model | Model produced by train() (possibly from another Percolator instance, or loaded from disk). feature_names must be populated and match input.feature_names positionally; weight count must equal n_features+1. |
- Returns
- Per-row SVM scores (rescaled per above), q-values, PEPs, aligned 1:1 with
input.features.
- Exceptions
-
| Exception::InvalidValue | on malformed input, model mismatch, empty feature_names on either side, or pathological score distributions that fail Percolator's sanity checks. |
| void storeWeights |
( |
const std::string & |
filename | ) |
const |
Writes the models of the last rescore()/train() call as the percolator executable writes them with its option -w (–weights).
Three comment lines, then per cross-validation fold three tab-separated lines: the feature names and "m0" (the bias); the weights of the normalized features and the bias; the weights of the raw features and the bias. The vendored code of the executable writes it (CrossValidation::printAllWeights), at the same point: after the cross-validation has scored its test sets, i.e. each fold model rescaled by its test set's scores, or replaced by the default direction if the sanity check rejected the learned one. getSvmWeights() is the average of the raw weights (with subset_mode 'out_of_fold' etc.: the fold models before that rescaling). After a capped training (parameter subset_max_train), the models trained on the sample; with subset_mode 'learning_curve', each fold's from the sample size kept for it.
score() doesn't change the models written.
- Parameters
-
| filename | The output file (usually with the extension .weights or .tsv) |
- Exceptions
-
| bool usedDefaultDirection |
( |
| ) |
const |
Whether the last rescore()/train() call replaced the learned model by the default direction.
Percolator's sanity check rejects the learned SVM weights if they identify no target at q < test_fdr on the held-out folds, fewer targets than the default direction, or (with initial_direction) give the initial direction's feature the wrong sign. The scores then come from the default direction: the initial_direction feature, or else the single feature that separates the most targets (the external percolator binary prints "Resetting score vector, using default vector"). Typical for inputs without real identifications, e.g. a blank run, and for small inputs: q-values count the decoys + 1, so a fold needs about 1 / test_fdr targets above its first decoy for any q < test_fdr.
- Returns
- True if the default direction was used; false if the learned model was kept, and before the first rescore()/train() call. score() does not change it.
1.9.8