Running a Simple Search Using Tide and Percolator

Now that you have your environment set up and the two input files in your working directory, you can conduct the search. The search process compares each spectrum in demo.ms2 to peptides (subsequences of the proteins) in fasta files provided in a dirctory, yeast-index/. Peptides whose precursor mass is close to that of the observed spectrum are scored against that spectrum, and the top scores are reported in the output. To conduct the search, we first create a peptide index using tide-index and then execute the search using tide-search.

  1. $ crux tide-index small-yeast.fasta yeast-index

    While generating the peptide index, you will see output like this:

    INFO: CPU: pyrrolysine.gs.washington.edu
    INFO: Crux version: 2.1
    INFO: Fri Feb  5 11:24:02 PST 2016
    COMMAND LINE: ./crux tide-index small-yeast.fasta yeast-index
    INFO: Running tide-index...
    INFO: Writing results to output directory 'yeast-index'.
    INFO: Reading small-yeast.fasta and computing unmodified peptides...
    INFO: Writing decoy fasta...
    INFO: Reading proteins
    INFO: Precomputing theoretical spectra...
    INFO: Elapsed time: 0.0973 s
    INFO: Finished crux tide-index.
    INFO: Return Code:0
    		

    This command produces the peptide index in yeast-index and also produces a directory crux-output containing the following files:

    1. tide-index.decoy.fasta – a set of decoy proteins, derived from the proteins in the input set,
    2. tide-search.params.txt – a record of all the parameters used in the search, and
    3. tide-search.log.txt – a log file containing a copy of all the messages printed to the screen during the search.

    Now you can run this command:

  2. $ crux tide-search --compute-sp T demo.ms2 yeast-index
  3. While the search is running, you will see output like this:

    INFO: CPU: pyrrolysine.gs.washington.edu
    INFO: Crux version: 2.1
    INFO: Fri Feb  5 11:24:23 PST 2016
    COMMAND LINE: ./crux tide-search --compute-sp T demo.ms2 yeast-index
    INFO: Running tide-search...
    INFO: Reading index yeast-index
    INFO: Converting demo.ms2 to spectrumrecords format
    INFO: Reading spectra file crux-output/demo.ms2.spectrumrecords.tmp
    INFO: Sorting spectra
    INFO: Running search
    INFO: Time per spectrum-charge combination: 0.002318 s.
    INFO: Average number of candidates per spectrum-charge combination: 15.204820 
    INFO: Elapsed time: 0.389 s
    INFO: Finished crux tide-search.
    INFO: Return Code:0
    		

    The crux-output directory now contains four new files containing the search results:

    1. tide-search.target.txt – search results in tab-delimited format.
    2. tide-search.decoy.txt – search results from a decoy database in tab-delimited format.
    3. tide-search.params.txt – a record of all the parameters used in the search.
    4. tide-search.log.txt – a log file containing a copy of all the messages printed to the screen during the search.

    Note that the peptide-spectrum matches (PSMs) in the tide-search.target.txt are sorted by the precursor m/z value associated with the spectrum. If you want to see which PSMs got the highest XCorr scores, you can do so like this:

  4. $ crux sort-by-column --column-type real --ascending T crux-output/tide-search.target.txt "xcorr score" > crux-output/tide-search.target.sort.txt
  5. The first lines of the resulting sorted output file should look like this:

    filescanchargespectrum precursor m/zspectrum neutral masspeptide massdelta_cnsp scoresp rankxcorr scorexcorr rankb/y ions matchedb/y ions totaldistinct matches/spectrumsequencecleavage typeprotein idflanking aa
    demo.ms2853497.6181489.831488.820.9369322430.5615.20757127483NFLETVELQVGLKtrypsin-full-digestYGL135W(27)RN
    demo.ms211831031.943092.83093.410.9496982383.1614.762041441082ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009C(500)KD
    demo.ms215631032.443094.33093.410.9383581929.7714.74761431082ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009C(500)KD
    demo.ms21831032.43094.183093.410.9020561732.2314.489331401082ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009C(500)KD
    demo.ms2112745.2691488.521488.820.8228512816.614.44855121244NFLETVELQVGLKtrypsin-full-digestYGL135W(27)RN
    demo.ms2532745.7491489.481488.820.9356482797.7814.39828121243NFLETVELQVGLKtrypsin-full-digestYGL135W(27)RN
    demo.ms23831032.393094.153093.410.9757411460.2814.396271371082ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009C(500)KD
    demo.ms24231032.263093.763093.411.02161509.4714.356761381082ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009C(500)KD
    demo.ms211131033.023096.043093.410.9527271366.7114.34981361082ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009C(500)KD
    demo.ms2622745.8591489.71488.820.9137112766.6514.34057121243NFLETVELQVGLKtrypsin-full-digestYGL135W(27)RN

    The final step is to post-process the search results using Percolator. Each spectrum has been compared to many peptides and we would like to return only the best match for each spectrum. We also expect that some fraction of the spectra will not be identifiable as peptides (due to chemical noise, multiple peptides co-eluting, poor fragmentation, etc.). The analysis step filters out those spectra and ranks the matches by quality.

  6. $ crux percolator --test-fdr 0.1 crux-output/tide-search.target.txt

    While the analysis is running, you will see output like this

    INFO: CPU: pyrrolysine.gs.washington.edu
    INFO: Crux version: 3.1-ccddffef5395314ce3a224ad5b34167229341653
    INFO: Tue Dec 12 15:47:13 PST 2017
    COMMAND LINE: ./crux percolator --test-fdr 0.1 crux-output/tide-search.target.txt
    INFO: Beginning percolator.
    INFO: Reading file crux-output/tide-search.target.txt
    INFO: Converting input to pin format.
    INFO: Parsing crux-output/tide-search.target.txt
    INFO: Assigning index 0 to demo.ms2.
    INFO: Parsing crux-output/tide-search.decoy.txt
    INFO: There are 690 target matches and 690 decoys
    INFO: Maximum observed charge is 3.
    INFO: File conversion complete.
    INFO: Percolator version 3.01.nightly-22-2f4e2a6, Build Date Nov 22 2017 10:07:29
    INFO: Copyright (c) 2006-9 University of Washington. All rights reserved.
    INFO: Written by Lukas Käll (lukall@u.washington.edu) in the
    INFO: Department of Genome Sciences at the University of Washington.
    INFO: Issued command:
    INFO: percolator --results-peptides crux-output/percolator.target.peptides.txt --decoy-results-peptides crux-output/percolator.decoy.peptides.txt --results-psms crux-output/percolator.target.psms.txt --decoy-results-psms crux-output/percolator.decoy.psms.txt --verbose 2 --protein-decoy-pattern decoy_ --seed 1 --subset-max-train 0 --trainFDR 0.01 --testFDR 0.1 --maxiter 10 --search-input auto --no-schema-validation --protein-enzyme trypsin --post-processing-tdc crux-output/make-pin.pin
    INFO: Started Tue Dec 12 15:47:13 2017
    INFO: Hyperparameters: selectionFdr=0.01, Cpos=0, Cneg=0, maxNiter=10
    INFO: Reading tab-delimited input from datafile crux-output/make-pin.pin
    INFO: Features:
    INFO: lnrSp deltLCn deltCn XCorr Sp IonFrac PepLen Charge1 Charge2 Charge3 enzN enzC enzInt lnNumDSP dM absdM 
    INFO: Found 1380 PSMs
    INFO: Separate target and decoy search inputs detected, using target-decoy competition on Percolator scores.
    INFO: Train/test set contains 690 positives and 690 negatives, size ratio=1 and pi0=1
    INFO: Selecting Cpos by cross-validation.
    INFO: Selecting Cneg by cross-validation.
    INFO: Split 1:	Selected feature 2 as initial search direction. Could separate 44 training set positives in that direction.
    INFO: Split 2:	Selected feature 5 as initial search direction. Could separate 41 training set positives in that direction.
    INFO: Split 3:	Selected feature 2 as initial search direction. Could separate 32 training set positives in that direction.
    INFO: Found 39 test set positives with q<0.1 in initial direction
    INFO: Reading in data and feature calculation took 0.077138 cpu seconds or 0 seconds wall clock time.
    INFO: ---Training with Cpos selected by cross validation, Cneg selected by cross validation, fdr=0.01
    INFO: Iteration 1:	Estimated 71 PSMs with q<0.1
    INFO: Iteration 2:	Estimated 73 PSMs with q<0.1
    INFO: Iteration 3:	Estimated 73 PSMs with q<0.1
    INFO: Iteration 4:	Estimated 73 PSMs with q<0.1
    INFO: Iteration 5:	Estimated 73 PSMs with q<0.1
    INFO: Iteration 6:	Estimated 73 PSMs with q<0.1
    INFO: Iteration 7:	Estimated 73 PSMs with q<0.1
    INFO: Iteration 8:	Estimated 73 PSMs with q<0.1
    INFO: Iteration 9:	Estimated 73 PSMs with q<0.1
    INFO: Iteration 10:	Estimated 73 PSMs with q<0.1
    INFO: Learned normalized SVM weights for the 3 cross-validation splits:
    INFO:  Split1	 Split2	 Split3	FeatureName
    INFO:  0.3140	 0.0590	 0.2375	lnrSp
    INFO:  0.1862	-0.1121	 0.1878	deltLCn
    INFO:  0.1340	 0.0416	 0.3738	deltCn
    INFO:  0.0642	 0.3209	 0.6567	XCorr
    INFO:  0.7780	 0.3290	 1.2113	Sp
    INFO:  0.2765	 0.0558	-0.4342	IonFrac
    INFO:  0.3376	 0.0918	 0.3700	PepLen
    INFO:  0.0230	 0.0536	 0.1603	Charge1
    INFO:  0.1090	 0.0586	 0.3298	Charge2
    INFO: -0.1604	-0.1274	-0.5759	Charge3
    INFO:  0.0000	 0.0000	 0.0000	enzN
    INFO:  0.0000	 0.0000	 0.0000	enzC
    INFO: -0.0195	-0.0015	 0.0231	enzInt
    INFO:  0.1450	 0.0298	 0.6177	lnNumDSP
    INFO: -0.1984	-0.0750	-0.2956	dM
    INFO: -0.1965	-0.0327	-0.1631	absdM
    INFO: -1.5185	-1.1520	-2.0882	m0
    INFO: Found 54 test set PSMs with q<0.1.
    INFO: Selected best-scoring PSM per scan+expMass (target-decoy competition): 104 target PSMs and 62 decoy PSMs.
    INFO: Tossing out "redundant" PSMs keeping only the best scoring PSM for each unique peptide.
    INFO: Calculating q values.
    INFO: Final list yields 0 target peptides with q<0.1.
    INFO: Calculating posterior error probabilities (PEPs).
    INFO: Processing took 3.472 cpu seconds or 3 seconds wall clock time.
    INFO: Elapsed time: 3.35 s
    INFO: Finished crux percolator.
    INFO: Return Code:0
    		

    The crux-output directory will now contain eight new files:

    1. percolator.target.psms.txt – a list of peptide-spectrum matches (PSMs), ranked by quality,
    2. percolator.target.peptides.txt – a list of peptides, ranked by quality,
    3. percolator.decoy.psms.txt – a ranked list of decoy PSMs,
    4. percolator.decoy.peptides.txt – a ranked list of decoy peptides,
    5. percolator.pout.xml – a single XML output file containing all of the Percolator results,
    6. make-pin.pin.xml: an intermediate XML format file that is used by Percolator.
    7. percolator.params.txt – parameter file, and
    8. percolator.log.txt – log file.

    As before, you might want to sort the Percolator output files, this time by the "percolator score" column:

  7. $ crux sort-by-column --column-type real --ascending T crux-output/percolator.target.psms.txt "percolator score" > crux-output/percolator.target.psms.sort.txt

    The beginning of the resulting percolator.target.psms.sort.txt file will look like this:

    filefile_idxscanchargespectrum precursor m/zspectrum neutral masspeptide masspercolator scorepercolator rankpercolator q-valuetotal matches/spectrumsequencecleavage typeprotein idflanking aa
    crux-output/tide-search.target.txt111831031.94073092.80003095.23668.6190662102ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009CKD
    crux-output/tide-search.target.txt1262692.37731382.74001382.44677.0481362206TASEFDSAIAQDKtrypsin-full-digestYLR043CKL
    crux-output/tide-search.target.txt1532745.74731489.48001489.73186.7469611303NFLETVELQVGLKtrypsin-full-digestYGL135WRN
    crux-output/tide-search.target.txt1622745.85721489.70001489.73186.6489625403NFLETVELQVGLKtrypsin-full-digestYGL135WRN
    crux-output/tide-search.target.txt11462692.67721383.34001382.44676.6027999505TASEFDSAIAQDKtrypsin-full-digestYLR043CKL
    crux-output/tide-search.target.txt11312745.84731489.68011489.73186.5627294603NFLETVELQVGLKtrypsin-full-digestYGL135WRN
    crux-output/tide-search.target.txt1502651.28731300.56011301.41606.37577637010LDVDELGDVAQKtrypsin-full-digestYLR043CKN
    crux-output/tide-search.target.txt14231032.26063093.76003095.23665.8047724802ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009CKD
    crux-output/tide-search.target.txt19031032.00063092.98003095.23665.4775882902ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009CKD
    crux-output/tide-search.target.txt111131033.02063096.04003095.23665.43644191002ELESAAYDHAEPVQPEDAPQDIANDELKtrypsin-full-digestYGL009CKD

    In this output, the PSMs are ranked by "percolator score," with higher scores indicating a higher quality match. The associated statistical confidence estimate is reported as a "percolator q-value," interpreted as the minimal false discovery rate threshold at which this match is deemed significant. In the list above, all of the matches have q-values of 0, meaning that they are highly significant. The meanings of the remaining columns are described here. Note that when you run Percolator on your own computer, the results may be somewhat different than the ones reported here. This is because Percolator involves randomly subdividing the data in a cross-validation scheme (described in detail here.)