Four open-source software suites for the analysis of oligonucleotide data from tandem MS have been reported. These tools are all based on matching theoretical spectral libraries generated by silico digestion. While some software publications have included simple comparisons with previously published tools, a discussion of oligonucleotide identification within complex backgrounds is lacking. In this study, we acquired multiple complex oligonucleotide datasets using an Orbitrap instrument to evaluate various search engines (Ariadne, RNAModMapper, NASE and Pytheas) and assess their performance in terms of accuracy, false discovery rate (FDR), performance at certain FDR, consensus of oligonucleotide identification, consistency of outputs in chemical measurements, and unknown modification discovery.In this study, we prepared two sets of samples, which involve three RNase T1 digested and three RNase A digested E. coli total RNA. Then, digests were analyzed by LC-MS/MS.