Updated publication reference for PubMed record(s): 27701404. The consistent and accurate quantification of proteins is a challenging task for mass spectrometry (MS)-based proteomics. SWATH-MS uses data-independent acquisition (DIA) for label-free quantification. Here we evaluated five software tools for processing SWATH-MS data: OpenSWATH, SWATH2.0, Skyline, Spectronaut, DIA-Umpire, in collaboration with the respective developers to ensure an optimal use of each tool. We analyzed data from hybrid proteome samples of defined quantitative composition acquired on two different MS instruments applying different SWATH isolation windows setups. Using the resulting high-complexity datasets we benchmarked precision and accuracy of quantification and evaluated identification performance, robustness and specificity of each software tool. To consistently evaluate the high complexity datasets, we developed the LFQbench R-package. LFQbench results enabled developers to improve their software tools, thereby underlining the value of the reference datasets for software development and benchmarking. All tools provided highly convergent identification and reliable quantification performance, underscoring their robustness for label-free quantitative proteomics.