Add benchmark dataset inventory

[swaraj-neu]

Reference note from going through the benchmark folder on Explorer. Closes #2.

Mostly a working note so this doesn't live only in my head. No close review needed, just flag anything wrong.

Two things worth a look:

Format coverage. With TMT out of scope, there's data for five label-free formats: MaxQuant, FragPipe, DIA-NN, Spectronaut, MetaMorpheus. The plan assumed around seven for the results table. We can source more data, count instrument variants separately, or report five and note the limit. Flagging early since it affects benchmarking.

Pre-processing scope, and I'd like an answer on this one. The Solivais script does a lot before calling the converter: dropping semicolon-packed protein groups, dropping decoys, joining QuantifiedProteins.tsv to get the organism, then tagging protein names with | ECOLI or | HUMAN. The species labels the FDR calculation needs aren't in the raw file.

My reading: that's dataset curation, not format mapping, so it sits outside the LLM and the benchmark applies the same pre-processing to both paths. Does that match how you'd scope it?

Also: should this go to main, or do you want a devel branch here like the other packages?

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: f175ce31-abe2-487e-8630-3fb00e5c69b6

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}