Home / Directory / Index and benchmark

Index and benchmark: a buyer's guide

What we list in this category, how the field splits between long-established products and newer ones, and the questions worth pressing on before you commit.

32
products
25
vendors
16
established
2
newer

Every entry below was recorded from a page we opened, and each carries the source we read. Where we could not establish something it is left blank rather than guessed. Read how this site works if you want the detail.

Long-established products

Recorded as established: in production for many years, usually with a deep reference base and broad coverage. Those are real advantages, and so is the fact that a fifteen-year horizon is easier to argue for a supplier that has already lasted one.

Newer products

Recorded as modern or emerging. Newer suppliers often offer capability and pricing the incumbents do not, and they carry risks the incumbents do not: fewer references at your size, less operating history, and concentration if they are small. Both belong in the same comparison, on the same fields.

What to ask

These come from the same question bank our request-for-proposal process uses, so a guide and an RFP never drift apart. The four sections below carry the most weight in that bank.

Functional fit

Whether the system does the job for YOUR instruments, markets and volumes — not for the demonstration portfolio. The questions separate what runs in production today from what is on the roadmap, because that distinction disappears in a sales meeting and reappears during implementation. It also asks what still has to be typed in by hand, which every system has and few volunteer.

Security and resilience

Whether the vendor's security is independently attested or merely asserted. The questions ask for certifications you can read, penetration tests you can see the summary of, and recovery objectives that are written down. Incident history is asked directly, because a vendor that has handled a breach well is often a safer choice than one that has never been tested.

Commercial terms

Total cost across the whole term, and what happens if you leave. Year one is asked separately from years two to five because year one hides where the money goes. The section asks explicitly what is NOT included, since the gap between the quote and the invoice lives there, and it asks about exit terms while you still have the leverage to negotiate them.

Artificial intelligence

If a model touches your data, on what terms. This section exists because the answers differ enormously between vendors and are rarely offered unprompted. It asks whether your data trains models serving other clients, whether you can opt out and keep full function, whether a human reviews output before it affects a book or a client report, and who is liable when a model is wrong. Skip it only if you genuinely do not care.

You can send all 150 of these questions to the vendors you choose, and get every answer back side by side. Run an RFP — it is free to you and free to them.

See all 32 products in Index and benchmark · Glossary