For database vendors
Vendors
Get your database measured on the same terms as every other engine, and tell us when a result is wrong.
We publish speed and protocol compatibility for every engine we can run. Adding yours and contesting a result both go through public issues, so anyone can follow what was asked and what changed.
- Same tests for every engine
- Every number opens its source
- Corrections published with dates
Add your database
Open an issue in Stroppy’s repository with what we need to run your database the way we run the others. The form comes filled in.
- Databasename and version
- Protocol it claimse.g. PostgreSQL 17, MySQL 8.4, MongoDB 8.0
- How to run ita container image, or install steps on Linux
- Settings for one 32 vCPU · 128 GB nodewhat you would set
- Contactwho answers our questions
Run the tests on your build
The workload behind the rating and the compatibility checks are open source and need no account. Run them on a branch or a nightly before a release.
- Speed:
tpcc/tx, the rating’s workload, against your endpoint. - Compatibility: your endpoint in a small YAML file, checked against the original it claims.
Terminal
# speed
stroppy run tpcc/tx -d pg \
-D url=postgres://user:password@host:5432/db
# compatibility, endpoint in mine/my-engine.yaml
bin/conformity golden -t postgres-18
bin/conformity run -t my-engine
How we keep it fair
Every engine gets the same tests, and every result says exactly what was measured.
- Speed: the same TPC-C on one 32 vCPU · 128 GB node; each engine deployed the same way and tuned for that machine.
- The rating compares runs of one workload on one machine class only.
- Compatibility: an engine is checked against the original it claims; the expected answers come from the original, never written by hand.
- A crash counts only when the server is seen to go down or restart.
- Every result records the image it ran and when it was measured.
- A rating rowopens the run’s report in Stroppy Cloud: throughput, latency, errors
- A compatibility cellopens its checks: what was sent, what the original answered, what the engine answered
Report an error
Think a result is wrong? Open an issue with the result, what is wrong and how to see it. We rerun it; if we were wrong, the result changes and the correction is listed below.
- Resulta link to the rating row, the run report or the conformity finding
- What is wrongwhat we published and what it should be
- How to see itthe commands, settings or version that show it
Corrections
None yet. Each correction will be listed here with its date, what changed and why.