Point two TPC-C load generators at the same database, same data, same concurrency, same duration — and they report noticeably different throughput. Stroppy (ours) said ~3.6k transactions/sec; go-tpc said ~6.1k. We wanted to know where that gap comes from before trusting either number.
This is the trail of guesses and tests we went through. We reached an answer we believe, and found two real bugs in Stroppy along the way.
In performance-critical systems, ideally we want to validate every source code change and make sure no performance regressions are introduced. Functional tests handle correctness, but performance is trickier — how do you know a change won't slow things down?
We worked with the OrioleDB maintainers to build a CI pipeline that runs TPC-C benchmarks on every PR and flags regressions before merge.
OrioleDB is a PostgreSQL extension that combines the advantages of both on-disk and in-memory storage engines, using PostgreSQL's pluggable storage to improve performance and reduce costs.