Designing a warehouse POC the vendor can't win by default

Every warehouse vendor offers to run your proof of concept, and every vendor-run POC concludes the same way: impressively. The demo data is shaped for the engine, the queries flatter the optimizer, and the discount expires Friday. A POC you design yourself costs two weeks and produces something rarer than a benchmark: a decision you can defend a year later.

Bring your ugliest workloads, not your cleanest. Pick five real queries: the dashboard that times out, the join across your two widest tables, the transformation job that dominates the bill, one concurrency burst, one ingest at production volume. The platform field differentiates on exactly the workloads demos avoid.

Write pass criteria before the first query runs. Latency thresholds, cost per workload at list price, an operability checklist (who handles maintenance, how access control integrates with your identity provider). Criteria written after the runs get negotiated to fit whichever vendor bought lunch.

Price the list price. POC credits and first-year discounts are marketing, not economics. Model year-two cost from the metering data your POC generated, at published rates, with your growth estimate. The renewal-quarter version of your team will thank you.

Log the friction diary. Every "we had to open a ticket", every setting that required a solutions architect. Two weeks of friction notes predict three years of operations better than any latency table.

Run the candidates concurrently, not sequentially. Sequential POCs let the team's queries improve between candidates, which hands the later vendor a tuned workload the earlier one never saw; we have watched that ordering effect flip a decision. Concurrent runs cost one more busy fortnight and produce the only comparison that is actually a comparison. And keep the vendors out of the driving seat during the test itself: solutions architects are welcome at setup and at the readout, but a POC where the vendor writes the queries is a demo with your logo on it.

The output is a one-page scorecard per candidate against your criteria. If two candidates pass, pick on exit cost and team familiarity; past the pass line, benchmark deltas are marketing again, just pointed the other way.