memsql / benchmarks-tpc Goto Github PK

Your benchmark blog compares against published TPC-DS results using v2.10.1 of the benchmark kit. I'm not sure what version these queries are from but a quick diff shows at least 15 queries that do not match what's in v2.10/v2.11. This excludes any permitted MQM changes or variable substitution.

Quick diff list (not necessarily all-inclusive):

1. query4 (missing columns in select and order by)
2. query11 (missing columns in select, group by, and order by)
3. query14 (missing 2nd query)
4. query17 (catalog_sales_quantitystdev)
5. query22 (missing warehouse table and join)
6. query23 (missing 2nd query)
7. query24 (missing 2nd query)
8. query39 (missing 2nd query)
9. query49 (order by)
10. query56 (order by)
11. query62 (missing columns in select)
12. query72 (duplicate predicates)
13. query75 (order by)
14. query77 (missing group by)
15. query98 (missing column in select)

I've compared to this file from v2.11.0 w/o any mods (which has no template diffs from v2.10.1)
https://gist.github.com/gregrahn/1b08898606f181120a8e75932fefd9a4