trinodb / tpch Goto Github PK

Hello developers,
I was searching for java utility to generate TPCH data and found your code, Can you tell me how to use this as a API, I was looking for Readme file but i didn't found one.

Thanks and Regards

Prashant

Hi,

first of all: Great project. I'm currently looking into using the code for creating a distributed TPC-H data generator.
I'm pretty confident that your code is suitable for that. Most likely, I'll contribute some stuff back, if thats okay for you.

One question (that could lead to a first contribution from my side): Is there a particular reason why all classes implementing TpchEntity don't use the toLine() method also for the toString() method?

Its very handy when using a debugger and for "system out" debugging.

running this code:

 public void generate() throws Exception {
        double scale = 10; 

        System.out.println("Line Item:     " + Iterables.size(TpchTable.LINE_ITEM.createGenerator(scale, 1, 1)));
        System.out.println("Nation:        " + Iterables.size(TpchTable.NATION.createGenerator(scale, 1, 1)));
        System.out.println("Region:        " + Iterables.size(TpchTable.REGION.createGenerator(scale, 1, 1)));
        System.out.println("Part:          " + Iterables.size(TpchTable.PART.createGenerator(scale, 1, 1)));
        System.out.println("Customer:      " + Iterables.size(TpchTable.CUSTOMER.createGenerator(scale, 1, 1)));
        System.out.println("Orders:        " + Iterables.size(TpchTable.ORDERS.createGenerator(scale, 1, 1)));
        System.out.println("Part Supplier: " + Iterables.size(TpchTable.PART_SUPPLIER.createGenerator(scale, 1, 1)));
        System.out.println("Supplier:      " + Iterables.size(TpchTable.SUPPLIER.createGenerator(scale, 1, 1)));
}

with scale == 10 yields

Line Item:     59986052
Nation:        25
Region:        5
Part:          2000000
Customer:      1500000
Orders:        15000000
Part Supplier: 8000000
Supplier:      100000

scale == 1

Line Item:     6001215
Nation:        25
Region:        5
Part:          200000
Customer:      150000
Orders:        1500000
Part Supplier: 800000
Supplier:      10000

scale == 0.1

Line Item:     600572
Nation:        25
Region:        5
Part:          20000
Customer:      15000
Orders:        150000
Part Supplier: 80000
Supplier:      1000

While all the other counts match the spec perfectly, the Line Item count is off by a little bit (and for scale == 10 there are actually fewer lines than the spec requires).

The LineItemGenerator uses the OrderGenerator.SCALE_BASE constant (which is 1,500,000) and the PartSupplierGenerator uses the PartGenerator.SCALE_BASE (which is 200,000). According to http://www.tpc.org/tpc_documents_current_versions/pdf/tpc-h_v2.17.3.pdf, these should be 6,000,000 (for LineItem) and 800,000 (for PartSupplier). any reason why the SCALE_BASE values differ from the spec?

TPC-H specification uses prefix for each column name. For example nation columns are named:

NATION Table Layout
Column Name Datatype Requirements Comment
N_NATIONKEY identifier 25 nations are populated
N_NAME fixed text, size 25
N_REGIONKEY identifier Foreign Key to R_REGIONKEY
N_COMMENT variable text, size 152
Primary Key: N_NATIONKEY

airlift tpch defines nation columns as:

    NATION_KEY("nationkey", TpchColumnTypes.IDENTIFIER) {
        public long getIdentifier(Nation nation) {
            return nation.getNationKey();
        }
    },
    NAME("name", TpchColumnTypes.varchar(25L)) {
        public String getString(Nation nation) {
            return nation.getName();
        }
    },
    REGION_KEY("regionkey", TpchColumnTypes.IDENTIFIER) {
        public long getIdentifier(Nation nation) {
            return nation.getRegionKey();
        }
    },
    COMMENT("comment", TpchColumnTypes.varchar(152L)) {
        public String getString(Nation nation) {
            return nation.getComment();
        }
    };

See lacking n_ in column names.

This causes that TPC-H queries cannot be simply generated and then executed in Presto, but require all the column names to be modified.

Specification file: http://cs.fit.edu/~pbernhar/teaching/databases/tpch.pdf

As in the title, is it possible for this "tpch" to support "tpc-bih" in TPC-BiH: A Benchmark for Bitemporal
Databases? Bih benchmark adds columns related to temporal feature in SQL:2011 to origin tpch tables.

Was wondering why the decision was made to not use the column names that match the TPC-H specification? This means that all queries that match the TPC-H standard (and run everywhere else) need to be rewritten to work with this (which is a bummer). Thanks.

Not available LICENSE file in source directory structure
Please. Added license and copyright notice.
the fedora pakaging guideline is very strictly precise about this problem
https://fedoraproject.org/wiki/Packaging:LicensingGuidelines?rd=Packaging/LicensingGuidelines#License_Text
thanks
regards

To have nice implementation of statistics interface in Presto it would be nice that table statistics

• rows count
• data size for individual columns
  is clearly exposed by airlift-tpch.

E.g via methods in io.airlift.tpch.TpchTable.