Configuration ideas

One idea I have about running multiple configurations is to expand our regular config file to allow us to run some options that can multiply the configuration and serialize it.

Example of config:

data-source:
  type: SIMULATOR
  ... # skipping details here
loader:
  db-specific:
    # ...
    host: "some-ip-here"
    partition-index: true
    partitions: 1
    time-partition-index: false
    use-hypertable: true
    use-jsonb-tags: false
    # ... a lot more configs here
  runner:
    batch-size: 10000
    channel-capacity: "0"
    db-name: benchmark
    do-abort-on-exist: false
    do-create-db: true
    do-load: true
    flow-control: false
    hash-workers: true
    limit: 2016000000
    reporting-period: 30s
    seed: 135
    workers: 24

Now let's imagine we would like to test the following scenarios:

• 3 machines
• different set of workers
• different batch sizes

I could do the following changes, only transforming the configs into an array of values:

data-source:
  type: SIMULATOR
  ... # skipping details here
loader:
  db-specific:
    # ...
    host:
      - "some-ip-here"
      - "second-machine-ip-here"
      - "third-machine-ip-here"
    # ... a lot more configs here
  runner:
    batch-size: [1000, 5000, 10000]
    workers: [8, 16, 24]

And then, the tsbs_load could prepare the plan of configs and run sequentially in the target machines.

Collecting and storing results

From the storage perspective, I'd love if we set a TSBS server where people around the world could push different benchmarks to our server. Dumping different time-series related to benchmark, even tracking how CPU and IO behave while executing some specific benchmark task.

It can be the first step to have a tsbs website with all database benchmarks exposed. It can be a rich source to learn what technology works better in what scenario.

from tsbs.