mapbox / tilelive-decorator Goto Github PK

I think it's time we look at the requiredKeys / keepKeys and requiredKeysRedis / keepKeysRedis options and consider what effects we actually want to achieve by filtering and selecting data.

I'm mentally breaking down decoration processing into 3 stages:

• source - the tile's property data
• redis - the data we want to decorate onto the tile
• output - the tile's properties post decoration

We can think of our two levers as filtering (via required keys) and selecting (via keep keys) for each stage. Here's a layout of the levers in each stage and what I think they let us do:

source:

• required - allows us to filter out features that don't meet minimum criteria before we ever query redis. This has a potential for big performance savings when there's very mixed data in the tile.
• keep - ​allows us to specify which properties to keep on the tile before applying decoration data. Possibly useful.

redis:

• required - allows us to throw out records from decorate data that don't meet minimum requirements. Definitely useful.
• keep - allows us to simplify data to be decorated onto tiles. Could be a performance boost when we only want a couple of properties to be applied from a heavy dataset. Possibly useful.

output:

• required - allows us to specify that data must exist on the tile, regardless of where it came from earlier in the process.
• keep - allow us to specify exactly what we expect output properties on the tile to look like.

We should listen to these.

cc @willwhite

Features with the same properties within a tile should be joined to avoid the overhead of having distinct geometries. Ideally this is exposed as an option that is passed to the decorator.

For example: { linejoin: true, linejoinUniqueBy: [ 'speedClass', 'FRC' ] }

@mourner - Can you grab this?

cc @yhahn

All number values are encoded as floats. This has two consequences:

• integers are not efficiently packed using varint, leading to bulkier size
• bigger integers loose precision when packed as 32-bit floats

Instead integers should be encoded as either uint or sint depending on sign.

cc @aaronlidman