Query Time Lookups for Dimensions in Druid about superset HOT 9 CLOSED

Will this feature be implemented anytime soon?

from superset.

+1, Looking forward to this too.

from superset.

+1
We also need this feature

from superset.

@sid88in can you share your work in progress code ?

from superset.

@kkalyan I moved from Yahoo to Glassdoor while working on Caravel :) Was able to hack the code to implement query time lookups in Caravel. Basically you will need to modify the query function (where the actually query gets formulated). I modified it to use map function and it worked. I'll talk to Maxime to see what is his thought on this and we may want to engineer it properly to be used by everyone.

from superset.

I just added support for dimensionsSpec here #1545 which exposes ways to do specify mapping on a per-column basis:
http://druid.io/docs/latest/querying/dimensionspecs.html

To use it, you have to create a new calculated column on the datasource and paste your valid dimensionSpec expression in the text box.

You can define your mapping as a hard coded JSON object in our dimensionSpec, and it also looks like you can define "namespace" mappings that are stored on all the nodes on the Druid cluster and you can reference them in dimensionSpec.
http://druid.io/docs/latest/querying/lookups.html

Maybe a complementary feature on the Caravel would be for a way to manage your Druid Lookups in Caravel and have Caravel sync them with the Druid cluster. This is not on our current roadmap but we get to it at some point.

from superset.

Update - 2017-26-01

Tested it again today after adding various mappings and it works fine both for jdbc and static mapping. I think this issue should be closed.

Original

@mistercrunch I used the feature today, and it seems the columns is ignored when building the result data from the Druid query (I can see the lookup dimension working if I do the query by myself to the Druid API).

For instance :
I have users buying products, my metric is an aggregate of product price, or just the global count. Naturally I try to lookup a table of the following :

{
    "dimensions": [
        "user_id",
        {
            "replaceMissingValueWith": "N/A",
            "name": "product_name",
            "dimension": "product_id",
            "type": "lookup",
            "injective": false,
            "outputName": "product_name_lookup"
        }
    ],
    "aggregations": [
        {
            "type": "count",
            "name": "count"
        }
    ],
    "intervals": "2017-01-12T20:43:43+00:00/2017-01-19T20:43:43+00:00",
    "limitSpec": {
        "limit": 5000,
        "type": "default",
        "columns": [
            {
                "direction": "descending",
                "dimension": "count"
            }
        ]
    },
    "granularity": "all",
    "postAggregations": [],
    "queryType": "groupBy",
    "dataSource": "user-purchases"
}

Druid will return n of :

{
    "version": "v1",
    "timestamp": "2017-01-12T20:43:43.000Z",
    "event": {
      "count": 20,
      "user_id": "10",
      "product_name_lookup": "A Nice Vase"
    }
  },

But the 'product_name_lookup' column will not appear on reports in superset.

from superset.

I am happily using superset - and I have a calculated column on the datasource with a dimensionSpec expression that describes a query time lookup.

Grouping works well but filtering does not on this calculated column... Anyone succeeded in doing this? Maybe I need to update the query generator?

from superset.

This should be fixed via #4740

from superset.