after

godotenv -f .env go run ./src/server/main.go

2022-05-26T19:50:41Z FTL src/server/http/staticfilehandler.go:29 >  error="open build/index.html: no such file or directory"

• Be able to query Amazon Redshift from Dekart

Please add 👍 you want this feature sooner.

• auto complete for BigQuery, Snowflake, Athena
• auto complete with data schema

It would be nice to have the filters available in the view mode of the report

• load any file or query result
• transform/filter or do spatial transformation on client using SQLLite/SpatiaLite

Is this a new bug in Dekart?

• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Potentially reproduces when query returns empty result

Example query

select * from DEMO.PUBLIC.PLANET_INTERNAL AS p
where
    type = 'way'
    and (lon between 8.814205010251918 and 14.51758675055565)
    and (lat between 52.031742766976095 and 53.85100739159098)
limit 10

Expected Behavior

Query shows as completed

Steps To Reproduce

No response

Relevant log output

No response

Environment

- OS:
- Dekart version:
- Browser:

Which database adapter are you using with dekart?

Snowflake

Additional Context

No response

Is this your first time submitting a feature request?

• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing Dekart functionality, rather than a Big Idea better suited to a discussion

Describe the feature

When submitting queries to be ran by AWS Athena, one optional parameter is the workgroup that will run the query. Specifying the workgroup can have a material impact on a query, since it affects things like IAM permissions, the Athena engine version being used, etc.

I would propose that a new environment variable like DEKART_ATHENA_WORKGROUP be created and used when starting a new query execution, and alternatively use the default value of primary when not set, since that's what's implicitly being used now.

Describe alternatives you've considered

No response

Who will this benefit?

Any Dekart users that utilize AWS Athena for their analytics engine.

Are you interested in contributing this feature?

Yes, I can probably add it. Wanted to make sure this is something that wouldn't be rejected for whatever reason before starting it.

Anything else?

Wanted to add a thank you for all of the work that's previously been done, Dekart is a great project!

• Add/Remove users via web interface
• Require Auth with login and password

Please add 👍 you want this feature sooner.

Example implementation https://github.com/fujiwara/go-amzn-oidc

with zipcode_geom as (
    SELECT
        ST_GEOGFROMTEXT(zipcode_geom) as geom,
        bqcarto.h3.ST_ASH3_POLYFILL(ST_GEOGFROMTEXT(zipcode_geom), 10) as h3,
        zipcode
    FROM `bigquery-public-data.utility_us.zipcode_area`
    WHERE city = 'Seattle city' and zipcode='98105'
)

select
 ARRAY_LENGTH(h3),
 *
FROM zipcode_geom

cell is empty

• support Postgres as data source

Please add 👍 you want this feature sooner.

Details on this issue in deck.gl visgl/deck.gl#6133

Let users share and reuse same query between different reports

• Click View source to see SQL when query is read only
• Fork Button to create same report

• Instance of Dekart deployed and available for everyone to try
• BugQuery public datasets are available to query
• Every user can save and share their own reports

Please add 👍 you want this feature sooner.

Makefile target "docker" Dockerfile requires .npmrc:
https://github.com/dekart-xyz/dekart/blob/main/Makefile#L23
https://github.com/dekart-xyz/dekart/blob/main/Dockerfile#L5

However, this file is excluded by .gitignore:
https://github.com/dekart-xyz/dekart/blob/main/.gitignore#L40

Proposed solution:

• add a file "npmrc"
• modify Dockerfile to COPY the file to .npmrc

Here's the error from make docker:

npm ERR! code E401
npm ERR! Incorrect or missing password.
npm ERR! If you were trying to login, change your password, create an
npm ERR! authentication token or enable two-factor authentication then
npm ERR! that means you likely typed your password in incorrectly.
npm ERR! Please try again, or recover your password at:
npm ERR!     https://www.npmjs.com/forgot
npm ERR! 
npm ERR! If you were doing some other operation then your saved credentials are
npm ERR! probably out of date. To correct this please try logging in again with:
npm ERR!     npm login
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2022-02-09T18_01_07_946Z-debug.log

First of all, thanks everyone for the excellent work! 👍

I wonder if we can have a UI enhancement on the opening mode.

When I open a report from the list, it enters Edit Mode first. However, there are many times that I just want to show/view the report.
Is it better to open the report in the Present Mode first? If needed, a user can click on the Edit button to start editing.

Maybe it is better to have a configuration for the open mode.

Please let me know if there is a better workflow. Thanks.

It would be helpful to have the ability to rename and re-order the queries.

• Support Snowflake as data source for Dekart

Please add 👍 you want this feature sooner.

prototype

Hi there!

Apparently there is a major bug within the deck.gl version so that it doesn't work with HEXBINS or GRID mode.

Other errors examples:
An error in deck.gl: Cannot read property 'sortedBins' of undefined in d0urgfs-grid-cell
An error in deck.gl: Cannot read property '12' of undefined in d0urgfs
An error in deck.gl: Cannot read property 'sortedBins' of undefined in v8elplw-hexagon-cell
..

Nonetheless, thank you very much for this wonderful work 👍

Hi, is there a way to run Dekart (via docker) without postgresql? I don't really need to save any metadata, and I don't want to have to manage a postgres database in order to use Dekart.

Not passing in the postgres environmental variables gets me this error:

{"level":"fatal","error":"dial tcp 127.0.0.1:5432: connect: connection refused","time":1638744281,"message":"WithInstance"}

I suppose another solution would be to include postgres in the Dockerfile so that one could be available on localhost.

Thank you!

Uncaught TypeError: Class constructor Geometry cannot be invoked without 'new'
    at new ColumnGeometry

test-data.csv.zip

Is this a new bug in Dekart?

• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When you rerun dekart with a new query on the same query tab, it can result in infinite load time for data. On reload of the page, new data is shown, however.

Expected Behavior

The new data should be shown after a finite period of time on the screen (Downloading Map Data step should be finished on the front end. This is likely because the front end is not getting the status update from the backend in a correct manner).

Steps To Reproduce

Go to dekart instance (i.e. playground)

run :

SELECT
    primary_type,
    district,
    latitude,
    longitude,
    date
from `bigquery-public-data.chicago_crime.crime`
LIMIT 1500

Then run :

SELECT
    primary_type,
    district,
    latitude,
    longitude,
    date
from `bigquery-public-data.chicago_crime.crime`
LIMIT 1550

On the same query tab. It should get stuck on Downloading Map Data step.

Even on the playground, this issue can be reproduced.

However, if you run a query that has been saved before, this does not seem to happen.

Relevant log output

This is the error from the playground

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading '1')
    at e.value (row-data-container.js:71:23)
    at t.maybeToDate (data-utils.js:137:13)
    at i (kepler-table.js:321:37)
    at Array.map (<anonymous>)
    at t.getQuantileDomain (data-scale-utils.js:29:61)
    at e.value (kepler-table.js:332:16)
    at n.value (base-layer.js:1037:20)
    at base-layer.js:928:36
    at Array.forEach (<anonymous>)
    at n.value (base-layer.js:921:40)


A better output can be found by running it locally:

Error: Failed to add data to map: Cannot read properties of undefined (reading '1')
    at dataset.js:88:9
I @ message.js:26
(anonymous) @ dataset.js:87
await in (anonymous) (async)
(anonymous) @ index.js:16
(anonymous) @ redux.js:51
dispatch @ redux.js:691
(anonymous) @ report.js:125
(anonymous) @ report.js:119
(anonymous) @ index.js:16
(anonymous) @ redux.js:51
dispatch @ redux.js:691
(anonymous) @ report.js:34
onMessage @ grpc.js:64
(anonymous) @ grpc-web-client.umd.js:1
e.rawOnMessage @ grpc-web-client.umd.js:1
(anonymous) @ grpc-web-client.umd.js:1
e.onTransportChunk @ grpc-web-client.umd.js:1
(anonymous) @ grpc-web-client.umd.js:1
Promise.then (async)
e.pump @ grpc-web-client.umd.js:1
(anonymous) @ grpc-web-client.umd.js:1
Promise.then (async)
e.send @ grpc-web-client.umd.js:1
e.sendMessage @ grpc-web-client.umd.js:1
e.send @ grpc-web-client.umd.js:1
t.invoke @ grpc-web-client.umd.js:1
F @ grpc.js:56
a @ grpc.js:72
(anonymous) @ grpc-web-client.umd.js:1
e.rawOnEnd @ grpc-web-client.umd.js:1
e.onTransportEnd @ grpc-web-client.umd.js:1
(anonymous) @ grpc-web-client.umd.js:1
Promise.then (async)
e.pump @ grpc-web-client.umd.js:1
(anonymous) @ grpc-web-client.umd.js:1
Promise.then (async)
e.pump @ grpc-web-client.umd.js:1
(anonymous) @ grpc-web-client.umd.js:1

Environment

- Docker
- Playground

Which database adapter are you using with dekart?

BigQuery

Additional Context

No response

Small feature request.
I have multiple different queries, which use 3 different tables. It’s confusing to know which data set is which table/query if they are just called “Query 1”, “Query 2", “Query 3”.. would be cool to be able to change that.

Similar to https://data.apps.fao.org/ (Use Printing -> Custom Print)

• Support Presto as datasource for Dekart

Please add 👍 you want this feature sooner.

Reproduce: Write a query that uses SQL scripting, then execute

Expected behaviour: Dekart retrieves the results of the query

Actual behaviour: Dekart returns a "destination table is nil" error, as SQL scripts do not have a destination table in the job config.

There are other ways to retrieve a query result by job ID that don't require a destination table in the JobConfig which would allow results retrieval for both regular SQL and SQL scripting jobs.

Updated only on page reload

Our team set up Dekart.xyz on our own GCP using the instructions provided in https://dekart.xyz/docs/self-hosting/app-engine/, using Google App Engine. However, an issue we are running into is - whenever we run an invalid query, no error message comes up (either in Dekart or in the docker logs), and we have to reset the docker image every time, and only then do we see the error message. Is there better error handling available that doesn't cause Dekart to crash every time we submit a query with syntax or permissions errors (e.g. a column doesn't exist, or a user doesn't have permission to a table). The crash also prevents users from returning to the home 'reports' screen. It goes completely blank.

the map crashed when tried to see the report "Fork of COVID vs Population Density in ".
Couldn't reproduce anymore.

Configuration:

Browser Google Chrome Version 96.0.4664.110 (Official Build) (x86_64)
MacBook Pro with Intell Core i7 / Radeon Pro 555 2 GB / 16Gb

Steps:

• Open the report https://play.dekart.xyz/reports/348a60af-24ca-4c46-a5cd-9555c1c6702f/source
• Tried to execute the query for RU with ST_SIMPLIFY(geometry, 6)
• Here I got an error googleapi: Error 400: UDF worker timed out during execution., timeout
• Tried to execute the query for RU with ST_SIMPLIFY(geometry, 100)
• Map crashed

Output

console log

TypeError: Cannot read properties of undefined (reading 'map')
    at data-table-modal.js:96
    at index.js:67
    at d (defaultMemoize.js:123)
    at index.js:81
    at r.d [as columns] (defaultMemoize.js:123)
    at r.value (data-table-modal.js:176)
    at qa (react-dom.production.min.js:187)
    at Ga (react-dom.production.min.js:186)
    at $s (react-dom.production.min.js:269)
    at Pl (react-dom.production.min.js:250)
w @ message.js:25
onError @ ReportPage.js:198
componentDidCatch @ ReportPage.js:168
i.componentDidCatch.n.callback @ react-dom.production.min.js:217
gi @ react-dom.production.min.js:131
vs @ react-dom.production.min.js:220
Rl @ react-dom.production.min.js:259
t.unstable_runWithPriority @ scheduler.production.min.js:18
Xo @ react-dom.production.min.js:122
kl @ react-dom.production.min.js:252
bl @ react-dom.production.min.js:243
(anonymous) @ react-dom.production.min.js:123
t.unstable_runWithPriority @ scheduler.production.min.js:18
Xo @ react-dom.production.min.js:122
Ko @ react-dom.production.min.js:123
Yo @ react-dom.production.min.js:122
je @ react-dom.production.min.js:292
Kt @ react-dom.production.min.js:73

What is the recommended setup if someone wants to create H3 indexes based on athena data in order to do the visualizations on Dekart?

Error

TypeError: Cannot read property 'find' of undefined
    at coordHasLength4 (0.chunk.js:369482)
    at isTripGeoJsonField (0.chunk.js:369552)
    at 0.chunk.js:369395
    at Array.filter (<anonymous>)
    at Function.findDefaultLayerProps (0.chunk.js:369394)
    at 0.chunk.js:389415
    at Array.reduce (<anonymous>)
    at findDefaultLayer (0.chunk.js:389414)
    at 0.chunk.js:380215
    at Array.reduce (<anonymous>)
    at addDefaultLayers (0.chunk.js:380214)
    at updateVisDataUpdater (0.chunk.js:379846)
    at 0.chunk.js:374048
    at 0.chunk.js:374079
    at 0.chunk.js:374065
    at Array.reduce (<anonymous>)
    at 0.chunk.js:374064
    at Object.addDataToMapUpdater [as @@kepler.gl/ADD_DATA_TO_MAP] (0.chunk.js:373931)
    at 0.chunk.js:374219
    at 0.chunk.js:376271
    at Array.forEach (<anonymous>)
    at 0.chunk.js:376270
    at keplerGl (main.chunk.js:5663)
    at combination (0.chunk.js:602310)
    at dispatch (0.chunk.js:602063)
    at 0.chunk.js:601823
    at 0.chunk.js:554160
    at dispatch (0.chunk.js:602490)
    at main.chunk.js:4250

Example query

SELECT
    ST_ASGEOJSON(ST_SIMPLIFY(ST_GEOGFROMTEXT(zipcode_geom), 250)) as zipcode_geom,
    -- *
FROM `bigquery-public-data.utility_us.zipcode_area`
WHERE
    area_land_meters > 0
    and Rand() < 5/100.0
limit 100

Is this a new bug in Dekart?

• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

No response

Expected Behavior

No response

Steps To Reproduce

No response

Relevant log output

Logs


2023-03-16T06:25:10Z DBG src/server/dekart/query.go:192 > RunQuery query_id=5216c369-4515-4d73-9c0d-8643ef1d680e
2023-03-16T06:25:11Z DBG src/server/storage/storage.go:152 > object is successfully uploaded DEKART_CLOUD_STORAGE_BUCKET=dekart-dev name=ca0c3f339d04657d2e9b9b6fdec9a2d7145f53e6.sql
2023-03-16T06:25:11Z DBG src/server/dekart/query.go:240 > Job created jobID=e7fcc322-a264-409e-9f59-4249eace74a6
2023-03-16T06:25:11Z DBG src/server/report/report.go:84 > Ping reportID=5a5821fe-cd23-4b27-ae4a-ffb525d01e70
2023-03-16T06:25:11Z DBG src/server/report/report.go:99 > Update subscriber rid=5a5821fe-cd23-4b27-ae4a-ffb525d01e70 sequence=40
2023-03-16T06:25:11Z DBG src/server/dekart/stream.go:102 > Sending report message reportID=5a5821fe-cd23-4b27-ae4a-ffb525d01e70 sequence=40
2023-03-16T06:25:43Z DBG src/server/dekart/stream.go:105 > GetReportStream ctx done reportID=dd4b2bc1-43ac-4116-b832-d9a962ef1129
2023-03-16T06:25:57Z ERR src/server/snowflakejob/snowflakejob.go:104 >  error="279301: 01aafa81-3201-abee-0001-c28e0002e0ba: : status from server: [RUNNING]" queryID=5216c369-4515-4d73-9c0d-8643ef1d680e reportID=5a5821fe-cd23-4b27-ae4a-ffb525d01e70
2023-03-16T06:25:57Z DBG src/server/job/job.go:148 > Removing job from store queryId=5216c369-4515-4d73-9c0d-8643ef1d680e
2023-03-16T06:25:57Z DBG src/server/job/job.go:150 > lock acquired jobs=1 queryId=5216c369-4515-4d73-9c0d-8643ef1d680e
ERRO[1331]restful.go:283 gosnowflake.postRestfulQueryHelper failed to get response. err: context canceled 
2023-03-16T06:25:57Z DBG src/server/report/report.go:84 > Ping reportID=5a5821fe-cd23-4b27-ae4a-ffb525d01e70
2023-03-16T06:25:57Z DBG src/server/report/report.go:99 > Update subscriber rid=5a5821fe-cd23-4b27-ae4a-ffb525d01e70 sequence=41
2023-03-16T06:25:57Z DBG src/server/dekart/stream.go:102 > Sending report message reportID=5a5821fe-cd23-4b27-ae4a-ffb525d01e70 sequence=41
ERRO[1331]connection.go:356 gosnowflake.(*snowflakeConn).queryContextInternal error: context canceled                      
2023-03-16T06:25:57Z ERR src/server/snowflakejob/snowflakejob.go:129 > Error starting query execution error="context canceled" queryID=5216c369-4515-4d73-9c0d-8643ef1d680e reportID=5a5821fe-cd23-4b27-ae4a-ffb525d01e70

### Environment

```markdown
- Dekart version: 0.12.0
- Browser: Google Chrome

Which database adapter are you using with dekart?

Snowflake

Additional Context

Hi!

I have a use case where I want to be able to click a geography, and copy an associated ID (which is also shown in the tooltip). The problem is that the "data view" that pops up when you click the point appears below the view port. This means that it is both:

1. Hard to notice that it even exists
2. You have to scroll down to it, which is tiresome.

If you are unsure of what I mean, see the screenshot.

Is there a way to fix this? I would be willing to attempt to create a PR if you agree that this is something that could be improved and we agree on a reasonable new behaviour when one clicks a geography.

Screenshot:

• Apparently when on of the queries returns empty result
• "Downloading Map Data" keeps rolling
  https://play.dekart.xyz/reports/38874916-3919-4636-abac-904e4cbaacf6/source