Comments (8)
Wanted to get some movement here:
It also tried to look for ephemeral models
What is the exact output? With a verbose flag, what it actually says is something like "Skipping ephemeral model IMD_MONDAY_SCHEDULING_HISTORY not manifested in database". It doesn't mean its looking for ephemeral, just being very verbose.
With the folder parser, dbt_path pulls in all files since it cannot resolve schema except for in source files and even then only if the source file is not using jinja for its schema:
key.
The manifest parser on the other hand is able to use schema to filter the subset of models early.
So the no-op you are experiencing almost has to come from the dbt_schema or dbt_database arguments not matching up to what's in the manifest.json
Very likely that at least. We need an answer to that before we can rule this an issue.
Best way might be to control-F in a manifest artifact and inspect the schema field as well as the database field in pretty much any model
from dbt-metabase.
Can you try adding --verbose
to the command and seeing what happens?
I would only imagine the command terminating if it found no objects to loop through matching your supplied dbt_prod
schema/database. We can tell more with the flag though.
from dbt-metabase.
Thanks for the answer! Indeed, it didn't find the models in dbt_prod
. Surprisingly:
- It also tried to look for ephemeral models
- If I run the same command but with
--dbt_path
instead of--dbt_manifest_path
, it works fine.
from dbt-metabase.
Thank you Alexander for your assistance! Indeed, the problem appears to be with the schema names. FYI, I am using BigQuery as a data store. It seems like in the build_metadata_lookups()
method falls back to using public
, even though the schema is defined in dbt. I was able get the synchronization to work by fixing schema_name
to 'PUBLIC' in export_model()
. The resolution of foreign keys still doesn't work.
As a next step I will check what Metabase is returning to see if maybe the schema isn't being identified correctly.
from dbt-metabase.
I can confirm that metabase is returning null
for table schemas, but in dbt the schema is defined as dbt_prod
from dbt-metabase.
Ahh I think its coming back to me @mmmatthew,
Doesn't metabase api return dataset
as a key in the json response if using bigquery instead of schema
?
Would it be possible for you to confirm the network response from hitting this endpoint?
https://your_metabase_host/api/table/{id}
where id is the id of a table
you can find table ids here pretty easily
https://your_metabase_host/reference/databases/2/tables
where 2 is the id of whatever dataset has the table
If this is the case, it might be an easy fix.
from dbt-metabase.
We're getting closer! I found both the project_id
(corresponds to dbt_database
) and dataset_id
(corresponds to dbt_schema
) in the metabase responses, under ./details
when hitting the api/database/{id}/metadata
endpoint or under ./db/details
when hitting the api/table/{id}
endpoint.
{
...
"details": {
"use-service-account": null,
"dataset-id": "dbt_prod",
"service-account-json": "**MetabasePass**",
"use-jvm-timezone": false,
"include-user-id-and-hash": false,
"ssl": true,
"project-id": "bigquery_project",
"let-user-control-scheduling": true
},
...
}
`
from dbt-metabase.
I would be happy to make a PR for this, btw, once we agree on how to fix it.
from dbt-metabase.
Related Issues (20)
- Shouldn't we sync the documentation even if tables schemas have mismatches between dbt manifest and metabase? HOT 3
- Making exposures output idempotent HOT 1
- Unable to align models between dbt target models and Metabase HOT 4
- Upgrade to v0.9.11 from v0.9.10 introduces timeout exception HOT 5
- use human friendly exposure names
- dbt docs not syncing across to Metabase - only Jinja templates are showing HOT 2
- Propagate dbt analyses to Metabase HOT 1
- Fix the docstring of DBTInterface because schema seems not ignored by the manifest parser HOT 1
- Configure API timeout for calls made to Metabase HOT 4
- Supporting metabase "Display values" for foreign keys (/api/field/XXX/dimension) HOT 1
- Accept True in the --metabase_verify option to default to the default CAs. HOT 2
- Requests failing due to "include_hidden" not being valid boolean string HOT 1
- Collection inclusion for exposures
- Exposures command fails. dbtmetabase/_exposures.py; Function: __extract_card_exposures(); dbt-metabase: 1.0.1
- --metabase_exclude_sources - Doesn't work after release HOT 1
- Add support for constraints HOT 1
- Add option for choosing continuation strategy for when models or fields are not synced HOT 1
- Add wildcard syntax to table filters HOT 1
- Exposure Extraction Error | In Case of Missing Cards in collection HOT 3
- Ability to manage additional table metadata via DTB documentation HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dbt-metabase.