Comments (7)
right on @darrenhaken - this would be a great change. Let us know if you're interested in contributing a change for the feature! I think the change will be self-contained within the BigQuery materialization code
from dbt-bigquery.
I'd be interested in working on it. Can you sign post where the change would need to go?
from dbt-bigquery.
Sure thing - I think we'll want to:
- Add a macro in plugins/bigquery/.../adapter.sql that runs something like
alter table {{table}} set options (labels=...)
- Call that macro in the incremental materialization, but only if the model is building incrementally. That code should go somewhere around here.
I can dig deeper into this if you'd benefit from any more direction, but those should be the general places to check out!
from dbt-bigquery.
I think the issue applies to snapshots in BigQuery.
from dbt-bigquery.
@drewbanin @jtcohen6 I've got a version of this working where the labels config replaces what is already on the table.
One difference to the other materialization options is the case when all the labels have been removed. What should the behaviour be in this case? Looking at the BigQuery documentation, you aren't able to remove all the labels with SQL, you have to have at least one for it to work. You can have at least one label but removing them all can't be done, only with Python in this case.
Doing it within SQL, I see two options.
- Update the labels with a blank i.e. no_labels = true. This would remove all the previous labels but would require a dummy label to be put back in it's place which would stay until a
--full-refresh
is triggered.
- Not updating the labels at all, leaving them all intact. This is my least favourite of the two since it wouldn't be possible to ever remove all the labels (until of course a
--full-refresh
is triggered). Despite this being my least favourite, it wouldn't be a regression on the current capabilities.
If done in Python, I think it would be a case of getting fetching all the labels in the table and then subsequently setting them all to None
like this. If this was the prefered option, I might require some assistance to get this working.
from dbt-bigquery.
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.
from dbt-bigquery.
Hey there,
Has this issue been solved?
from dbt-bigquery.
Related Issues (20)
- [Release Improvements] Refresh workflows for the `pyproject.toml` migration
- [Bug] persist_docs not working for seeds HOT 2
- Import relevant pytest(s) for cross-database `cast` macro
- [Bug] `docs generate` appears to be returning no table metadata when run with the `--no-compile` option HOT 1
- [Tech Debt] `test_dbt_debug` is not dropping its test schema
- [Bug] `docs generate` does not find all schemas when there are more than 10K schemas
- Cross-database `date` macro
- [Bug] incremental run with __dbt_tmp table does not log the real bytes_billed in run_results.json HOT 1
- [Bug] Requested entity not found when writing a Python model to BigQuery
- [Feature] support copy multiple tables in parallel using copy_partitions
- [Feature] Implement batch metadata freshness using `INFORMATION_SCHEMA.TABLE_STORAGE` instead of `client.get_table` HOT 2
- BigQuery authorized dataset HOT 1
- [Bug] --empty flag not working on Pseudo-columns HOT 3
- [Feature] Support overriding `grant_access_to`
- [Bug] many timeouts with `priority: interactive` config HOT 7
- [Bug] Incremental models don't update new elements in struct column
- [Regression] `docs generate` raises error when querying dataset with BQ ML models HOT 1
- dbt bigquery 1.3 Flake8 is failing a mypy check
- [Bug] Cannot parse partition config when specified in property YAML file HOT 2
- [Test] testing workflow
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dbt-bigquery.