Comments (5)
Regarding the view, should we create it optionally in https://github.com/google/slo-generator/blob/master/slo_generator/exporters/bigquery.py?
from slo-generator.
I just confirmed the BQ view cannot be created with the documented query:
$ cat create_view.sql
CREATE VIEW `slo-generator-demo.slo_generator.last_report` AS
SELECT
r2.*
FROM
(
SELECT
r.service_name,
r.feature_name,
r.slo_name,
r.window,
MAX(r.timestamp_human) AS timestamp_human
FROM
`slo-generator-demo.slo_generator.custom_metrics` AS r
GROUP BY
r.service_name,
r.feature_name,
r.slo_name,
r.window
ORDER BY
r.window
)
AS latest_report
INNER JOIN
`slo-generator-demo.slo_generator.custom_metrics` AS r2
ON r2.service_name = latest_report.service_name
AND r2.feature_name = latest_report.feature_name
AND r2.slo_name = latest_report.slo_name
AND r2.window = latest_report.window
AND r2.timestamp_human = latest_report.timestamp_human
ORDER BY
r2.service_name,
r2.feature_name,
r2.slo_name,
r2.error_budget_policy_step_name
$ bq query `cat create_view.sql`
Error in query string: Error processing job 'slo-generator-demo:bqjob_r7e99d09194d84a93_000001848af1cff1_1': 1.1 - 0.0:
Unrecognized token CREATE.
[Try using standard SQL (https://cloud.google.com/bigquery/docs/reference/standard-sql/enabling-standard-sql)]
Then switching to Google Standard SQL with --use_legacy_sql=false
or --nouse_legacy_sql
works as expected:
$ bq query --use_legacy_sql=false `cat create_view.sql`
Waiting on bqjob_r73661e3c9c6d85a4_000001848af3acc9_1 ... (0s) Current status: DONE
Created slo-generator-demo.slo_generator.last_report
@gplasky Regarding the use of cat
, I am under the impression bq
is able to read from the standard input as $ bq query --use_legacy_sql=false < create_view.sql
works just fine. Does that work for you instead of $(<create_view.sql)
?
Now on to the Data Studio part.
from slo-generator.
Hi @gplasky, @tuananhnguyen-ct,
I will review the deploy instructions, create a sample BigQuery report to confirm they are still up-to-date, then advise on the next steps and get back to you.
from slo-generator.
I was able to create a Data Looker Studio report by following the instructions. We just need to update the screenshots so they match the current UI.
@tuananhnguyen-ct Regarding the creation of the view: it is something that needs to be done only once. As such, IMHO, it does not make sense to delegate the creation to the exporter. In a "separation of concerns" and "least privilege" approach, I think the exporter should focus on doing one thing only (exporting metrics) and doing it well under a dedicated identity. Then the creation of the view can be delegated to another identity/job role (for example someone from the BI team). WDYT?
from slo-generator.
Reopening for the Terraform part.
from slo-generator.
Related Issues (20)
- 💡 [REQUEST] - Document everything going on in GCP project `slo-generator-ci-a2b4`
- 💡 [REQUEST] - Automate dependency updates with `renovate-bot` HOT 1
- 💡 [REQUEST] - Investigate the results of the Scorecards GitHub Action
- 💡 [REQUEST] - Add a "Why?" section to `README.md`
- 💡 [REQUEST] - Multiwindow, multi-burn-rate alerts HOT 2
- 🐛 [BUG] - CI is unable to deploy new images to Cloud Run
- 🐛 [BUG] - Warnings in GitHub Actions workflows
- 🐛 [BUG] - Similar GitHub Actions workflows have different triggers
- 💡 [REQUEST] - Make the Docker image smaller HOT 1
- 💡 [REQUEST] - Prometheus SLO Recording Rule examples HOT 1
- 🐛 [BUG] - `safety check` fails during CI HOT 2
- 🐛 [BUG] - `safety` reports CVE in `pip` with Python 3.11 HOT 7
- 🐛 [BUG] - latest releases aren't pushed into the GCR HOT 10
- 🐛 [BUG] - `safety` reports CVE in `pip <23.3` HOT 1
- 🐛 [BUG] - Unit tests are failing with all versions of Python HOT 2
- 🐛 [BUG] - Image not available from gcr HOT 3
- 🐛 [BUG] - Failed to release `v2.6.0` HOT 13
- 🐛 [BUG] - Synthetic Probes show a high number a 504 errors HOT 7
- 💡 [REQUEST] - Collect Cloud Trace data to troubleshoot latency issues and timeouts HOT 4
- 💡 [REQUEST] - Enable timestamp on logs
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from slo-generator.