Git Product home page Git Product logo

google / starthinker Goto Github PK

View Code? Open in Web Editor NEW
166.0 22.0 55.0 31.4 MB

Reference framework for building data workflows provided by Google. Accelerates authentication, logging, scheduling, and deployment of solutions using GCP. To borrow a tagline.. "The framework for professionals with deadlines."

License: Apache License 2.0

Python 64.79% Shell 2.43% HTML 2.45% Jupyter Notebook 30.34%
python automation bigquery airflow colab-notebook cloud-functions app-engine django dv360 cm360

starthinker's Introduction

End of Google Support: The StarThinker project will no longer receive new solution contributions from the Google team. Please read the full StarThinker Open Source Support Ends At Google article for more details.

Disclaimer: This is not an officially supported Google product. It is a reference implementation. There is absolutely NO WARRANTY provided for using this code. The code is Apache Licensed and CAN BE fully modified, white labeled, and disassembled by your team.

Scaled Automation Application For Ad Operations Teams

At gTech, we believe every ad operations team should be faster, nimbler, and able to use all their data sources to drive client impact. To that end, we've created StarThinker, a simple and intuitive web UI that allows users to create, edit, run, and schedule data pipelines consisting of data transfer and data processing jobs. StarThinker deploys on Google Cloud in minutes without engineering or coding required. Read the WhitePaper.

StarThinker Screen Shots

For Ad Operations Teams...

Sheets only go so far, with a StarThinker running, the ad operations team can operate on larger data sets, scale solutions across clients fast, and even automate campaign controls. All using a drag and drop UI.

Steps From Tutorial:

For Ad Engineering Teams....

No one enjoys building boiler plate code, it's slow, error prone, tedious, and honestly drains the energy out of any project. At best StarThinker lets you skip all that, at worst it's working code you can strip down for your needs. Take what you need.

  • Out of the box authentication.
  • Out of the box scaling.
  • Out of the box logging.
  • Out of the box testing.
  • All Apache Licensed to give you a head start and complete development control.

Steps From Tutorial:

Where Do I Get Help?

Authors

Additional contributors to individual handlers and recipes are listed in each file.


© 2020 Google LLC - Apache License, Version 2.0

starthinker's People

Contributors

ceoloide avatar dependabot[bot] avatar hgrias avatar mauriciodesiderio avatar pkenjora avatar rantwijk avatar rmstanford avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

starthinker's Issues

DV360 Bulk Editor Preview Missing Deltas

When 'Insert' ing a new Segment, or new Flight and Budget at Insertion Order, the 'Insert' row is not showing up when generating the Preview, yet is picked up when we load our changes, as a Success. Because it ultimately worked, this issue is likely deemed low-med priority, as users can always go to BigQuery to validate.

Reporting this following a ~2:40 pm cst deployment today to our production environment, which contained code fixes for previous Preview bugs, where deltas were not showing up.

starthinker-help group email not working

Got this back,

We're writing to let you know that the group you tried to contact (starthinker-help) may not exist, or you may not have permission to post messages to the group. A few more details on why you weren't able to post:

 * You might have spelled or formatted the group name incorrectly.
 * The owner of the group may have removed this group.
 * You may need to join the group before receiving permission to post.
 * This group may not be open to posting.

The email address is in a lot of places in the project.

DV Bulk Editor: Bid Strategy Edit Wont Allow Change of Bid Strategy/Type for Line Items

In the context of DV360 Bidding Strategies, the following editable values in the sheet/UI are mutually exclusive:

  1. Fixed Bid
  2. Bid Goal
  3. Bid Algorithm

For example, if we wanted to change the bid type from fixed bid to bid goal, we would have to delete the value in the record's Fixed Bid Edit column and add a value to the Auto Bid Goal Edit column. When this happens, we get the following NoneType error as the original value for Fixed Bid Edit is no longer a float, but an empty cell.

Float(row['Fixed_Bid_Edit']) * 100000 TypeError: float() argument must be a string or a number, not 'NoneType'

This prevents the user from changing a Line Item's bid strategy from one to another.

Adding / Deleting Exchanges at Advertiser & Line Item Layer

We just walked-through with a user on how to delete and add exchanges to an existing Line Item using the DV360 Bulk Targeter. It is a similar issue to reported below, yet different error reported and the exchange updates were to the Line Items, not the Advertisers. We used both the Delete and Add actions, and once we got an Error on our Preview, we updated to Delete Line Items and Add Line Items actions to see if that addressed our issue. However, it resulted in the same error.

The Warning upon generating the Preview said “Already deleted at this layer TARGETING_TYPE_EXCHANGE” for all our line items. Yet, when we verify the Line Item within the UI, it was not deleted and currently selected.

The updates included:

  1.   Delete Exchange_Triplelift to the filtered line items
    
  2.   Add Exchange_Inmobi to the filtered line item (this did show up in Preview, but was an Error upon updating)
    

It looks like the Warning / Preview knows what we are looking to do, yet thinks those updates have already been applied? But we checked in UI, and Triplelift was very much still selected as an Exchange for those Line Items.

Additionally, in our drop-down of Exchanges on the Designation Targeting tab, I noticed we are still missing a few options, including Sharethrough and Verizon Media Exchange. I noticed the API documentation updated to include Sharethrough, yet your list does not include, and Exchange is not a list I can add to within the Targeting Options or Inventory Sources tabs. Verizon Media Exchange does appear to be missing from the API documentation list.

Two Functions Named setup_developer_token() In install/config.sh

Summary

The first function is meant to setup the user's Adwords API developer token and the second is meant to set an API key used for endpoints that need special approval from Google.

381 setup_developer_token() {
382   forced=$1
383
384   echo ""
385   echo "----------------------------------------"
386   echo "Set Developer Token - ${STARTHINKER_DEVELOPER_TOKEN}"
387   echo "----------------------------------------"
388   echo ""
414 setup_developer_token() {
415   forced=$1
416
417   echo ""
418   echo "----------------------------------------"
419   echo "Set API Key - ${STARTHINKER_API_KEY}"
420   echo "----------------------------------------"
421   echo ""

Proposed Solution

Rename the second function setup_api_key()

414 setup_api_key() {
415   forced=$1
416
417   echo ""
418   echo "----------------------------------------"
419   echo "Set API Key - ${STARTHINKER_API_KEY}"
420   echo "----------------------------------------"
421   echo ""

LI Preview Bid Amount

LI Preview - Bid Amount should be sourced from DCM (or from the IO preview field - Performance Goal Value), but it currently is empty upon preview generation.

LI Preview not de-duping

It appears as though when doing the LI preview, it correctly strips the creative size from the placements in DCM, but it does not then de-dupe, so the number of Line Items generated in the preview is much greater than it should be.

Insert/Preview action

This is more of a general usability issue, but with the current workflow it would be incredibly easy for a user to duplicate IOs when moving on to generating and inserting Line Items, as the action stays as Insert in the IO preview sheet, and the recipe has the combined LOAD/PREVIEW/INSERT actions. Is it possible to revert back to preview after insertion?

Parameters in email task

The script schema for the email script seems slightly different than the email task, in particular the send function.
There is an extra 'email' object in the json that's not accounted in the task

project.task['read']['from'],

https://github.com/google/starthinker/blob/master/scripts/email_to_bigquery.json#L20

    { "email":{
      'auth': 'user',
      "read":{
        "email":{
          "from":...

vs this one that seems to be working, without the additional "email" object

    { "email":{
      "auth":"user",
      "read":{
        "from":"

Thank you

Performance Tuning: DV360 Bulk Editor Recipe Run Times Lagging

For users who filtered on a non-TEST DV360 advertiser, and only 1 campaign under selected advertiser, when they went to Load Insertion Orders & Line Items, it was taking 60+ minutes to barely load the first tab of Insertion Orders, let alone all the other tabs. However, users who filtered on the TEST advertiser had theirs complete within normal running time.

The following will show you that it was taking 19+ minutes to read the Advertisers tab in my sheet.
recipe_running

Not sure if we can pin point what might have been the cause. Since everyone filtered on an Advertiser AND a Campaign, I can’t imagine the Campaign under non-TEST advertisers were significantly larger, if anything the opposite, because our TEST campaigns have a ton of IOs / Line Items loaded under them from QA.

UPDATE: User confirmed the following day that recipe eventually Errored and shared the following log:

Hour Task Output / Error
done_outline   13 dataset - 1 JOB_END 19 Hours 31 Minutes Ago TASK: dataset DATE: 2021-04-22 HOUR: 13 DATASET starthinker-dev-0561-1465-0921 DV_Editor_Dasher_Computing_Bulk_Editor DATASET CREATE DATASET ACCESS
done_outline   13 drive - 1 JOB_END 19 Hours 31 Minutes Ago TASK: drive DATE: 2021-04-22 HOUR: 13 Drive Drive Copy https://docs.google.com/spreadsheets/d/18G6cGo4j5SsY08H8P53R22D_Pm6m-zkE6APd3EDLf2c/ DV Editor Dasher Computing Bulk Editor Drive: File exists.
error   13 dv_editor - 1 JOB_ERROR 15 Hours 6 Minutes Ago TASK: dv_editor DATE: 2021-04-22 HOUR: 13 COMMAND: Load Insertion Orders and Line Items DISCOVERY FETCH: https://displayvideo.googleapis.com/$discovery/rest?version=v1&key= SHEETS CLEAR DV Editor Dasher Computing Bulk Editor Creatives B2:Z DISCOVERY FETCH: https://displayvideo.googleapis.com/$discovery/rest?version=v1&key= BIGQUERY JSON TO TABLE:  starthinker-dev-0561-1465-0921 DV_Editor_Dasher_Computing_Bulk_Editor DV_Creatives SHEETS READ DV Editor Dasher Computing Bulk Editor Advertisers A2:A BigQuery Buffer Size 230077472 Uploaded 100% BIGQUERY JOB WAIT: job_Tt3Xj9r4GoaU-hmc7IEiNw9loKJ9 .....JOB COMPLETE: starthinker-dev-0561-1465-0921:US.job_Tt3Xj9r4GoaU-hmc7IEiNw9loKJ9 SHEETS WRITE DV Editor Dasher Computing Bulk Editor Creatives B2 BIGQUERY QUERY: starthinker-dev-0561-1465-0921 DV_Editor_Dasher_Computing_Bulk_Editor HTTP ERROR: [Errno 32] Broken pipe HTTP RETRY / WAIT: 3 31 Traceback (most recent call last):   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 909, in execute     headers=self.headers,   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 204, in _retry_request     raise exception   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 177, in _retry_request     resp, content = http.request(uri, method, *args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/google_auth_httplib2.py", line 201, in request     uri, method, body=body, headers=request_headers, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/httplib2/init.py", line 1709, in request     conn, authority, uri, request_uri, method, body, headers, redirections, cachekey,   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/httplib2/init.py", line 1424, in _request     (response, content) = self._conn_request(conn, request_uri, method, body, headers)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/httplib2/init.py", line 1347, in _conn_request     conn.request(method, request_uri, body, headers)   File "/usr/lib/python3.7/http/client.py", line 1260, in request     self._send_request(method, url, body, headers, encode_chunked)   File "/usr/lib/python3.7/http/client.py", line 1306, in _send_request     self.endheaders(body, encode_chunked=encode_chunked)   File "/usr/lib/python3.7/http/client.py", line 1255, in endheaders     self._send_output(message_body, encode_chunked=encode_chunked)   File "/usr/lib/python3.7/http/client.py", line 1069, in _send_output     self.send(chunk)   File "/usr/lib/python3.7/http/client.py", line 991, in send     self.sock.sendall(data)   File "/usr/lib/python3.7/ssl.py", line 1015, in sendall     v = self.send(byte_view[count:])   File "/usr/lib/python3.7/ssl.py", line 984, in send     return self._sslobj.write(data) BrokenPipeError: [Errno 32] Broken pipe   During handling of the above exception, another exception occurred:   Traceback (most recent call last):   File "/home/cjones8/starthinker/task/dv_editor/run.py", line 168, in     dv_editor()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/dv_editor/run.py", line 81, in dv_editor     creative_load()   File "/home/cjones8/starthinker/task/dv_editor/creative.py", line 124, in creative_load     rows   File "/home/cjones8/starthinker/util/data/init.py", line 315, in put_rows     destination['sheets'].get('append', False),   File "/home/cjones8/starthinker/util/sheets/init.py", line 131, in sheets_write     valueInputOption=valueInputOption   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 129, in API_Retry     return API_Retry(job, key, retries - 1, wait * 2)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 502 when requesting https://sheets.googleapis.com/v4/spreadsheets/1tic_KyD0WmNU8v1QKKQ3lIfu0363txetEuIVYAm_zIA/values/Creatives%21B2?valueInputOption=RAW&alt=json returned "Bad Gateway">

Missing module lineitem_beta

I'm trying to deploy, https://google.github.io/starthinker/solution/dv3po_custom_signals/ but having issues.

It’s returning an error,

ModuleNotFoundError: No module named 'starthinker.task.lineitem_beta'

Going to https://github.com/google/starthinker/tree/master/starthinker/task I do see that it is missing.

Digging through branches, I see in some lineitem and lineitem_beta modules. I however can’t find a reason why it was removed and if it was replaced with something else.

Not sure if it's an error that it is missing. Is there any information or background?

Google Sheet Filters are Not Reliable

Not sure if this should be logged here or under Google Sheet's github. Of the StarThinker recipes that integrate with Google Sheets to display data, the filter feature within sheets is not always reliable.

Use case: User loads a large number of Line Items (rows) within the DV360 Bulk Editor recipe, and needs to be able to sort A-Z or filter on a key value.

Issue seen: Filter on Line Item column within template does not even list the available values, only sees blanks. User is not able to sort or filter as a result.

Priority: Not a critical bug or blocker.

Not able to get passed generating /starthinker_assets/service.json

Hey team, I'm super excited about this project (just found it!) But I've spent a few hours debugging why I can't get passed any step where it checks for /starthinker_assets/service.json.

I'm seeing:

created key [xxxxxxxxxxxxxxxxxx] of type [json] as [/Users/todd/Github/open-source/starthinker/starthinker_assets/service.json] for [[email protected]]
Failed To Create Service Credentials, Fix Then Run This Script Again
 1. Check permissions of gcloud user.
 2. Check if service accounts are enabled for the project.
 3. Manually paste service JSON credentials into: /starthinker_assets/service.json

I see /starthinker_assets/service.json is created and then deleted again. I'm running this using the correct GCP IAM project in scope and I'm a full-access project owner. I've manually run all of these and none cause errors:

https://github.com/google/starthinker/blob/master/install/config.sh#L596-L599

https://github.com/google/starthinker/blob/master/install/config.sh#L596-L599

https://github.com/google/starthinker/blob/master/install/config.sh#L596-L599

I'm unsure what the script is expecting to not error out. Any pointers would be super appreciated.

Thanks so much

Error When Running Create IO Preview

Running the IO preview leads to the following errors. The recipe settings are against my own project with all user toggles selected.

Hour Task Output / Error
done_outline   10 dataset - 1 JOB_END 7 Minutes Ago TASK: dataset DATE: 2021-04-16 HOUR: 10 DATASET sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training DATASET CLEAR DATASET CREATE DATASET ACCESS
done_outline   10 drive - 1 JOB_END 6 Minutes Ago TASK: drive DATE: 2021-04-16 HOUR: 10 Drive Drive Copy https://docs.google.com/spreadsheets/d/1HZ2yU9hAnQgu7K74aYtRaw9hH-vtv9BV7iqQTngCbks IO Preview Create IO Preview - Nate Training Drive: File exists.
error   10 sheets - 1 JOB_ERROR 6 Minutes Ago TASK: sheets DATE: 2021-04-16 HOUR: 10 SHEETS SHEETS READ IO Preview Create IO Preview - Nate Training IO Rules A1:P Traceback (most recent call last):   File "/home/cjones8/starthinker/task/sheets/run.py", line 143, in     sheets()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/sheets/run.py", line 102, in sheets     project.task['tab'], project.task.get('range', 'A1'))   File "/home/cjones8/starthinker/util/sheets/init.py", line 98, in sheets_read     range=sheets_tab_range(sheet_tab, sheet_range)   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 400 when requesting https://sheets.googleapis.com/v4/spreadsheets/1_6bcW1bqfyH4qqVMK2Ul9kV7JIA3qqYrDPKlqYmk6bc/values/IO%20Rules%21A1%3AP?alt=json returned "Unable to parse range: IO Rules!A1:P". Details: "Unable to parse range: IO Rules!A1:P">
error   10 sheets - 2 JOB_ERROR 6 Minutes Ago TASK: sheets DATE: 2021-04-16 HOUR: 10 SHEETS SHEETS READ IO Preview Create IO Preview - Nate Training Excludes A1:B Traceback (most recent call last):   File "/home/cjones8/starthinker/task/sheets/run.py", line 143, in     sheets()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/sheets/run.py", line 102, in sheets     project.task['tab'], project.task.get('range', 'A1'))   File "/home/cjones8/starthinker/util/sheets/init.py", line 98, in sheets_read     range=sheets_tab_range(sheet_tab, sheet_range)   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 400 when requesting https://sheets.googleapis.com/v4/spreadsheets/1_6bcW1bqfyH4qqVMK2Ul9kV7JIA3qqYrDPKlqYmk6bc/values/Excludes%21A1%3AB?alt=json returned "Unable to parse range: Excludes!A1:B". Details: "Unable to parse range: Excludes!A1:B">
error   10 google_api - 1 JOB_ERROR 6 Minutes Ago TASK: google_api DATE: 2021-04-16 HOUR: 10 GOOGLE_API dfareporting v3.4 campaigns.list DISCOVERY FETCH: https://dfareporting.googleapis.com/$discovery/rest?version=v3.4&key= BIGQUERY QUERY: sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training Traceback (most recent call last):   File "/home/cjones8/starthinker/task/google_api/run.py", line 381, in     google_api()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/google_api/run.py", line 374, in google_api     for kwargs in kwargs_list:   File "/home/cjones8/starthinker/util/data/init.py", line 177, in get_rows     for row in rows:   File "/home/cjones8/starthinker/util/bigquery/init.py", line 760, in query_to_rows     projectId=project_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/queries?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US">
error   10 google_api - 2 JOB_ERROR 6 Minutes Ago TASK: google_api DATE: 2021-04-16 HOUR: 10 GOOGLE_API dfareporting v3.4 sites.list DISCOVERY FETCH: https://dfareporting.googleapis.com/$discovery/rest?version=v3.4&key= BIGQUERY QUERY: sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training Traceback (most recent call last):   File "/home/cjones8/starthinker/task/google_api/run.py", line 381, in     google_api()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/google_api/run.py", line 374, in google_api     for kwargs in kwargs_list:   File "/home/cjones8/starthinker/util/data/init.py", line 177, in get_rows     for row in rows:   File "/home/cjones8/starthinker/util/bigquery/init.py", line 760, in query_to_rows     projectId=project_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/queries?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US">
error   10 google_api - 3 JOB_ERROR 5 Minutes Ago TASK: google_api DATE: 2021-04-16 HOUR: 10 GOOGLE_API dfareporting v3.4 advertisers.list DISCOVERY FETCH: https://dfareporting.googleapis.com/$discovery/rest?version=v3.4&key= BIGQUERY QUERY: sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training Traceback (most recent call last):   File "/home/cjones8/starthinker/task/google_api/run.py", line 381, in     google_api()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/google_api/run.py", line 374, in google_api     for kwargs in kwargs_list:   File "/home/cjones8/starthinker/util/data/init.py", line 177, in get_rows     for row in rows:   File "/home/cjones8/starthinker/util/bigquery/init.py", line 760, in query_to_rows     projectId=project_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/queries?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US">
error   10 google_api - 4 JOB_ERROR 5 Minutes Ago TASK: google_api DATE: 2021-04-16 HOUR: 10 GOOGLE_API dfareporting v3.4 placements.list DISCOVERY FETCH: https://dfareporting.googleapis.com/$discovery/rest?version=v3.4&key= BIGQUERY QUERY: sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training Traceback (most recent call last):   File "/home/cjones8/starthinker/task/google_api/run.py", line 381, in     google_api()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/google_api/run.py", line 374, in google_api     for kwargs in kwargs_list:   File "/home/cjones8/starthinker/util/data/init.py", line 177, in get_rows     for row in rows:   File "/home/cjones8/starthinker/util/bigquery/init.py", line 760, in query_to_rows     projectId=project_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/queries?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US">
error   10 google_api - 5 JOB_ERROR 5 Minutes Ago TASK: google_api DATE: 2021-04-16 HOUR: 10 GOOGLE_API dfareporting v3.4 placementGroups.list DISCOVERY FETCH: https://dfareporting.googleapis.com/$discovery/rest?version=v3.4&key= BIGQUERY QUERY: sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training Traceback (most recent call last):   File "/home/cjones8/starthinker/task/google_api/run.py", line 381, in     google_api()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/google_api/run.py", line 374, in google_api     for kwargs in kwargs_list:   File "/home/cjones8/starthinker/util/data/init.py", line 177, in get_rows     for row in rows:   File "/home/cjones8/starthinker/util/bigquery/init.py", line 760, in query_to_rows     projectId=project_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/queries?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US">
error   10 google_api - 6 JOB_ERROR 5 Minutes Ago TASK: google_api DATE: 2021-04-16 HOUR: 10 GOOGLE_API displayvideo v1 advertisers.campaigns.list DISCOVERY FETCH: https://displayvideo.googleapis.com/$discovery/rest?version=v1&key= BIGQUERY QUERY: sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training Traceback (most recent call last):   File "/home/cjones8/starthinker/task/google_api/run.py", line 381, in     google_api()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/google_api/run.py", line 374, in google_api     for kwargs in kwargs_list:   File "/home/cjones8/starthinker/util/data/init.py", line 177, in get_rows     for row in rows:   File "/home/cjones8/starthinker/util/bigquery/init.py", line 760, in query_to_rows     projectId=project_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry    data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/queries?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US">
error   10 google_api - 7 JOB_ERROR 4 Minutes Ago TASK: google_api DATE: 2021-04-16 HOUR: 10 GOOGLE_API displayvideo v1 advertisers.insertionOrders.list DISCOVERY FETCH: https://displayvideo.googleapis.com/$discovery/rest?version=v1&key= BIGQUERY QUERY: sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training Traceback (most recent call last):   File "/home/cjones8/starthinker/task/google_api/run.py", line 381, in     google_api()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/google_api/run.py", line 374, in google_api     for kwargs in kwargs_list:   File "/home/cjones8/starthinker/util/data/init.py", line 177, in get_rows     for row in rows:   File "/home/cjones8/starthinker/util/bigquery/init.py", line 760, in query_to_rows     projectId=project_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/queries?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules was not found in location US">
error   10 bigquery - 1 JOB_ERROR 4 Minutes Ago TASK: bigquery DATE: 2021-04-16 HOUR: 10 QUERY TO VIEW DV_IO_Insert_Review Traceback (most recent call last):   File "/home/cjones8/starthinker/task/bigquery/run.py", line 262, in     bigquery()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/bigquery/run.py", line 255, in bigquery     bigquery_query()   File "/home/cjones8/starthinker/task/bigquery/run.py", line 222, in bigquery_query     project.Project.task['to'].get('replace', False)   File "/home/cjones8/starthinker/util/bigquery/init.py", line 275, in query_to_view     projectId=project_id, datasetId=dataset_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/datasets/Create_IO_Preview___Nate_Training/tables?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules">
error   10 bigquery - 2 JOB_ERROR 4 Minutes Ago TASK: bigquery DATE: 2021-04-16 HOUR: 10 QUERY TO VIEW DV_IO_Insert_Audit Traceback (most recent call last):   File "/home/cjones8/starthinker/task/bigquery/run.py", line 262, in     bigquery()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/bigquery/run.py", line 255, in bigquery     bigquery_query()   File "/home/cjones8/starthinker/task/bigquery/run.py", line 222, in bigquery_query     project.Project.task['to'].get('replace', False)   File "/home/cjones8/starthinker/util/bigquery/init.py", line 275, in query_to_view     projectId=project_id, datasetId=dataset_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/datasets/Create_IO_Preview___Nate_Training/tables?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.SHEET_Rules">
error   10 bigquery - 3 JOB_ERROR 4 Minutes Ago TASK: bigquery DATE: 2021-04-16 HOUR: 10 QUERY TO SHEET IO Preview Create IO Preview - Nate Training SHEETS CLEAR IO Preview Create IO Preview - Nate Training Audit A2 SHEETS WRITE IO Preview Create IO Preview - Nate Training Audit A2 BIGQUERY QUERY: sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training Traceback (most recent call last):   File "/home/cjones8/starthinker/task/bigquery/run.py", line 262, in     bigquery()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/bigquery/run.py", line 255, in bigquery     bigquery_query()   File "/home/cjones8/starthinker/task/bigquery/run.py", line 184, in bigquery_query     rows   File "/home/cjones8/starthinker/util/sheets/init.py", line 116, in sheets_write     body = {'values': list(data)}   File "/home/cjones8/starthinker/util/csv/init.py", line 245, in rows_to_type     for row in rows:   File "/home/cjones8/starthinker/util/bigquery/init.py", line 760, in query_to_rows     projectId=project_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/queries?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.DV_IO_Insert_Audit was not found in location US". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.DV_IO_Insert_Audit was not found in location US">
error   10 bigquery - 4 JOB_ERROR 3 Minutes Ago TASK: bigquery DATE: 2021-04-16 HOUR: 10 QUERY TO SHEET IO Preview Create IO Preview - Nate Training SHEETS CLEAR IO Preview Create IO Preview - Nate Training IO Preview A2 SHEETS WRITE IO Preview Create IO Preview - Nate Training IO Preview A2 BIGQUERY QUERY: sc-amea-usr-starthin-natstavn Create_IO_Preview___Nate_Training Traceback (most recent call last):   File "/home/cjones8/starthinker/task/bigquery/run.py", line 262, in     bigquery()   File "/home/cjones8/starthinker/util/project/init.py", line 475, in from_parameters_wrapper     func()   File "/home/cjones8/starthinker/task/bigquery/run.py", line 255, in bigquery     bigquery_query()   File "/home/cjones8/starthinker/task/bigquery/run.py", line 184, in bigquery_query     rows   File "/home/cjones8/starthinker/util/sheets/init.py", line 116, in sheets_write     body = {'values': list(data)}   File "/home/cjones8/starthinker/util/csv/init.py", line 245, in rows_to_type     for row in rows:   File "/home/cjones8/starthinker/util/bigquery/init.py", line 760, in query_to_rows     projectId=project_id, body=body).execute()   File "/home/cjones8/starthinker/util/google_api/init.py", line 386, in execute     self.response = API_Retry(self.job)   File "/home/cjones8/starthinker/util/google_api/init.py", line 91, in API_Retry     data = job.execute()   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper     return wrapped(*args, **kwargs)   File "/home/cjones8/starthinker_virtualenv/lib/python3.7/site-packages/googleapiclient/http.py", line 915, in execute     raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 404 when requesting https://bigquery.googleapis.com/bigquery/v2/projects/sc-amea-usr-starthin-natstavn/queries?alt=json returned "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.DV_IO_Insert_Review was not found in location US". Details: "Not found: Table sc-amea-usr-starthin-natstavn:Create_IO_Preview___Nate_Training.DV_IO_Insert_Review was not found in location US"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.