Git Product home page Git Product logo

app-ml-accelerator's Introduction

Machine Learning Accelerator

The ML Accelerator is a purpose-built Looker application designed to give business users access to BigQuery and Vertex AI's machine learning capabilities. It provides a user-friendly interface designed to guide the user through each step of creating a machine learning model. Because of its simple, no-code interface, the application serves as a pathway for business analysts to learn and use predictive analytics in Looker.

View the ML Model Creation Flow document for an example of an end-to-end user journey.

IMPORTANT

Additional configuration is required after installation. A Looker Admin should complete the application configuration using the Installation Instructions below.

Report bugs or feature requests with this form. Contact [email protected] with questions or comments.


INSTALLATION INSTRUCTIONS

1. Choose a BigQuery Connection

You will need to select a BigQuery connection during installation. The application can only be used with a single connection to prevent data from moving between connections. The connection chosen will determine which Looker Explores will be accessible from within the application.

2. Adjust Service Account Roles

The service account used by the BigQuery connection chosen in Step 1 should have the following IAM predefined roles.

  • BigQuery Data Editor
  • BigQuery Job User
  • Vertex AI User

3. Create BigQuery Dataset for ML Models

Create a dataset (e.g., looker_bqml) in the BigQuery connection's GCP project.

4. Install Application

The application can be installed directly from Looker Marketplace (recommended) or manually installed following the steps below.

Option A: Marketplace Install

Refer to the Looker Docs for installing a tool from Marketplace. Select the BigQuery connection name chosen in Step 1 during installation.

Option B: Manual Install

5. Configure Application with User Attributes

The application uses three Looker user attributes to store its configuration settings. The following user attributes are required for the application to work properly. Each user attribute needs to be named exactly as listed below with a data type of String. The recommended setting for user access is View.

Create the following user attributes and set their default values.

Required User Attribute Name Default Value Description
marketplace_bqml_ext_ml_accelerator_bigquery_connection_name Connection name chosen in Step 1
marketplace_bqml_ext_ml_accelerator_gcp_project Projectd ID of the BigQuery dataset created in Step 3
marketplace_bqml_ext_ml_accelerator_bqml_model_dataset_name Name of BigQuery dataset created in Step 3 (e.g., looker_bqml)

6. Create a Looker Role to Manage User Access

The application is designed for users with access to Explores and SQL Runner in Looker. Users will need the following permissions to use the application.

  • explore
  • use_sql_runner

We recommend creating a new Looker role to easily manage user access to the application and guarantee users have the required permissions above.

  • Create a new Looker model set named ML Accelerator containing the LookML model ml_accelerator
  • Create a new Looker permission set named ML Accelerator containing all the permisions in the default User permission set AND the use_sql_runner permission
  • Create a new Looker role named ML Accelerator using the new model and permission set
  • Assign the ML Accelerator role to Looker users and groups

7. Setup AI-Generated Model Evaluation Summaries

After release 2.2, the application can use text generating AI to summarize the model evaluation to more clearly communicate model performance. This optional feature requires additional setup.

7a: Add an External Connection from Bigquery to Vertex

In BigQuery, an external connection is required to connect it to pre-trained models in Vertex AI. If one is not already set up, you must do so. A tutorial can also be found here.

  1. Under the same gcp project already in use for the application, verify the BigQuery Connection and Vertex AI APIs are both enabled.
  2. In BigQuery click “add,” then "Connections to external data sources."
  3. Select "BigLake and remote function" and use the same location as the dataset already in use by the application
  4. The ID will be the name of your connection. Since it could be used to connect to any number of pre-trained models in vertex it is wise to choose something generic, such as “ext-vertex-ai”
  5. Create the connection
  6. Go to the connection and copy the service account ID. In order to access remote functions from Vertex AI, the BigQuery connection delegation service agent (of the form bqcx-[#]@gcp-sa-bigquery-condel.iam.gserviceaccount.com) that is associated with this connection must have the "Vertex AI User" role, which can be added in IAM.
7b: Create the Remote Text-Generation Model

In BigQuery, enter the following statement in the query editor (this code uses the suggested naming conventions for the the steps above and assumes region is US-Multi). The text-bison@002 model is suggested, but other LLM models with good performance generating text could also be used. The model_name will be later added as a User Attribute value. A suggestion for model_name is "mla-text-bison"

CREATE OR REPLACE MODEL project_id.dataset_id.model_name
  REMOTE WITH CONNECTION `us.ext-vertex-ai`
  OPTIONS (endpoint = 'text-bison@002');

This will take a few minutes to load and will not return any results.

7c: Update the Relevant User Attribute

Similar to section 5 above.

Required User Attribute Name Default Value Description
marketplace_bqml_ext_ml_accelerator_generate_text_model_name Name chosen in step 7b above

app-ml-accelerator's People

Contributors

chris-m-schmidt avatar tpakeman avatar rbob86 avatar thjennifer avatar ajcrutch avatar annaserova avatar naushkhiljee avatar davidtamaki avatar lizbeacham avatar jeremytchang avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.