Git Product home page Git Product logo

interactive-canvas-devtools's Introduction

Interactive Canvas DevTools

Interactive Canvas DevTools is a Chrome DevTools extension for debugging an Interactive Canvas web app by allowing for events and commands to be routed through a DevTools tab. This allows development to occur unbundled from a webhook and conversational model and testing everything together.

Usage

You can install the extension from the Chrome Web Store.

Once installed, you can open your Interactive Canvas project in your web browser, either from a hosted URL or localhost. In Chrome DevTools, find the Interactive Canvas tab and select it.

Screenshot of DevTools alongside web app

Sending JSON data and SSML marks

In the Send Data tab, you can type in JSON data to manually call the onUpdate callback.

In the TTS Mark tab, you can type in SSML marks to manually call the onTtsMark callback.

After sending data from the Chrome extension to the web app, a suggestion chip will appear underneath the textbox to allow you to quickly reenter the same data.

Viewing Outbound Events

Web app calls to sendTextQuery and setCanvasState are routed to the Chrome extension instead of the Google Assistant or your webhook. They are viewable in the History tab in reverse-chronological order.

Screenshot of how canvas state has changed

Multiple changes to the canvas state will result in a JSON diff.

Additional Preferences

There are other preferences that you can enable in the extension to better target their development environment.

  • The Show Header button will add a header to the webpage that resembles what the header will look like on a smart display. This can let you optimize your layout for that screen size.
  • The Import /SDK button will let you navigate to and select the project's sdk folder if you're using Actions SDK. The extension will then find JSON payloads and TTS marks from your project and make them available as suggestion chips.
  • Enabling Client-side logging will show messages in your DevTools console.
  • Enabling Extension debug logging will show messages for development of this tool.
  • Methods that are not supported in Interactive Canvas can be presented as a console warning or throw an error so your local development environment can better resemble a physical device.

Local Development Setup Instructions

If you would like to compile and test this extension with a local build, follow these instructions:

This project is built using Node.js >=10.18.0 and Angular >=12.

  1. Run npm i to install dependencies
  2. Run npm run build to build the extension. It will build the entire project. Alternatively, you can build each part separately.
    1. Run npm run build:client to build the code which will be run in the context of the webpage. The Node module artifacts are stored in the dist/client directory.
    2. This command also uses Parcel to transpile these artifacts to run in a browser environment. These transpiled artifacts are stored in the dist/client-bundle directory.
    3. Run npm run build:extension to build the project components that make up the DevTools page. It also will package the artifacts in dist/client-bundle and outputs the bundle in the dist/interactive-canvas-extension directory.
  3. Open Chrome extensions menu: chrome://extensions/
  4. Enable Developer Mode.
  5. Select Load Unpacked.
  6. Select project directory /path-to-project.../dist/interactive-canvas-extension.
  7. The extension will now be loaded.

Running Tests

Style is enforced using ESLint. There are also several unit tests on specific logical functions in this repo. There are not higher-order integration tests between the various components of the overall architecture.

The /example directory contains an SDK project based on the public Snowpal sample. This example project is used in several unit tests. It is only used for development purposes and is not included in the packaged extension.

  1. Run npm run test to run both ESLint and unit tests. Alternatively, you can run each test separately.
    1. Run npm run lint to run ESLint.
    2. Run npm run fix to run ESLint and automatically fix style errors.
    3. Run npm run test:unit to run unit tests with Ava.

References & Issues

Make Contributions

Please read and follow the steps in the CONTRIBUTING.md.

License

See LICENSE.

interactive-canvas-devtools's People

Contributors

fleker avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

interactive-canvas-devtools's Issues

Improve documentation for console commands

The function addDebuggingMethodsInJsConsole adds commands to the DevTools console to call onUpdate/onTtsMark/etc. methods. However, this is currently a secret set of APIs as we don't document it anywhere, neither in the README nor the extension itself.

We should:

  • Add a section to the README
  • Add a console.info message that these APIs are available when the extension loads

Important to consider for #1 too.

Store suggestion chips in some persistent storage

When you reload the page in a new tab, the suggestion chips for data payloads and TTS marks are cleared out. There should be a cache for these values so that the developer does not need to import the /sdk path every time.

There should also be some sort of way to reset this storage.

Investigate theming in light mode

Development of the extension design was primarily done in the dark mode. When in light mode, colors may not look as good as possible. This should be remedied where it makes sense.

Extension may cause websites to crash

This Chrome Extension is designed to work with a developer's Interactive Canvas web app. As part of this process it will add scripts to the webpage when it loads. These scripts may interfere with certain websites they visit and can cause problems.

Due to the extension itself not knowing what URLs belong to a developer, it will add scripts to every webpage you visit. Most of the time this is benign since that page won't have any callbacks to handle. However, there may be some which do cause problems.

To get around this issue, you can go to the settings for this Chrome Extension. Under Site Access you can select whether to enable it for all sites, for user-defined ones, or only if manually enabled on that site.

Screen Shot 2021-10-15 at 12 28 38 PM

Support outputTts

Similar to client-side fulfillment in #5, this issue is related to supporting outputTts entirely. This is a string that may be direct TTS or SSML.

How should this be implemented? Using Chrome's default TTS APIs, or through Google Cloud TTS by providing an API key?

Should this just override the outputTts method with new behavior? Do we want to have some sort of full-blown SSML simulator in the console, or is that too much?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.