VS Code extension for Dataform with following features
Feature | Description |
---|---|
Dependancy graph | Interative dependancy graph with external sources higlighted in distinct colors |
Inline diagnostics on .sqlx file ❗ |
Native lsp like experience with diagnostics being directly put on both the sqlx file & compiled query |
Go to definition | Go to definition for source in $ref{("MY_SOURCE")} . Takes you to MY_SOURCE.sqlx or sources.js at the line where MY_SOURCE is defined |
Auto-completion | - declarations in ${ref("..")} trigger when $ character is typed - Dependencies when " or ' is typed inside the config block which has dependencies keyword is in the line prefix - tags when " or ' is typed inside the config block which has tags keyword is in the line prefix |
Code actions | Apply dry run suggestions at the speed of thought |
Compilation & Dry run stats | - Live compiled query in a vertical split on save which is in sync with the current cursor position of your .sqlx file - Data processed by query on bottom right on successful dry run |
Run a specific file/tag | Run a file/tag, optionally with dependencies/dependents with vscode command pallet / menu icons |
Format using Sqlfluff 🪄 | Fromat .sqlx files based on sqlfluff config using formatdataform cli |
-
npm i -g @dataform/cli
-
To enable formatting using sqlfluff config install formatdataform extension [ optional ]
# install python and run pip install sqlfluff # install formatdataform cli curl -sSfL https://raw.githubusercontent.com/ashish10alex/formatdataform/main/install_latest.sh | bash
You can verify if
formatdataform
is installed correctly by runningformatdataform --help
-
To enable prettier diagnostics install Error Lens extension [ optional ]
Note
Trouble installing ? Please see FAQ section, if you are still stuck, please raise an issue here
Auto completion support for dependencies
when "
or '
is typed inside the config block which has dependencies
keyword is in the line prefix
Declarations in ${ref("..")}
trigger when $ character is typed
Auto completion support for tags
when "
or '
is typed inside the config block which has tags
keyword is in the line prefix
Go to definition for source in $ref{("MY_SOURCE")}
. Takes you to MY_SOURCE.sqlx
or sources.js
at the line where MY_SOURCE
is defined
Live compiled query in a vertical split on save which is in sync with the current cursor position of your sqlx file. Data processed by query on bottom right on successful dry run
Open vscode command pallet by pressing CTLR + SHIFT + p or CMD + SHIFT + p on mac and run one of the required commands
Commands |
---|
Dataform: Run current file |
Dataform: Run current file with dependencies |
Dataform: Run current file with dependents |
Dataform: Run current tag |
Dataform: Run current tag with dependencies |
Dataform: Run current tag with dependents |
Dataform: Format current file |
-
Unable to execute command e.g. error
command vscode-dataform-tools.xxx not found
- It is likely that the vscode workspace folder is not opened at the root of your dataform project. For example, if your dataform project is located at
~/Documents/repos/my_dataform_project
ensure that workspace is opened at~/Documents/repos/my_dataform_project
NOT~/Documents/repos/my_dataform_project
- The above design is to facilitate the exection of
dataform compile --json
command without infering the dataform root at run time
- It is likely that the vscode workspace folder is not opened at the root of your dataform project. For example, if your dataform project is located at
-
Error compiling Dataform, process existed with exit code 1
- Check if correct dataform cli version is installed by running
dataform --version
in your terminal - Ensure that dataform cli version matches the version required by the project
- Try compiling the project by running
dataform compile
on your terminal from the root of your dataform project - In case you need to install a specific dataform cli version by running
npm i -g @dataform/[email protected]
. Make sure you verify the version by running thedataform --version
- In case the error is not due to all the above reasons it is likely that you have a compilation error in your pipeline
- Check if correct dataform cli version is installed by running
-
- Run
dataform init-creds
from the from the root of your dataform project in your terminal - You will be promted to pick the location and type of authentication
json/adc
. Choosing adc will be use your default gcp credentials that you had setup usinggcloud
- Run
- Features such as go to definition / dependancy graph might not work with consistantly with
${ref("dataset", "table")}
- sync feature flickers when user tries to scroll a non-active editor. Fixes when user selects the active editor by clicking on it
- Show pre and post operation queries in compiled query
- Use
.sqlx
specific grammar. Docs on how to do it - Preview query results
- Bundle javascript files in the extension using esbuild or webpack
- Handle case where user is not connected to internet or on vpn where network request for dry run cannot be made
- Add proper logging, winston-transport-vscode