sub configs .sqlfluff not recognized about vscode-sqlfluff HOT 8 CLOSED

@RobertOstermann Yes that fixes the issue. Thanks a lot.

from vscode-sqlfluff.

@roivant-matts Are you still seeing this issue in 0.1.0? If so, possibly try setting the workingDirectory configuration. You should be able to use workingDirectoyr="${fileDirname}" or something similar to set the working directory at the same location as the file. SQLFluff should then pick up the configs.

from vscode-sqlfluff.

Thanks! I am still having trouble. It could be my lack of knowledge with sqlfluff config though. Linting is working, but I can't get it to pickup config of jinja variables. e.g.
This is what I currently have:
[sqlfluff]
dialect = bigquery

[sqlfluff:templater:jinja:context]
staging_prefix=tcb_raw
files_to_ingest=files_to_ingest

I can't be sure if either config is picked up, but the variables are not and I get for example:
Undefined jinja template variable: 'staging_prefix' in a template using that var

I set the workingDirectory by var and explicitly to the root of my repo where I have a .sqlfluff file. I also have a .sqlfluff file living with a specific set of templates. Even if I put the more specific jinja:context config in either it does not seem to pickup.

I had an environment to debug the extension, but this is lost now :)

from vscode-sqlfluff.

ok, actually I have it working again as it was before (was pyenv mistake). Basically, onSave takes config properly but onType ignores my nested config.

edit: I am setting the working directory in the extension json settings:
"sqlfluff.workingDirectory"=root_of_repo

from vscode-sqlfluff.

Interesting, this might be a SQLFluff issue. When doing onSave I pass in the file path and let SQLFluff read from the file directly, but when doing onType I have to stream the contents of the file directly via stdin to SQLFluff. My guess is SQLFluff does configuration detection differently for those two options. I had hoped directly using the workingDirectory might fix that, but thinking about it further it should already be using the file location as the default working directory so I am not sure that setting really matters in this case. I will take a closer look when I get some time, but my guess now is you might have to use the onSave option.

from vscode-sqlfluff.

Agree, I remember seeing the same thing and my impression was that sqlfuff would need to be changed. I posted about that issue on their repo:
sqlfluff/sqlfluff#3640
Thanks for checking into this and offering suggestions!

from vscode-sqlfluff.

I have the same issue with nested configs not being detected.

My structure is the following:

swlp-main (project root)
swlp-main/.sqlfluff (main config with 'dialect=none')

swlp-main/packages/swlp-models/migrations/mysql (dir containing migration files for mysql in 3 subdirectories)
swlp-main/packages/swlp-models/migrations/mysql/.sqlfluff (config file only setting 'dialect=mysql')

swlp-main/packages/swlp-models/migrations/sqlite (dir containing migration files for sqlite in 3 subdirectories)
swlp-main/packages/swlp-models/migrations/mysql/.sqlfluff (config file only setting 'dialect=sqlite')

When I run sqlfluff lint or sqlfluff fix while my CWD is the project root all works as intended.

The linting via the VSCode extension ("sqlfluff.linter.run": "onSave") also works like intended. Config files are picked up and the different dialects are used for their respective directories.

I don't use onType but I can kind of understand the issue with "sqlfluff.linter.run": "onType" because there is no file being linted, so sqlfluff has no "file working directory" and can propably only rely on a passed config file (via --config) I guess.

My problem lies with the formatting part of sqlfluff. Just like with "lint onType" the config files are not being picked up when I format a file. And I get the same result when I use "format onSave" or manually format via the command palette.

Here is the output:

------------------------------------------------------------

Format triggered for /home/admin-riedelc/projects/swlp-main/packages/swlp-models/migrations/sqlite/deploy/add_users.sql

------------------------------------------------------------

Reading from stdin, not file, input may be dirty/partial

--------------------Executing Command--------------------

sqlfluff fix --force -

------------------------------------------------------------

Received close event, code 1 signal null
Raw stdout output:

------------------------------------------------------------



------------------------------------------------------------

Raw stderr output:

------------------------------------------------------------

Traceback (most recent call last):
  File "/home/admin-riedelc/.local/bin/sqlfluff", line 8, in <module>
    sys.exit(cli())
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 1017, in fix
    _stdin_fix(lnt, formatter, fix_even_unparsable)
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 755, in _stdin_fix
    result = linter.lint_string_wrapped(stdin, fname="stdin", fix=True)
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 991, in lint_string_wrapped
    linted_path.add(self.lint_string(string, fname=fname, fix=fix))
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 829, in lint_string
    parsed = self.parse_string(
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 761, in parse_string
    rendered = self.render_string(in_str, fname, config, encoding)
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 697, in render_string
    config.verify_dialect_specified()
  File "/home/admin-riedelc/.local/pipx/venvs/sqlfluff/lib/python3.10/site-packages/sqlfluff/core/config.py", line 981, in verify_dialect_specified
    raise SQLFluffUserError(
sqlfluff.core.errors.SQLFluffUserError: No dialect was specified. You must configure a dialect or specify one on the command line using --dialect after the command. Available dialects:
ansi, athena, bigquery, clickhouse, databricks, db2, duckdb, exasol, greenplum, hive, materialize, mysql, oracle, postgres, redshift, snowflake, soql, sparksql, sqlite, teradata, trino, tsql


------------------------------------------------------------

When I set "sqlfluff.config": "${workspaceFolder}/.sqlfluff" I get the same result except that the executed command has the --config flag included.

Based on the Output I assume that the issue with formatting is the same as with "lint onSave". But I don't understand why for manual formatting stdin is used? Couldn't you pass the file path instead?
The same question goes for "format onSave". Or does the formatter hook in before the file is acctually saved on disk?

from vscode-sqlfluff.

@Cielquan You could try setting sqlfluff.experimental.format.executeInTerminal to true. This forces the extension to use the file path when formatting instead of using stdin. I use stdin by default because allowing sqlfluff to format the file directly can cause issues if any changes are made to the document while formatting is occurring. Using this option also requires the file to be saved for formatting to occur, this extension saves the file for you so not much of an issue, but worth noting.

Let me know if that setting fixes your issue

from vscode-sqlfluff.