sveetch / boussole Goto Github PK

See about at least one other available format for a settings file.

• Yaml ?
• Standard config ini file ?

CLI will need an option to choose the settings backend, keeping JSON format as default.

Describe the bug
Build on RTD has failed with the last release to fix the .readthedoc.yml file

Environment
Describe your environment:

• Plateform: Linux distribution
• Python version: 3.10
• Boussole version: 2.1.1
• libsass-python version: Pointless

To Reproduce
I didn't reproduced it locally yet, this only occurs on RTD build CI.

Here is a part of the long traceback (full version only available on RTD):

Running Sphinx v1.8.6
loading translations [en]... done
making output directory...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 17 source files that are out of date
updating environment: 17 added, 0 changed, 0 removed
reading sources... [  5%] api/compiler

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/cmd/build.py", line 304, in build_main
    app.build(args.force_all, filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/application.py", line 341, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 345, in build_update
    self.build(to_build,
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 360, in build
    updated_docnames = set(self.read())
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 468, in read
    self._read_serial(docnames)
[....]
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py", line 177, in __init__
    self._parse()
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py", line 578, in _parse
    lines = self._sections[section.lower()](section)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py", line 691, in _parse_parameters_section
    fields = self._consume_fields()
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py", line 258, in _consume_fields
    _name, _type, _desc = self._consume_field(parse_type, prefer_type)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py", line 250, in _consume_field
    _descs = self.__class__(_descs, self._config).lines()
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py", line 123, in __init__
    elif isinstance(obj, collections.Callable):  # type: ignore
AttributeError: module 'collections' has no attribute 'Callable'

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/boussole/envs/latest/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py", line 123, in __init__
    elif isinstance(obj, collections.Callable):  # type: ignore
AttributeError: module 'collections' has no attribute 'Callable'
The full traceback has been saved in /tmp/sphinx-err-d6st5y__.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

Looks to be related to napoleon shipped in Sphinx, strangely it seems a very old sphinx 1.x is used.

Expected behavior
It should works !

We should open a dedicated branch for testing to update docs/requirements.txt to use boussole[doc] instead of just boussole then try building doc for this branch and see if it resolves issue.

After following the docs for a basic installation, I can't seem to get Boussole to recognize .sass files. Am I missing something?

Boussole: 1.2.3 (libsass-python 0.14.5) (libsass 3.5.4)
Python: 3.6.6

$ ls -r *
sass:
test.sass

compiled:
$ boussole startproject 
Project base directory [.]: 
Sources directory [scss]: sass
Target directory [css]: compiled
Settings format name [json]: 
17:08:42 - Project directory structure and configuration file have been created.
17:08:42 - Now you should start to create some Sass sources into '/Users/tehfink/…/my_project/static/css/sass', then compile them using:
17:08:42 -     boussole compile --config=/Users/tehfink/…/my_project/static/css/settings.json
$ boussole -v 5 compile
17:10:11 - Building project
17:10:11 - Settings file: settings.json (json)
17:10:11 - Project sources directory: /Users/tehfink/…/my_project/static/css/sass
17:10:11 - Project destination directory: /Users/tehfink/…/my_project/static/css/compiled
17:10:11 - Exclude patterns: []

It seems the .sass file is ignored? Replacing it with a .scss file works:

$ boussole -v 5 compile
17:10:48 - Building project
17:10:48 - Settings file: settings.json (json)
17:10:48 - Project sources directory: /Users/tehfink/…/my_project/static/css/sass
17:10:48 - Project destination directory: /Users/tehfink/…/my_project/static/css/compiled
17:10:48 - Exclude patterns: []
17:10:48 - Compile: /Users/tehfink/…/my_project/static/css/sass/test.scss
17:10:48 - Output: /Users/tehfink/…/my_project/static/css/compiled/test.css

And manually calling sassc works:

$ sassc sass/test.sass compiled/test.css
$ cat compiled/test.css 
body {
  font: 100% Helvetica, sans-serif;
  color: #333; }

Contents of both files take from here:
https://sass-lang.com/guide

test.sass:

$font-stack:    Helvetica, sans-serif
$primary-color: #333

body
  font: 100% $font-stack
  color: $primary-color

test.scss:

$font-stack:    Helvetica, sans-serif;
$primary-color: #333;

body {
  font: 100% $font-stack;
  color: $primary-color;
}

Hey,

I installed boussole on my Ubuntu laptop and it works great, thanks so much!!!

However, this week I tried installing in on my Mac and unfortunately it doesn't seem to work. 😢

I've tried a simple pip install boussole and pip install boussole --user. It installs fine but when I try to run a command like boussole startproject I get an error command not found: boussole.

Somehow the command boussole doesn't seem to be in my path. Any ideas why this would be and how I can fix it?

With issue #29 we added usage of custom_import_extensions libsass option to be able to continue to keep behavior to include CSS sources (files like *.css) since it was deprecated.

But things have changed now and behavior is not the same and so libsass-python warns about usage of custom_import_extensions:

.venv/lib/python3.5/site-packages/sass.py:662: FutureWarning: `custom_import_extensions` has no effect and will be removed in a future version.

Have to verify again CSS behavior will still be forever or not and drop usage of custom_import_extensions.

PyYAML 5.1 adds a deprecation warning and wants yaml.load() calls to specify a Loader= argument, so boussole now emits this warning:

/opt/venvs/boussole/lib/python3.7/site-packages/boussole/conf/yaml_backend.py:60: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
    parsed = yaml.load(content)

The page linked in the message explains what it wants: https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation

Improve command line interface informations through colored logging with coloram if available.

All tests that stands on os.walk / listdir usage are not consistant from a machine to another, since os.walk and listdir return result in arbitrary order.

Seems we should sort results from core api.

Describe the bug
Not really a bug but a misinformation in Install documentation which evoke some C module compilation but this is not true anymore since libsass-python supply wheels since a long time, so there is no compilation required during its install, except maybe for some exotic architectures.

Environment
Describe your environment:

• Plateform: IOS/Linux distribution
• Python version: Any
• Boussole version: Any
• libsass-python version: Any

Expected behavior
Update this documentation to remove this note, i don't think there is a need to notice about some exotic architectures.

Comes with libsass==3.4.1, break one test with sourcemap that generate some differents mappings values, seems ok but require to update dependencies to 0.12.1 according to unittest.

During compile with watch mode, it seems we have an encoding bug:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/watchdog/observers/api.py", line 199, in run
    self.dispatch_events(self.event_queue, self.timeout)
  File "/usr/local/lib/python2.7/dist-packages/watchdog/observers/api.py", line 368, in dispatch_events
    handler.dispatch(event)
  File "/usr/local/lib/python2.7/dist-packages/watchdog/events.py", line 446, in dispatch
    self.on_any_event(event)
  File "/usr/local/lib/python2.7/dist-packages/boussole/watcher.py", line 173, in on_any_event
    self.index()
  File "/usr/local/lib/python2.7/dist-packages/boussole/watcher.py", line 91, in index
    library_paths=self.settings.LIBRARY_PATHS
  File "/usr/local/lib/python2.7/dist-packages/boussole/inspector.py", line 106, in inspect
    self.look_source(sourcepath, library_paths=library_paths)
  File "/usr/local/lib/python2.7/dist-packages/boussole/inspector.py", line 79, in look_source
    self.look_source(path, library_paths=library_paths)
  File "/usr/local/lib/python2.7/dist-packages/boussole/inspector.py", line 63, in look_source
    finded_paths = self.parse(fp.read())
  File "/usr/local/lib/python2.7/dist-packages/boussole/parser.py", line 134, in parse
    self.remove_comments(content)
  File "/usr/local/lib/python2.7/dist-packages/boussole/parser.py", line 83, in remove_comments
    return self.REGEX_COMMENTS.sub("", content)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 102: ordinal not in range(128)

This happens on the first change, even if the file is empty without any comment. Maybe i introduced the bug during porting to Python3 ?

Is your feature request related to a problem? Please describe.
Recent projects which want to use last Click version are blocked if they use Boussole since it is pinned to <6.0,>=5.1.

This is now an important issue for Boussole longevity.

Describe the solution you'd like
At least upgrade the CLI to fit to Click 7.x and update requirement to >7.0 without max limit.

Describe alternatives you've considered
The same but with requirement pinned to <8.0,>7.0 but the same jobs will come again in some time.

Something like:

boussole compile -f sample.scss

Boussole should then start silently a temporary config to compile from file directory and so compile this file.

Describe the bug
Users using Python3.6 to install Boussole will have an error since last version click version dropped Python3.6 support.

Environment
Describe your environment:

• Plateform: any
• Python version: 3.6
• Boussole version: 2.0
• libsass-python version: any

To Reproduce

1. Try to install Boussole with Python3.6
2. You get ERROR: Package 'click' requires a different Python: 3.6.15 not in '>=3.7'

Expected behavior
If Python 3.6 support is keeped, we should pin click to <8.1.0 to ensure it will not fail.

However we should pin click to the last working version (checking with test/tox) and add a boundary to the next version since it may have incompatible changes (like a refactoring).

Additional context
I'm a kiwi

Is your feature request related to a problem? Please describe.
setup.cfg have all development requirements in the extra requirement "dev" that is used by tox.

Tox does not need about flake or documentation requirements and installing them is longer than needed.

Describe the solution you'd like
All of these requirements should be move into a new extra requirement ("quality" ?):

    flake8
    Sphinx
    sphinx-rtd-theme
    livereload
    twine

Then tox will be faster to proceed to run. We may profit about this to add more Python versions coverage.

Reported error from pure sass syntax cause is not a problem, output is relevant and sufficient enough.

BUT when libsass cause a segmentation fault/core dump, there is no output.

I don't think we can manage something to have more output than just the core dump message, but at least the main compiled file causing it should be displayed.

Actually compiled file name is displayed only when the compile is finished. It should be usefull to know what file to search for.

To ease development on unittests through different environments.

Add some settings validations:

• Validate paths;
• Validate required settings: SOURCES_PATH and TARGET_PATH;
• Ensure source directory is not also setted in LIBRARY_PATHS ?

Ensure everything is exposed for end users.

Use the new method where setup.py is just an entrypoint for setuptools to get packaging info, pytest and tox config moved into setup.cfg, package version lives in setup.cfg, etc..

Describe the bug

I have written the following function in python that use python-libsass to preprocess sass on runtime in production, and in development, that watch for change using boussole.

• Boussole documentation: https://pypi.org/project/boussole/
• Python libsass: https://pypi.org/project/libsass/

This work well for production, however, with boussole it watch for change, but I keep having my python process crashing with :

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/shutil.py", line 713, in rmtree
ImportError: sys.meta_path is None, Python is likely shutting down

I already took caution not to rerun the process when I have the flask hotreload on *.py edition by clearing the previous subprocess on restart. I checked and I only have one process running all the time.

It work fine when I do not run the boussole subprocess with python.

This is my boussole.yml configuration:

---
SOURCES_PATH: "."
TARGET_PATH: "../css/compiled"
SOURCE_MAP: "true"
EXCLUDES: "../css/compiled"

Environment
Describe your environment:

• Plateform: Debian
• Python version: 3.10
• Boussole version: 2.1.0
• libsass-python version: 0.22.0

To Reproduce

import glob
import os
from pathlib import Path
import signal
from subprocess import DEVNULL
from subprocess import Popen
from subprocess import STDOUT

import sass

from pcapi import settings


def preprocess_scss(watch: bool) -> None:
    source = Path("src/pcapi/static/backofficev3/scss")
    destination = Path("src/pcapi/static/backofficev3/css/compiled")
    configuration = Path("src/pcapi/static/backofficev3/scss/boussole.yml")
    pid_file_path = Path("src/pcapi/static/backofficev3/scss/boussole.pid")

    Path(destination).mkdir(parents=True, exist_ok=True)

    if settings.IS_RUNNING_TESTS is not True:
        if watch:
            if os.environ.get("WERKZEUG_RUN_MAIN") != "true":
                has_never_compiled_css = len(glob.glob(f"{destination}/**/*.css", recursive=True)) == 0
                if has_never_compiled_css:
                    sass.compile(
                        dirname=(source, destination),
                        output_style="compressed",
                        source_map_contents=True,
                        source_map_embed=True,
                        source_map_root=destination,
                    )

                # kill previous boussole process if python previously crashed
                try:
                    if os.path.isfile(pid_file_path):
                        with open(pid_file_path, "r", encoding="utf8") as pid_file:
                            pid = int(pid_file.read())
                            pid_file.close()
                            os.kill(pid, signal.SIGTERM)
                except Exception:  # pylint: disable=broad-except
                    pass

                proc = Popen(  # pylint: disable=consider-using-with
                    ["boussole", "watch", "--config", configuration, "--backend", "yaml"],
                    stdout=DEVNULL,
                    stderr=STDOUT,
                )

                # save new process pid in case of python crash
                with open(pid_file_path, "w", encoding="utf8") as pid_file:
                    pid_file.write(str(proc.pid))
                    pid_file.close()

                print("💅 Scss compiler attached and watching, enjoy styling 💅", flush=True)
        else:
            sass.compile(
                dirname=(source, destination),
                output_style="compressed",
                source_map_contents=True,
                source_map_embed=True,
                source_map_root=destination,
            )
            print("💅 Scss compiler has compiled css 💅", flush=True)

Expected behavior

I expect boussole to watch for change without error

Additional context

NA

Actually, source map path is generated using source paths, not publish source path url.

So the map path builded inside compiled CSS files is not valid.

Is your feature request related to a problem? Please describe.
Purpose of this feature would be to create a JSON file of built files from a compilation alike webpack-bundle-tracker does for Webpack.

Describe the solution you'd like
Every time a compilation is performed, the compiler should write a file like boussole-stats.json into the target directory (as defined from settings) which would list every built CSS file item.

File item may contains some information like the main Sass source (as a relative path from source dir) it comes from, the filename, maybe the file size, its relative (to the target dir) path and possibly the absolute file path.

Additional context

It would help developer to integrate CSS built assets into their project with a little bit of code into an asset manager.

Also since the new feature HASH_SUFFIX, the built filenames can change on every compile. file.scss won't render just to file.css but to file.3f91ee4f01a2081b6127.css, then file.18e67c22d578d32b07e3.css, etc..

It can not be simply included anymore with file.css, this manifest would help to include it without to modify template each time a compile has been done.

Importing a CSS source like:

@import 'foo';

Does not find anymore an existing foo.css file.

See:

• sass/libsass-python#245
• sass/libsass#2636

Since packaging rewrite, everything should be in setup.cfg and setup.py is just an entrypoint to it.

Sadly, i forgot to clean up the setup.py

Boussole has its own internal code to inspect Sass sources since libsass does not expose parser method to do so.

And inspection does not have exactly the same path resolution process than libsass. Inspection is only used within watcher, compiler does not need it since it only gives some path to libsass compiler that just do the job on its own.

It results in compile action can build some sources that can fails with the watch action because the latter need to inspect sources to be able to find dependencies and choke on "unclear" path resolution.

This is troubling, at first it must be documented, then after possibly a check action that just perform source inspection would be nice.

logging.getLogger("boussole") should be enough to use within these objects, no need to require to pass the logger to each object.

When 0.9.0 milestone is done and beta has been somewhat tested from users, start to dig for Python 3.4 support.

Default filename for settings is currently settings.json, it is not enough since other software use a settings.* file, this may trouble user.

It would be nice that boussole search first for a boussole.json file then settings.json that will continue to be supported for ever.

Hi @sveetch,

I'm using Boussole inside a Vagrant VM, where my SCSS files are being edited on the host, but are available to the guest through a shared folder. The watch mode doesn't work in this kind of setup, since inotify events don't occur in a shared folder, so watchdog doesn't notice any changes. It looks like some other projects have addressed this by adding an option to use a watchdog PollingObserver instead. If Boussole could add a similar option, that would be very useful for similar development setups.

Here's a couple of links to related issues/PRs for more information:

• gorakhargosh/watchdog#283
• joeyespo/pytest-watch#10

This will launch an interactive prompt for:

1. Base directory [Default: .]: ...
2. Path to source directory [Default: scss/]: ...
3. Path for CSS destination [Default: css/]: ...

Libraries and exclude may not be editable through prompt.

Once it's done, create the settings file and possibly create directories if required.

Remove every call to click.echo/secho from API, only return strings. CLI have to print out infos itself.

Is your feature request related to a problem? Please describe.
Boussole stands on libsass (through libsass-python) that has been deprecated since years now in profit of dart-sass.

Sadly now Boussole is becoming to be obsolete, since more and more CSS framework tends to stand on dart-sass.

However, dart-sass pure js implementation is known to be dramatically slower than libsass or even node-sass.

Then the Sass team has developped sass-embedded which is something alike libsass in the terms that it allows other language to embed dart-sass in a package without relying directly on dart or node.

Describe the solution you'd like
Boussole could add a new compiler backend to use the sass-embedded with sass-embedded host in Python.

In resume, sass-embedded is somewhat an API (that is wrapper around dart-sass) that can be used through a host.

It seems sass-embedded system is almost efficient, still it would need some test on massive Sass project (with multiple main scss files to compile using some libraries) to check performance. If it leads to something around dart-sass pure js speed, this would be a no way. But if it leads to something like node-sass speed, it would be clearly something to adopt.

Then Libsass backend could live sometime as an alternative, depending its maintenance and elapsed time.

Problems

There is no sass-embedded Python host yet and i don't understand everything yet in embedded protocol to see where to start. Also i only know about two host implementation, the Node one and a Ruby one. It needs some R&D time.

• embedded-protocol
• dart-sass-embedded
• Node embedded host

Describe alternatives you've considered
If Boussole stills tied to libsass, it will just stop to be maintained in some years or before, since most frameworks won't compile with it.

Currently Boussole is very strict about import resolution: it refuses to compile sources which have import path that resolve to more than one source.

Note this is only an issue with "watch" mode which make introspection on Sass sources to find main sources to rebuild. With the "compile" mode, there is not introspection and this libsass which does everything, no boussole code is involved in import resolution.

Current behavior

If a source contains @import "foo" and there are foo.css and foo.scss files, it raises an error about unclear resolution since foo can be understood as foo.css or foo.scss. This can happen also with identical source file (relative) path with included libraries.

Expected behavior

This is not the way other compiler works especially node-sass. Even Sass documentation define another behavior: https://sass-lang.com/documentation/at-rules/import#load-paths

Also, we should fit to the node-sass behaviors so Boussole could be a valid replacement. And so we may have to implement the "~" behavior which seek for path from "node_modules" which path will have to be configured in settings.

Filename in current directory win against similar names from include libraries. This should be the default behavior to adopt (and strict could stay as an option).

Still there is some details to study to go further:

• How it should work when the only conflict is the extension (scss, css, sass) ? Keep raising an error ? Or should be resolve by priorities (which extensions are highest and lowest priority) ?
• Libraries priority comes from includes paths order as defined from settings ?

This must not replace the current import behavior and must be introduced as a new optional resolution engine (depending a setting?) and current behavior will still be the default one for some time as a deprecation process.

Some links from http://boussole.readthedocs.io/en/latest/api/index.html#modules are broken leading to a 404

First, need to see about conftest.py behavior with the one at tests root dir and the ones in module subdir, to see how to dispatch test utilities.

During compile with watch mode, it seems we have an encoding bug:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/watchdog/observers/api.py", line 199, in run
    self.dispatch_events(self.event_queue, self.timeout)
  File "/usr/local/lib/python2.7/dist-packages/watchdog/observers/api.py", line 368, in dispatch_events
    handler.dispatch(event)
  File "/usr/local/lib/python2.7/dist-packages/watchdog/events.py", line 446, in dispatch
    self.on_any_event(event)
  File "/usr/local/lib/python2.7/dist-packages/boussole/watcher.py", line 173, in on_any_event
    self.index()
  File "/usr/local/lib/python2.7/dist-packages/boussole/watcher.py", line 91, in index
    library_paths=self.settings.LIBRARY_PATHS
  File "/usr/local/lib/python2.7/dist-packages/boussole/inspector.py", line 106, in inspect
    self.look_source(sourcepath, library_paths=library_paths)
  File "/usr/local/lib/python2.7/dist-packages/boussole/inspector.py", line 79, in look_source
    self.look_source(path, library_paths=library_paths)
  File "/usr/local/lib/python2.7/dist-packages/boussole/inspector.py", line 63, in look_source
    finded_paths = self.parse(fp.read())
  File "/usr/local/lib/python2.7/dist-packages/boussole/parser.py", line 134, in parse
    self.remove_comments(content)
  File "/usr/local/lib/python2.7/dist-packages/boussole/parser.py", line 83, in remove_comments
    return self.REGEX_COMMENTS.sub("", content)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 102: ordinal not in range(128)

This happens on the first change, even if the file is empty without any comment. Maybe i introduced the bug during porting to Python3 ?