modelmat / sphinxcontrib-drawio Goto Github PK

There is no way to make sure that macos/windows pathing still works upon changes.

Included in the CI tests should be a sphinx project which gets built.
This will test that this package actually works.

Could probably integrate a test for #1 .

Describe the bug

{drawio-image} drawio/test.drawio
:format: pdf

Does not output any file. All other formats are producing files.

To Reproduce

{drawio-image} drawio/test.drawio
:format: pdf

then make html

Expected behaviour
According to documentation, a pdf file should be created.

Setup Information

• OS: Ubuntu 22.04.3
• Python: 3.10.12
• Version: 0.0.16

As of version 0.0.14, I get:

$ python3 -m pip show sphinxcontrib-drawio
Name: sphinxcontrib-drawio
Version: 0.0.14
Summary: Sphinx Extension to include draw.io files
Home-page: https://github.com/Modelmat/sphinxcontrib-drawio
Author: Modelmat
Author-email: [email protected]
License: MIT
Location: /usr/local/lib/python3.9/site-packages
Requires:
Required-by:

But on our build server:

$ python3 -m pip install -r requirements.txt

Results in:

      ...
      File "<frozen importlib._bootstrap>", line 991, in _find_and_load
      File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 848, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/tmp/pip-install-iei10cwa/sphinxcontrib-drawio/sphinxcontrib/drawio/__init__.py", line 16, in <module>
        from sphinx.application import Sphinx
    ModuleNotFoundError: No module named 'sphinx'
    ----------------------------------------

Which is plausible, since pip is permitted to install packages in any order if they don't depend on each other. I guess this is just a missing dependency.

Thanks a lot.

I have in my defaults drawio_output_format = "svg", and put :format: png in my directive. Still creates as a svg. I reverse it and same. it only follows what is in your sphinx defaults or conf.py settings

Nothing should be added to source directory during any sphinx build, this causes issues with CM and possible issues with clean builds. Should be using build folder which is usually configured to be wiped with make clean and is usually configured to be ignored under CM.

Workaround: add source/.drawio to .gitignore

It doesn't seem the extension converts drawio files one at a time, even when Sphinx is run in parallel mode. I thought adding "parallel_write_safe": True might help, but that doesn't seem to have an effect. Any ideas on how to convert drawio files in a parallel manner? On Sphinx projects with a large count of drawio diagrams this can become quite the performance bottleneck.

https://github.com/Modelmat/sphinxcontrib-drawio/blob/7155c36fa792b22553de6edf4da7e39c1ccd03d1/sphinxcontrib/drawio/__init__.py#L182

Mentions the issue 344 which is now fixed on later versions.

The current tagged commit system is slightly painful, as if the tags and commits aren't pushed at the same time it sometimes messed up and won't do a deploy.[1]

It would probably be beneficial to have a static version file containing the release version which gets used.

I'm not 100% sure on how we'd get the CI to automatically deploy to pypi [does it need to?] when it changes.

[1] I think, I haven't tested this extensively?
Does GitHub actions support this better?

We should keep a changelog so that users can see which features/bugfixes a new release brings.

Hi,

I have problems when running drawio within a gitlab-ci test setup. The following error occurs:

Running as root without --no-sandbox is not supported

The issue seems to be related to this issue one from the original drawio repository. A workaround seems to be to use '--no-sandbox --headless --disable-gpu'. These parameters are currently not used, right?

Or am I doing something wrong? This is the relevant snippet of my ci-file:

image: python:latest

build:
stage: build
script:

• apt update
• apt install -y wget
• apt install -y libgbm-dev
• apt install -y libasound2
• wget https://github.com/jgraph/drawio-desktop/releases/download/v13.9.9/draw.io-amd64-13.9.9.deb
• apt install -y -f ./draw.io-amd64-13.9.9.deb
• apt install -y xvfb
• pip install -r requirements.txt
• export SPHINXOPTS=-W # treat warnings as errors
• if [ $CI_COMMIT_REF_NAME != 'master' ] ; then export SPHINXOPTS="$SPHINXOPTS -t draft" ; fi
• make html

using ".. drawio-figure:: test.drawio" directive results in the following error (windows, whitespace in pathname)

DrawIO Error:
draw.io ("C:\Users\AB1234\OneDrive - ABC\repo\needs\docs\utils\draw.io-21.1.2-windows-no-installer.exe" --export --crop --page-index 0 --scale 1.0 --format png --output C
:\Users\AB1234\OneDrive - ABC\repo\needs\docs\build\html.doctrees\drawio\f7cee1ec9581edc5b575d34416e8fd53113b8808\test.png C:\Users\AB1234\OneDrive - ABC\repo
\needs\docs\source\snmp\test.drawio --enable-logging) exited with error:
[WinError 5] Access is denied

To Reproduce
whitespace in sphinx project name path

Setup Information

• OS: Windows
• Python: 3.10
• Version: 0.0.16

Additional context
when i add quotes to export and input path in init.py in drawio_args like this
....
line 261
# str(export_abspath),
# str(input_abspath),
f'"{str(export_abspath)}"',
f'"{str(input_abspath)}"',

then it is working fine

If :page-index: is outside of the number of pages in the document, it should raise an error. Currently, it falls back to the first page

EDIT: appears it falls back to the last page

This should be added and configured to test before it gets migrates to the main organization.

SVG/PNG and that

This is probably not a good idea.

Right now, starting Xvfb fails silently and an error is raised at the end of the build process when trying to kill it.

Additionally, have the location configurable

Need to find where this is.

drawio seems to support specifying the page to export when using multipage drawings.

https://github.com/jgraph/drawio/blob/abfe38fa94107509472e57ce6ee1731471b751bd/src/main/webapp/electron.js#L230

Might be an idea to add it as a parameter to the drawio directive.

I recently ran into an issue where someone had permission issues with Xvfb and it couldn't start up. It seems this line caused an infinite loop in that case, but it was hard to tell what the issue was at first:

I'm not quite sure the best approach, but do you think it would be a good idea to add a counter in the loop and output a "Xvfb is having difficulty starting" log message if it goes over a generous amount? I also don't know if it's possible (or a good idea) to kill the Sphinx process at the same time.

Occasionally we get a random failure (random in that we can re-run the build and it won't fail the next time), but the [stderr] and [stdout] is empty making it tricky to troubleshoot.

DrawIO Error:
draw.io (/opt/drawio/drawio --export --crop --page-index 0 --scale 1.0 --transparent --format png --output /path/to/image.png /path/to/image.drawio --no-sandbox) exited with error:
[stderr]
b''
[stdout]
b''

It looks like that output comes from these lines of code, do you think the reason they are blank has something to do with the subprocess.run() arguments or subprocess.CalledProcessError? I'm mainly trying to figure out what might be a good place to troubleshoot to find if there is stderr/stdout that is getting lost somewhere.

(Sorry about opening all these issues recently, I love this extension!)

os: archlinux
electron: 8.2.3
drawio: 13.0.1 (both aur/drawio-desktop and aur/drawio-desktop-bin packages)
python: 3.8.2
sphinx: 3.0.3
sphinxcontrib-drawio: 0.0.6

Аргументы командной строки drawio стал обрабатывать неожиданным образом.
Аргумент, предназначенный для electron, не изымается после обработки, а попадает в список неизвестных, становясь на место имени входного файла:
paths: ['--no-sandbox', 'my_file.drawio']
Это порождает Error: input file/directory not found при сборке документации.
Удаление '--no-sandbox' из drawio_args разрешило проблему.

CLI args began to be handled unexpectedly by drawio.
The argument that was intended for the electron is not removed after processing.
It shifts the input file name position, which generates a file search error.
I had to remove the '--no-sandbox' argument from drawio_args for a successful doc build.

Waiting on #38

Steps to reproduce:

• Build a Sphinx project using the rinoh builder
• (Sphinx caches the parsed input files in the doctrees directory)
• Build the same Sphinx project using the html builder
• (Sphinx will read the cached doctrees which reference the PDF images instead of the SVG images)

I think the proper way to handle this would be to subclass ImageConverter and handle conversion as a post-transform. The drawio exporting should be moved from DrawIOBase to this new class. sphinx.ext.imgconverter is an example of how ImageConverter can be subclassed.

A workaround is to use separate doctrees directories for each builder.

For step 3 in the Installation section "Add the binary to $PATH", can that be "Add the binary to $PATH or configure drawio_binary_path"? If so, I can make a PR to add that for clarity.

It seems to be uploading the release twice? No idea why

A </div> seems to be missing here:

https://github.com/Modelmat/sphinxcontrib-drawio/blob/5146c3010a1534f32a695f6352558119bd8076fe/sphinxcontrib/drawio.py#L165

This argument to drawio was added in 1da2e44 and later removed in 61d0f19 because it didn't have any effect. This argument is however required when running drawio inside a docker container and needs to be at the end of the argument list (see jgraph/drawio-desktop#249 (comment)). Can this argument be added again, possibly controlled by a configuration variable?

Tests were disabled, see 3418d45 (#62)..

Just wanted to say you should consider adding the sphinx-extension topic, https://github.com/topics/sphinx-extension, to the repo so it is easier for others to find on the GitHub ecosystem. There are currently 88 other sphinx extensions that use that topic.

sphinxcontrib is a namespace package shared by multiple distributions. I think you should not add an __init__.py in it. I believe this causes the import error on the tag 0.0.11 (released as 0.10.0).

See the section Native namespace packages:

It is extremely important that every distribution that uses the namespace package omits the init.py or uses a pkgutil-style init.py. If any distribution does not, it will cause the namespace logic to fail and the other sub-packages will not be importable.

When using this extension together with the extension sphinx-imgconvert I get an error due to the fact that at the following lines:

https://github.com/Modelmat/sphinxcontrib-drawio/blob/1dcdb8970c5a498db3cf1b4a54d52014684c3892/sphinxcontrib/drawio/__init__.py#L160-L165

the candidates and uri properties of the node are set to a Path object created in the _drawio_export function.

I'm not an expert of the docutils/sphinx codebase but I think that Docutils is setting uri for figure nodes to a string here:

https://github.com/docutils-mirror/docutils/blob/e88c5fb08d5cdfa8b4ac1020dd6f7177778d5990/docutils/parsers/rst/directives/images.py#L29

and Sphinx is setting the default candidate to a str here:

https://github.com/sphinx-doc/sphinx/blob/1285e0cabed517d4bf24b00c5fd68804a26e8a91/sphinx/environment/collectors/asset.py#L83

It seems therefore that sphinx-imgconvert is correct in expecting a string and not a Path object.

I think the extension could be adapted by casting the Path object to a str in these two locations:

https://github.com/Modelmat/sphinxcontrib-drawio/blob/1dcdb8970c5a498db3cf1b4a54d52014684c3892/sphinxcontrib/drawio/__init__.py#L206

https://github.com/Modelmat/sphinxcontrib-drawio/blob/1dcdb8970c5a498db3cf1b4a54d52014684c3892/sphinxcontrib/drawio/__init__.py#L272

This would be a more logical location for the exported images. Some thoughts:

• maybe you don't want it inside the output folder (e.g. build/html) because it makes sense to keep it when deleting the built documentation
• perhaps the location can be made configurable (like the doctrees directory)?

Global config and local config.

Seems good enough now and saves an additional account.

See #21 for the azure version.

I've a drawio file in an "sphinx" md file.
`
```{drawio-figure} drawio/my_graph.drawio
:alt: My graph
:align: center

My graph
```
`

On default this graph is tranformed as a svg file.

I'd like to make it clickable to download this graph as a pdf.

How could I do that ?
I could call drawio by hand, store the pdf file by hand, .... Is there any "all-in-one" solution, so that sphinxcontrib-drawio creates both file at "once" ?

Thx for your support...

I didn't find any other way to ask my question (The Discussion tab is not available).

Is your feature request related to a problem? Please describe.
I would like to export certain layers to an SVG.
draw.io supports layers, and draw.io CLI allows to specify the layers that shall be exported.

Describe the solution you'd like
sphinxcontrib-drawio should allow to specify the layers that shall be exported, and pass this to draw.io CLI.
It should be similar to --page-index but as a comma separated list.
If the parameter is not specified then it shall not be passed to draw.io in order to export all visible layers (default behavior).

Describe alternatives you've considered
At the moment I export the SVG manually from draw.io, with the desired layers manually made visible.
It does not work well if multiple exports with different layers of the same file are needed.

Additional context
Layer selection in CLI has been added in draw.io v20.6.0.

draw.io counts the layers from 0 where layer 0 is the one at the bottom of the list, usually the Background layer.

draw.io CLI:

-l, --layers <comma separated layer indexes>
selects which layers to export (applies to all pages), if not specified, all layers are selected

By default, drawio_headless is True. But that triggers errors on windows.

Autodetecting windows and setting drawio_headless to False removes one troubleshooting step for users on windows.

As far as I can tell, drawio crops the figure when exporting to formats other than PDF. It does offer the --crop command line option to also do this for PDF. Should we make use of it when exporting to PDF? I believe it makes sense to make this behavior consistent regardless of the output format (unlike drawio itself).

sphinxcontrib-drawio is an excellent tool, extremely handy when creating images etcetera for documentation. We use Sphinx since it allows us to create documents in different formats using the same source code (*.rst files).

Discovered a minor issue, when exporting images in svg format they look fine in documents exported as HTML whereas the very same images in documents exported as PDF will have a small text stating “Text is not SVG – cannot display”. When exporting images manually from draw.io in svg format and then using those when creating the PDF works fine.

We use simplepdf to create PDF files from .rst source files.

It is not clear to me what settings are used in sphinxcontrib-drawio when exporting in svg format but the only thing I can think of that might cause this is that images and text are not embedded when exporting? Would it be possible to add such a feature?

How it looks in the PDF file when using sphinxcontrib-drawio.

How it looks like in the PDF file when exporting images manually in draw.io and then include those images in the rst files.