Hi everyone,
As you might know, m2r2 is a fork of m2r. I forked it, fixed some code and deployed it to pypi out of personal need and lack of response on the original repo. Some people found it useful as well, so, here we are now.

Lately, I haven't got much time other than for university and work. PR #14 was stalled for no other reason than me not merging it, and I apologize. That got me thinking that I wouldn't like m2r2 to suffer the same fate as it's predecessor. In order to remove the SPOF (me and my schedule) I'm opening this.

Feel free to send me an email (in my profile) and let's talk. I don't think the project needs any crazy requirements tech-wise, so if you know some python, can work with other people and want to teach/learn stuff, you are most welcome.

The regular reST include directive has support for :start-after: and :end-before: options allowing specific to text to search for for setting the start/end points of the inclusion.

This is extremely useful since it allows including, say, only specific sections of a markdown file, without always having to chase the exact line numbers. Happy to make a PR since I see you're looking for help on this package.

Hello, Team!

I got an error when I try to run pip install -r requirements-dev.txt:
ReqDevInstallDoesNotWork

Can you please support?

| col 1 | col 2  |
| ----------- | ----------- |
| row 1: | [![some status image](https://some/Badge)](link) |
| row 2: | [![some status image](https://some/Badge)](link) |

gets translated to

   * - col 1
     - col 2
   * - row 1:
     - 
     .. image:: https://some/Badge
        :target: link
        :alt: some status image
   * - row 2:
     - 
     .. image:: https://some/Badge
        :target: link
        :alt: some status image

which should be

   * - col 1
     - col 2
   * - row 1:
     - .. image:: https://some/Badge
        :target: link
        :alt: some status image
   * - row 2:
     - .. image:: https://some/Badge
        :target: link
        :alt: some status image

This results in an error and the table is not gettings displayed, when converting manually to second variant it works.

I am not sure if second variant is like it really should be, but it looks ok and there are no errors when building docs anymore

Actually I will just not display the badges in tables anymore, but just so this issue is known :)

There have been changes in argparse output https://bugs.python.org/issue9694 which breaks https://github.com/CrossNox/m2r2/blob/development/tests/test_cli.py#L61.
There is a pull request in m2r which updates it miyakogi#62, it applies cleanly to m2r2. Ive tested with python3.9 and python3.10 with both passing tests.

Would you mind bringing .. only:: directive support, please? It is described here.

Currently, if you include a bold hyperlink using text the text is not being parsed correctly.
See evidence:

Currently, the links to internal sections of the README are not working on the docs generated by Sphinx.

When including a document that contains multiple headings with exactly the same names, such as a changelog like this:

# Version 1
## Bugfixes
- A
- B
- C

# Version 2
## Bugfixes
- A
- B
- C
## Features
- X
- Y

The h2 headings are duplicated, and so duplicate label warnings appear.

A potential solution to this could be to replicate what MyST is doing, and use placeholder ids like #id1, #id2, etc. for any such duplicate labels, to allow referencing any title easily, without any conflicts.

i.e. the final version shouldn't look like this:

But rather like this:

Really thank you for providing this library.
I hope this will overcome my usage issues with recommonmark also in the long run.

It would be helpful, if .. mdinclude:: /_static/whatever.md would always starts looking on the doc root level (where sphinx-build command is executed) when reference starts with /.
This would match the behaviour of sphinx's ..include: directive.

In markdown, one can do format the text of a link without any problems

[`my.api.method`](http://example.com)

However, from this m2r2 generates

``my.api.method` <http://example.com>`_

This is not valid rST and just renders literally.

We'd like to generate the following HTML:

<a href="http://example.com"><code>my.api.method</code></a>

This isn't really the "fault" of m2r2, as rST doesn't really support this construct. A workaround is here. Perhaps m2r2 could implement this workaround? Seems pretty messy unfortunately.

In the last version of docutils (0.19), the ErrorString class has been removed and this project has no fixed version for docutils so that causes crashes.

Fix proposal: Fix the version of docutils to the 0.18

Hello and thanks for the inherited maintaining of m2r!
I try to use sphinx for automatic html document generation.
Now I like to use markdown to write my README.md.
I thought I can use m2r2 to embed the readme into the index.rst of sphinx.
So I did the following steps:

• inserted the extension at conf.py
• added the .md at source_suffix
• added .. mdinclude::../README.md

But unfortunately nothing happens and the rst remains the same then before. But it got changed if I add some other input into the original rst.
Does someone have a clue what can cause this?

Thanks for the support in advance

v0.3.3 is incompatible with sphinx-rtd-theme 1.0.0. Because of ReadTheDocs this is probably the most popular theme for sphinx.

The conflict is caused by:
    m2r2 0.3.3 depends on docutils>=0.19
    sphinx 5.1.1 depends on docutils<0.20 and >=0.14
    sphinx-rtd-theme 1.0.0 depends on docutils<0.18

Hi,

I've been using python-semantic-release for generating the Changelog markdown files, but apparently m2r2 is not correctly converting to RST.
Example from Changelog.md

* Update setup.py ([`2513d97`](https://gitlab.com/felipe_public/badges gitlab/-/commit/2513d970f9b055675de2f5479856604a7774f430))

Result after m2r2 inclusion in sphinx

Update setup.py (``2513d97` <https://gitlab.com/felipe_public/badges-gitlab/-/commit/2513d970f9b055675de2f5479856604a7774f430>`_)

I believe insted of converting to double quotes (for .rst) it is missing the second double quotes?

Am I missing something?

Regards,

When I use multiple jobs in conjunctions with m2r2 I receive the following exception during pickling the environment:

pickling environment... failed

Traceback (most recent call last):
  File "/home/marscher/miniconda3/envs/rtd39/lib/python3.9/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/home/marscher/miniconda3/envs/rtd39/lib/python3.9/site-packages/sphinx/application.py", line 343, in build
    self.builder.build_update()
  File "/home/marscher/miniconda3/envs/rtd39/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 293, in build_update
    self.build(to_build,
  File "/home/marscher/miniconda3/envs/rtd39/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 324, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
AttributeError: Can't pickle local object 'Domain.role.<locals>.role_adapter'

Without m2r2 it works as expected (multiple jobs), but of course the markdown conversion breaks down. E.g. I simply comment out the m2r2 extension.

I will try to reproduce this with a minimum working example once I have the time to do it. I would have to break down a rather complex setup. In the meantime I provide the complex failing example https://github.com/BAMWelDX/weldx/tree/master/doc

In place of the relative file path to the md in the .. mdinclude:: file directive, is there a way we can provide a link to the md and render the md in the rst.

sdist tar ball does not contain docs/directory so I've been trying to use tagged tar ball to build new as rpm package but it fails

+ /usr/bin/python3 setup.py build '--executable=/usr/bin/python3 -s'
Traceback (most recent call last):
  File "setup.py", line 14, in <module>
    from m2r2 import parse_from_file
  File "/home/tkloczko/rpmbuild/BUILD/m2r2-0.3.1/m2r2.py", line 19, in <module>
    __version__ = get_distribution("m2r2").version
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 466, in get_distribution
    dist = get_provider(dist)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 342, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 886, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 772, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'm2r2' distribution was not found and is required by the application
error: Bad exit status from /var/tmp/rpm-tmp.2yI2j1 (%build)

I see first time ever such effect.
May I ask for some hint/help?

I am try to convert the following code (sample function, replacing back-ticks with apostrophes )

Markdown

''' python
def sample(args):
    return args[0]
'''

( Other docs here )

to Rst

... code-block:: python
    def sample(args):
        return args[0]

(Other docs here)

But the rst looks like this
rst

```python
    def sample(args):
        return args[0]

... code-block:: python
    (Other docs here)

Do you know what is going on? I looked into your code and saw that there is the function block_code that helps convert a markdown code block to a rst code block, and I see that m2r2 supports shell or bash syntax highlighting, but for some reason it's not working with python. Do you know what is going on?

It looks like there a number of breaking changes in the latest mistune:
lepture/mistune#290

I'm not sure if tracking the current branch is in scope or not.

Would it be possible to add support for github style callouts in markdown and have them render as sphinx admonitions?

Currently if I have a markdown document defined like this:

# Some Section

Some paragraph

> :warning: Some warning message

On github markdown, it renders like this with the 'warning' icon:

Some Section

Some paragraph

⚠️ Some warning message

However when using the m2r2 to convert the markdown, it treats this as a parameter listing like this:

Some section
============

Some paragraph

..

   :warning: Some warning message

Which in the sphinx compiled html output, it renders like this:

Ideally it would be nice if this could support github style callouts like this and render it as a sphinx warning/note/etc directive instead like this:

Some section
============

Some paragraph

.. warning::

   Some warning message

Thanks again for m2r2 and I hope it's okay to make requests here?

What I want to do

In my .md file I'm making use of collapsible blocks that makes the document much more readable:

<details><summary>Expand (CLICK ME)</summary>
<p>

Large Markdown formatted text:
- Many lists

</p>
</details>

Which turns on GitHub into:

Issue

When turning the above Markdown into HTML with Sphinx, it just becomes a literal string.
Checking the output of m2r2 README.md, we can see that everything is wrapped into a raw html wrapper:

.. raw:: html

   <details><summary>Expand (CLICK ME)</summary>
   <p>

   Large Markdown formatted text:
   - Many lists

   </p>
   </details>

Related issue

The problem might be fundamentally the same as encountered by this GitHub Pages user:
https://github.community/t/collapsible-markdown-inside-details-summary-summary-details-fails-to-render/10489

The solution for them was to set a different markdown parser: markdown: CommonMarkGhPages

Request / Possible solution

• Automatically detect that <details><summary> has been used, and only wrap that part into .. raw:: html OR
• Allow to manually specify that m2r2 should parse the contents inside collapsible HTML code OR
• Some other approach

Sorry for filling up your issues list. I am not an experienced python coder, so I am kind of wondering, what is possible for m2r2 in regards of supporting Sphinx provided features.

I would like to see for instance the rst_prolog feature to work, because it would help a lot with DRY documentation.

If it is not possible to provide that support or you are not planning so, it is okay. I just would like to get a clear understanding if I can count on m2r2 and build my use-case around it, or whether I should look for another approach.

Hi,

It looks like the new version on the 'mistune' (2.0.0 released Dec 5, 2021) package has different API and it throws AttributeError: module 'mistune' has no attribute 'BlockGrammar'
Previous version (0.8.4 released Oct 18, 2018) looks correct and works fine.

`

Running Sphinx v4.1.2

Exception occurred:
File "/home/osboxes/proba/taf_inherited_members/venv/lib/python3.9/site-packages/m2r2.py", line 83, in
class RestBlockGrammar(mistune.BlockGrammar):
AttributeError: module 'mistune' has no attribute 'BlockGrammar'
The full traceback has been saved in /tmp/sphinx-err-3daao_tt.log, if you want to report the issue to the developers.
`

Pipdeptree shows that this is a dependency of m2r2 package:

`

m2r2==0.3.1

• docutils [required: Any, installed: 0.18.1]
• mistune [required: Any, installed: 2.0.0]
  pipdeptree==2.2.0
• pip [required: >=6.0.0, installed: 21.3.1]
  setuptools==58.3.0
  wheel==0.37.0
  `

Full sphinx log:
`

Sphinx version: 4.1.2
Python version: 3.9.5 (CPython)
Docutils version: 0.17.1 release
Jinja2 version: 3.0.3
Last messages:

Loaded extensions:
Traceback (most recent call last):
File "/home/osboxes/proba/taf_inherited_members/venv/lib/python3.9/site-packages/sphinx/cmd/build.py", line 276, in build_main
app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
File "/home/osboxes/proba/taf_inherited_members/venv/lib/python3.9/site-packages/sphinx/application.py", line 237, in init
self.setup_extension(extension)
File "/home/osboxes/proba/taf_inherited_members/venv/lib/python3.9/site-packages/sphinx/application.py", line 393, in setup_extension
self.registry.load_extension(self, extname)
File "/home/osboxes/proba/taf_inherited_members/venv/lib/python3.9/site-packages/sphinx/registry.py", line 429, in load_extension
mod = import_module(extname)
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 855, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/osboxes/proba/taf_inherited_members/venv/lib/python3.9/site-packages/m2r2.py", line 83, in
class RestBlockGrammar(mistune.BlockGrammar):
AttributeError: module 'mistune' has no attribute 'BlockGrammar'
`

The recent sphinx 3.3.0 release has started failing docs builds. It raises:

Exception occurred:
  File "/home/runner/work/retworkx/retworkx/.tox/docs/lib/python3.8/site-packages/sphinx/registry.py", line 266, in add_source_parser
    for filetype in parser.supported:
AttributeError: 'str' object has no attribute 'supported'

https://github.com/Qiskit/retworkx/pull/166/checks?check_run_id=1342230330#step:7:16

Looking at the debug logging locally the error is coming from m2r2 when trying to add the source parser.

Hello!

I was wondering if there is any way to keep links that were created in an md file to the rst file. In my example I have a "Details" section in which there are links to another source (image_flop) and when these files are converted from md --> rst files the weblink breaks and the output is this:

I was wondering if there is any way to keep these links from breaking and "transfer" them as I convert my md files to rst files?

Desired outcome:

Here is my repo: https://github.com/sistia01/Giotto/tree/suite/docs

Here is an example: https://github.com/sistia01/Giotto/blob/suite/docs/source/subsections/md_rst/createGiottoImage.rst

I might have missed it, but are the include options such as :start-after: not available when working in sphinx?

https://docutils.sourceforge.io/docs/ref/rst/directives.html#include

Great work btw!

Our regular Jenkins pipelines to generate documentation have started failing due to a dependency issue caused by a new release of docutils:
view change history here

• docutils/nodes.py
  • Remove compatibility hacks nodes.reprunicode and nodes.ensure_str().

the following error is raised:

2024-04-17 16:32:18  Exception occurred:
2024-04-17 16:32:18    File "/data/jenkins/workspace/gda-project_gda-bb-utils_main/venv/lib/python3.10/site-packages/m2r2.py", line 611, in run
2024-04-17 16:32:18      path = nodes.reprunicode(path)
2024-04-17 16:32:18  AttributeError: module 'docutils.nodes' has no attribute 'reprunicode'

m2r2 only states a minimum requirement for docutils, and therefore my pipeline fails.
my pipeline exists of the following commands in case this can be of any help:

python3 -m pip install sphinx
python3 -m pip install sphinxcontrib-confluencebuilder
python3 -m pip install autoclasstoc
python3 -m pip install m2r2
sphinx-build -b confluence ./doc/source ./build/confluence

Unfortunately, I'm out of time for today, and if necessary I will add more information tomorrow!

Since miyakogi@eeb618b m2r supports inline math with dollar signs inside backticks, but gitlab uses backticks inside dollar signs.

We would like to render the same document with gitlab and with sphinx, and so would like m2r2 to support the gitlab way.

I used the extension previously with Sphinx=4.2.1 which worked as expected. After upgrading to Sphinx=5.1.0 and not changing anything else, the extension cannot be imported anymore. Though, I am not sure if this could also be related to the confluence builder.

Hi! Thanks for building this fork!

I am asking for official python3.9 support.

And also: your classifiers are missing 3.7 and 3.8
https://github.com/CrossNox/m2r2/blob/development/setup.py#L57

When I try to convert the following .md string, there is some unexpected \ in convert result:

import m2r2

description = [
    "When `createMode` is set to `recover`, skip it",
    "Local Inbound profile names to which Inbound is allowed. Use ['*'] to allow inbound to all profiles. It's default value is ['*'].",
]
for item in description:
    print(m2r2.convert(item).strip())

Hi,

I just wanted to generate some documentation of my current project and came across the problem mentioned in the topic. The log generated by Sphinx contains these lines:

# Sphinx version: 4.1.2
# Python version: 3.6.9 (CPython)
# Docutils version: 0.17.1 release
# Jinja2 version: 3.0.1
# Last messages:

# Loaded extensions:
Traceback (most recent call last):
  File "/path/to/.virtualenvs/idme-JkYIOqhd/lib/python3.6/site-packages/sphinx/cmd/build.py", line 279, in build_main
    args.tags, args.verbosity, args.jobs, args.keep_going)
  File "/path/to/.virtualenvs/idme-JkYIOqhd/lib/python3.6/site-packages/sphinx/application.py", line 237, in __init__
    self.setup_extension(extension)
  File "/path/to/.virtualenvs/idme-JkYIOqhd/lib/python3.6/site-packages/sphinx/application.py", line 393, in setup_extension
    self.registry.load_extension(self, extname)
  File "/path/to/.virtualenvs/idme-JkYIOqhd/lib/python3.6/site-packages/sphinx/registry.py", line 429, in load_extension
    mod = import_module(extname)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/path/to/.virtualenvs/idme-JkYIOqhd/lib/python3.6/site-packages/m2r2.py", line 83, in <module>
    class RestBlockGrammar(mistune.BlockGrammar):
AttributeError: module 'mistune' has no attribute 'BlockGrammar'

I do not know if this might be related to m2r2 or Sphinx, however, whenever I enable m2r2 as extension in Sphinx, I get this error.

Regards, Thomas

I have a README.md some others (CONTRIBUTING.md INSTALL.md) in the root of my project. They link to each other. In the sphinx built documentation these relative links don't work because it links to href="INSTALL.md" instead of href="INSTALL.html"

Any way to fix this?

This package currently uses pkg_resources module from setuptools package:

from pkg_resources import get_distribution

__version__ = get_distribution("m2r2").version

This is not part of the standard library and thus must be declared in setup.py.
As an example, if you are using the Poetry build system, you will not have setuptools by default.
In this case, your doc build will crash in a way that is in my experience hard to understand for lots of users.
Recommended to set lower bounds for the package only, if at all to avoid breaking other people's stuff.

Version 0.2.5

I'm not too sure what I've done, this package used to work but now I get this error:

WARNING: Unknown directive type "mdinclude".

I can still convert files without any problem, but when I run sphinx (3.3.1) I get this error when using the mdinclude directive.

Details about my install

$ poetry show m2r2
name         : m2r2
version      : 0.2.5
description  : Markdown and reStructuredText in a single file.

dependencies
 - docutils *
 - mistune *

I tried removing and adding m2r2 back but nothing seems to work.

How does m2r2 normally registers this new directive?

Working Markdown example:

Here is some math that will work: `$x^2$`
This math will render, too: `$z^3$`

Generated RST:

Here is some math that will work: :math:`x^2`
This math will render, too: :math:`z^3`

Broken Markdown example:

But only the first math (`$x^2$`) in this line will render. This one (`$x^2$`) will not.

Generated RST:

But only the first math (\ :math:`x^2$`) in this line will render. This one (`$z^3`\ ) will not.

Hello!

Thank you for updating this tool! I added m2r2 to my extensions list in my project (here) but when I built the website via the readthedocs host it caused a fatal crash error:

ModuleNotFoundError: No module named 'm2r2' -- see below for more information on the error.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/giottoteam-giotto/envs/suite/lib/python3.7/site-packages/sphinx/cmd/build.py", line 279, in build_main
args.tags, args.verbosity, args.jobs, args.keep_going)
File "/home/docs/checkouts/readthedocs.org/user_builds/giottoteam-giotto/envs/suite/lib/python3.7/site-packages/sphinx/application.py", line 245, in init
self.setup_extension(extension)
File "/home/docs/checkouts/readthedocs.org/user_builds/giottoteam-giotto/envs/suite/lib/python3.7/site-packages/sphinx/application.py", line 402, in setup_extension
self.registry.load_extension(self, extname)
File "/home/docs/checkouts/readthedocs.org/user_builds/giottoteam-giotto/envs/suite/lib/python3.7/site-packages/sphinx/registry.py", line 421, in load_extension
err) from err
sphinx.errors.ExtensionError: Could not import extension m2r2 (exception: No module named 'm2r2')

Extension error:
Could not import extension m2r2 (exception: No module named 'm2r2')

Any help with this would be appreciated!

Hello,
When a markdown document contains more than 1 link whose name is identical, m2r2 generates those links but fails to make them anonymous references.
According to sphinx-doc/sphinx#3921 this doesn't follow ReST specs.

How to reproduce:

% cat foo.md
[foo](https://foo.bar)
[foo](https://bar.foo)
% m2r2 foo.md
% cat foo.rst

`foo <https://foo.bar>`_
`foo <https://bar.foo>`_
% rst2html.py foo.rst > /dev/null
foo.rst:3: (WARNING/2) Duplicate explicit target name: "foo".

Thank you for keeping m2r alive!

I found this repo through your comment in this issue: miyakogi#51
In that same issue, I asked for a check to be added if .md is already registered. As this issue is also present in m2r2, I would like to ask the same here.

Full issue details:

If in conf.py, you have the following (order is unimportant):

extensions = [
    ...
    # .. mdinclude for Markdown documents
    'm2r2',
    # parse .md documents with sphinx
    'recommonmark',
]

Executing make html will raise the error:

make html
Running Sphinx v3.1.2

abs path: /some/path

Extension error:
source_suffix '.md' is already registered
Makefile:20: recipe for target 'html' failed
make: *** [html] Error 2

A possible solution?

Some modification to the code here?:
https://github.com/CrossNox/m2r2/blob/development/m2r2.py#L663-L667

Or can m2r2 and recommonmark not be used in the same documentation?

Related issues:

• readthedocs/recommonmark#191 (comment)
• sphinx-doc/sphinx#7000