That's really neat @chrisjsewell. I love the code completion and formatting. Now I look at my own set up and feel like I'm working in the dark ages.

More generally, I like the whole Myst approach a lot. If I look at @chrisjsewell 's example above where he is comparing the metadata block

{python id, echo=True, include=True, fig.cap="A figure caption", dpi=196}
print("hello")

against the more verbose rst version

.. exec-block:: python
    :label: id
    :echo: true
    :include: true
    :fig.cap: "A figure caption"
    :dpi: 196

    print("Hello")

I actually prefer the rst version because it's more readable. Yes, it's a bit more effort to write, but that's where a good editor extension comes in.

I also think that, if you want to make rst acceptable to markdown users, there's not that much you actually need to do, and the current Myst proposal already covers those bases.

Just my personal 2 cents...

from myst-parser.

if I remember correctly, @mmcky has already merged support for https://www.sympy.org/sphinx-math-dollar/.

We have added this sympy package as a dependency to the jupinx tool -- along with adding it to the quickstart.

Perhaps it would be possible to shift Jupinx over to Myst?

I think the best approach here would be to formally define myst as a specification (i.e. extensions to core rST. Then we could figure out if we support this in sphinxcontrib-jupyter either as an option or as the default; or try upstream.

• review myst and generate a specification (@mmcky) to assist further discussion.

I've heard others say that the docutils codebase is hard to work with. But that should be confirmed :)

Confirmed :-) -- had to dig in a few times to see if something is implemented in sphinx or docutils. But in the end of the day at least it is python code.

from myst-parser.

Anyone familiar with ASCIIDOC? Here is a cheatsheet:

https://www.reddit.com/r/Sacramento/comments/dj9ucd/sac_rt_proposes_routes_to_smf_and_ucd/

It looks similarly complete as RST but maybe the syntax is preferable?

from myst-parser.

Correct link: https://powerman.name/doc/asciidoc

from myst-parser.

thanks @choldgraf that will be really interesting to take a look at. I will do so this week.

from myst-parser.

cool - would love to hear your thoughts. the myst project is still very young with not a lot of specifics or code, mostly just conversations and ideas, so please do chime in with thoughts/changes/etc.

either way it may be a project worth moving forward regardless of the markdown conversations, a "slightly better" rST syntax would be really useful for the python world at the least.

from myst-parser.

In fact Jupinx is inching in that direction already -- if I remember correctly, @mmcky has already merged support for https://www.sympy.org/sphinx-math-dollar/.

Perhaps it would be possible to shift Jupinx over to Myst? What do you think @mmcky?

We would need to have some RAs working for us already, so we could shift the content at the same time.

from myst-parser.

Neat idea. I think RST is superior to markdown in many ways. There are a few things about the syntax that I always have to look up, but that's my only complaint. RST is pretty feature complete for technical writing. I wrote 400+ page dissertation in RST and only needed one extension to get bib file support so the reference list didn't have to be manually dealt with.

I'm not sure how different Myst is, but why not try to submit upstream changes to RST to support more options for simpler syntax?

from myst-parser.

I'm not sure how different Myst is, but why not try to submit upstream changes to RST to support more options for simpler syntax?

I don't think we got to the point of actual implementation etc, so we never really got to this question. But my impression is that one challenge of rST is that the implementation is extremely difficult to understand and has been fairly stagnant for many years now w/o a clear maintainer pushing it forward (this mirrors some of what @chrisjsewell mentioned about docutils more generally). That's more anecdotal than anything, though if it seems like there's a path to improving rST with upstream improvements, I agree that would be better.

from myst-parser.

implementation is extremely difficult to understand

I've heard others say that the docutils codebase is hard to work with. But that should be confirmed :)

from myst-parser.

@mmcky will also have some useful thoughts on this (context for @moorepants is that @mmcky is lead developer of jupinx, which is built on top of RST).

(PS I like your website @moorepants and I believe your mother.)

from myst-parser.

(PS I like your website @moorepants and I believe your mother.)

Ha!

from myst-parser.

My two cents worth on this is that:

As I've mentioned before, happy to use sphinx as the 'document engine', but it would be nice if the docutils API were improved 😃

But why not try to submit upstream changes to RST to support more options for simpler syntax?

I'm very doubtful this would ever happen. RST is a fairly well entrenched syntax at this point, and I imagine the authors would be very sceptical about how to achieve this, in a manner that is 'guaranteed' not to cause side-effects for existing users of the format.

myst is more like "can we make rST more friendly" rather than "can we extend CommonMark to support publishing".

Myst, to me, is basically asking for a mapping of certain Markdown elements to docutils elements which, to some extent recommonmark already does. What you really want is a hybrid parser of docutils.parsers.rst and recommonmark.

Looking at sphinx-math-dollar, for example, it is a regex-replace of dollar bracketed text with docutils.nodes.math, that is applied as a post processor (aka transform) to docutils.parsers.rst. But ideally it would just be applied during the parsing.

You could do this for pretty much every bit of Markdown syntax. Not only, the ATX Headers and link-syntax, that Myst mentions, but also the front-matter and RMarkdown chunks:
Similar to the discussion in executablebooks/meta#9, regarding YAML metadata blocks, general users might prefer using:

```{python id, echo=True, include=True, fig.cap="A figure caption", dpi=196}
print("hello")
```

Than the more verbose:

.. exec-block:: python
    :label: id
    :echo: true
    :include: true
    :fig.cap: "A figure caption"
    :dpi: 196

    print("Hello")

Again you could regex-replace the RMarkdown chunks with docutils directive elements (similar to what I have done in executablebooks/meta#12 process_doc; regex-replacing the chunks with pandoc divs), or handle this directly during the parsing.

At this point, you could effectively write your whole document as RMarkdown, adding in RST elements where needed.
(It's actually just a bit of an inversion to what I do in IPyPublish; converting RST elements to pandoc AST.)

Note, this hybrid document should not really have the .Rmd or .rst extension, since it cannot be directly used by the generic parser of either (it would use .mst or whatever).

from myst-parser.

I really like these ideas. If you used them to

• make Jupinx function with Myst and
• implemented jupytext mirroring of mst files and notebooks

then you would already have a highly usable system that's attractive to a range of users and a good benchmark for whatever we build next.

from myst-parser.

Anyone familiar with ASCIIDOC? Here is a cheatsheet:

https://www.reddit.com/r/Sacramento/comments/dj9ucd/sac_rt_proposes_routes_to_smf_and_ucd/

Don't know about asciidoc, but I'm all in favour of better transport to SAC IA 😂

from myst-parser.

Ha! I'm also working on a grass roots campaign here and had that in my clipboard :)

from myst-parser.

BTW, if you did want to down the 'hybrid RST/RMarkdown' route, I added some mixed syntax highlighting and code-folding to my test VS Code extension:
https://marketplace.visualstudio.com/items?itemName=chrisjsewell.vscode-imarkdown-tm-grammar#example

from myst-parser.

FYI just a quick update on the RST LSP.

• It works by extending the docutils/sphinx parser, to capture data about the position of elements in the document, then extracting this and other data (reference/target links, etc) to a database, for fast lookup. This could be relatively easily extended to incorporate the 'Myst syntaxes'.
• The server kernel is entirely client agnostic, so will work with any editor that supports LSP (VS Code, EMacs, Jupyter Lab, etc).
• Each 'feature' is a pluggy plugin, making it very easy to extend (e.g. to provide autocompletion for languages other than python).
• Also, it would be entirely plausible to 'execute' the code-blocks (in the same way formatting is applied in the 'Code Blocks' example) and preview the outputs; in essence turning the document into a pseudo notebook.

Outline & Diagnostics

References and Definitions

Autocompletion

Code Blocks

from myst-parser.

Pretty nifty - I like LSPs :-)

Just to clarify, building something like Myst would basically require us to re-implement the rST processing, no? Or would it be something we could piggy-back on top of the rST parser in Python? (e.g. a lightweight Myst -> rST conversion under the hood?)

from myst-parser.

Yep piggy backing, by sub-classing a number of the docutils classes to form a new parser. There are some upstream improvements I'd also like to make to docutils, to make this easier, but its such a pain using sourceforge! I don't know why they don't move to GitHub, but they've specifically said that there are no plans to do this: https://sourceforge.net/p/docutils/feature-requests/58/ 😒

from myst-parser.

nice - that does sound like a nice first-start. It's too bad that the underlying code-base is such a PITA to work with, I think some jupyter folks looked into that a while back and that was ultimately the blocker :-/

I guess if worst came to worst, one could re-implement pieces of the parser, but that also sounds like a lot of work

from myst-parser.

I wonder if there is a way to make CommonMark markdown also automatically Myst syntax compliant? This would be a huge deal for the literally tens of millions of Jupyter Notebooks already in existence...

from myst-parser.