Currently vim's :DiffOrig command starts a diff between the current modified buffer (which is a notebook converted to a markdown file using jupytext) and the on disk copy of the original .ipynb notebook. Can we make this plugin first convert the opened .ipynb file to markdown and then do a diff, when :DiffOrig is used?

When closing .ipynb files on Win 10, the plugin fails to delete the generated .md files. You can see the redacted log below:

DBG: filename: C:\path\to\file.ipynb
DBG: filename exists: 1
DBG: jupytext_file: C:\path\to\file.md
DBG: jupytext_file exists: 0
DBG: Generate file C:\path\to\file.md
DBG: cmd: jupytext --to=md --output="C:\path\to\file.md" "C:\path\to\file.ipynb"
DBG: [jupytext] Reading C:\path\to\file.ipynb in format ipynb^@[jupytext] Writing 'C:\path\to\file.md'
DBG: read C:\path\to\file.md
DBG: autocmd jupytext_ipynb BufUnload <buffer> call s:cleanup("C:\path\to\file.md", 1)
DBG: autocmd jupytext_ipynb BufWriteCmd,FileWriteCmd <buffer> call s:write_to_ipynb()
DBG: filetype: markdown
DBG: a:jupytext_file:C:pathtofile.md
DBG: deleting C:pathtofile.md

This happens because of the double quotes in s:cleanup("C:\path\to\file.md", 1): backslash is treated as an escape symbol. If there is no particular reason to use double quotes instead of single ones, changing 2 lines of code will solve the problem (at least it did for me).

I know that it's possible to set jupytext_fmt to convert the .ipynb to a given format, either, py, jl, md, etc. But I use both Julia and Python Jupyter Notebooks in my work. Is it possible to detect the notebook language and dynamically convert to .py or .jl?

I could convert to .md, but this would prevent me using https://github.com/jalvesaq/vimcmdline to run code in a repl.

I thought it might be possible to add support for reading metadata:language in the Notebook JSON, e.g.:

 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.6.2",
   "language": "julia",
   "name": "julia-1.6"

Is this a feature that already exists? Or could it be implemented?

When you set
let g:jupytext_filetype_map = {'md:myst': 'markdown'} let g:jupytext_extension_map = {'md:myst': 'md'} let g:jupytext_fmt = 'md:myst'
the new format isn't taken into account and opening a file yields an error about unrecognized format despite jupytext --to md:myst working

A clear and concise description of what the bug is.

Diagnostics

• vim --version: NVIM v0.4.4

• Operating system information (e.g. uname -a): SMP Debian 5.14.6-3 (2021-09-28) x86_64 GNU/Linux

• python --VV: Python 3.9.7 (default, Sep 24 2021, 09:43:00)

• Are you using Anaconda? No

• Does converting the notebook to/from ipynb with jupytext on the command line work? Yes

• Does it work when you set g:jupytext_command in ~/.vimrc to be the exact some jupytext that you used manually, with the exact same version of Python? Yes

When creating a notebook with vim and jupytext.vim (vim notebook.ipynb), if not any cell has a title when first writing the file (:w), then following writings of the file will make titles disappear. It does not happen if a title was present the first time the notebook was written.

What I call a title here is the short description of the cell that can be written next to # %% in a py:percent format.

# %% imports        <-- "title"
import os, sys

All of my configuration wrt jupytext is as follows:

Plug 'goerz/jupytext.vim'
let g:jupytext_fmt = 'py:percent'

Reproduce

1. vi notebook.ipynb, then type:

# %%
import os, sys

2. :w write the file
3. Add an header

# %% imports        <-- title added
import os, sys

4. :w write the file
5. :e reload the file, the header disappeared

Diagnostics

• Operating system information (e.g. uname -a):

$ uname -a
Linux IMT-21092020 5.11.0-38-generic #42~20.04.1-Ubuntu SMP Tue Sep 28 20:41:07 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

• python --VV:

$ python -V
Python 3.9.7

• Are you using Anaconda?

$ which jupytext
/home/p20aime/miniconda3/bin/jupytext

• Put let g:jupytext_print_debug_msgs = 1 in your ~/.vimrc. What is the output of :messages when reproducing the problem?

Writing the file the first time (:w) without header:

DBG: overwriting /tmp/python/nb3.py
DBG: Updating notebook from /tmp/python/nb3.py
DBG: cmd: jupytext --from=py:percent --to=ipynb --update '/tmp/python/nb3.py'
DBG: [jupytext] Reading /tmp/python/nb3.py in format py:percent^@[jupytext] Writing /tmp/python/nb3.ipynb
nb3.ipynb saved via jupytext.

Writing an header and writing the file (:w):

DBG: overwriting /tmp/python/nb3.py
DBG: Updating notebook from /tmp/python/nb3.py
DBG: cmd: jupytext --from=py:percent --to=ipynb --update '/tmp/python/nb3.py'
DBG: [jupytext] Reading /tmp/python/nb3.py in format py:percent^@[jupytext] Writing /tmp/python/nb3.ipynb (destination f
ile updated)
nb3.ipynb saved via jupytext.

• Does converting the notebook to/from ipynb with jupytext on the command line work? Does it work when you set g:jupytext_command in ~/.vimrc to be the exact some jupytext that you used manually, with the exact same version of Python?

When trying to replicate steps with command line it does work. That is weird because when doing through vim only then an inspection of the temporary .py file shows that it indeed contains the headers.

$ vi notebook.ipynb

$ jupytext --to py:percent notebook.ipynb
[jupytext] Reading notebook.ipynb in format ipynb
[jupytext] Writing notebook.py in format py:percent

$ cat notebook.ipynb
{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00069045",
   "metadata": {},                    <-- no title
   "outputs": [],
   "source": [
    "import os"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

$ vi notebook.py
# Adding a title to the cell

$ cat notebook.py
# %% imports
import os

$ jupytext --from py:percent --to notebook --update notebook.py
[jupytext] Reading notebook.py in format py:percent
[jupytext] Writing notebook.ipynb (destination file updated)

$ cat notebook.ipynb
{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00069045",
   "metadata": {
    "title": "imports"                    <- title
   },
   "outputs": [],
   "source": [
    "import os"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

$ vi notebook.ipynb
# I can see the title OK

Describe the bug

If the ipynb file is overwritten, for example if the user has an autosave autocmd, the update command fails.

• Does converting the notebook to/from ipynb with jupytext on the command line work?

No if using the --update argument, yes otherwise.

I'm unsure about the best way to go about this, but possibly, if jupytext fails to update, try again without the update? Potentially with a check for the ipynb file contents now being the same as the buffer?

Symptom

While opening an .ipynb file, an error will occur:

Error detected while processing BufReadCmd Autocommands for "*.ipynb"..function <SNR>17_read_from_ipynb:                                              
line   24:
jupytext --to=md --output='/private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Personal/IBDP/BusinessManage
ment/IA/j_p.md' '/private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Personal/IBDP/BusinessManagement/IA/j_
p.ipynb': 1

However, if an .md file with the same name generated by Jupytext is there, the .ipynb file can be opened, changes made in the .ipynb file is synced to the .md file.

After pressing Enter to proceed, the file will seem to be empty (as conversion failed).

Jupytext itself works well. For-back conversions can be done correctly.

Reproduction

1. Create an .ipynb file via Jupyter Lab/Notebook
2. Open the file via Vim
3. An error will occur as mentioned above

Specs

• OS: Darwin 22.5.0/iPadOS 16.5
• Arch: Arm64
• Machine: iPad Pro 3
• App: a-Shell, runs a Linux-like shell
• Python: 3.11.0
• Vim: 8.2
• Anaconda: not installed
• Pandoc: not installed

Debug Messages

Messages from :messages after let g:jupytext_print_debug_msgs = 1:

DBG: filename: /private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.ipynb                         
DBG: filename exists: 1
DBG: jupytext_file: /private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.md
DBG: jupytext_file exists: 0
DBG: Generate file /private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.md
DBG: cmd: jupytext --to=md --output='/private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.md' '/pr
ivate/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.ipynb'
DBG: pandoc: command not found^@[jupytext] Reading /private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Loca
l/j_p.ipynb in format ipynb^@Traceback (most recent call last):^@  File "/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/
Library/bin/jupytext", line 8, in <module>^@    sys.exit(jupytext())^@             ^^^^^^^^^^^@  File "/var/mobile/Containers/Data/Application/6B7E2B9
2-E388-481D-8893-F1E3F0AAE8B8/Library/lib/python3.11/site-packages/jupytext/cli.py", line 488, in jupytext^@    exit_code += jupytext_single_file(nb_f
ile, args, log)^@                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^@  File "/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1
E3F0AAE8B8/Library/lib/python3.11/site-packages/jupytext/cli.py", line 864, in jupytext_single_file^@    base_path(nb_dest, dest_fmt)^@  File "/var/mo
bile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Library/lib/python3.11/site-packages/jupytext/paired_paths.py", line 55, in base
_path^@    raise InconsistentPath(^@jupytext.paired_paths.InconsistentPath: Notebook path ''/private/var/mobile/Containers/Data/Application/6B7E2B92-E
388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.md'' was expected to have extension '.md'^@

Hello,
thanks for your plugin, I have been playing with it a bit and it is great.
I have a question regarding the usage though. I mostly code in R and find Rmd (rmarkdown) format superior to alternatives. However I see the advantage of being able to share ipynb to other people.
The question I have is then: is it possible for me to edit a Rmd file and have it "transformed" to ipynb in real time (basically the opposite of what I think this plugin allows to do)

EDIT:
Basically I think what I am asking is is it possible to setup a map from -> to like
{'ipynb' : 'rmarkdown', 'rmarkdown' : 'ipynb', 'md' : 'ipynb'}
Thanks and regards

This plugin does not play nice with various git-related plugins such as fugitive, gitgutter and signify.

For gitgutter, bogus signs are shown, based on the output of

git diff --no-color --no-ext-diff -U0 -- <ipynbfile>

Personally, I already use nbdime as a diff tool for notebooks, but its output is (obviously) not valid for jupytext. Probably, the only solution is to create a diff tool based on jupytext that is used in the context of git-gutter only. This would probably depend on airblade/vim-gitgutter#568 as well.

Alternatively, one would probably want to just disable git-gutter for notebook files.

For signfiy, no signs are shown at all (presumably because it can detect that the generated patch is bogus?). In principle, the solution should be the same as for gitgutter, except that signfiy appears to be better set up to handle using a custom diff-tool for notebook files. However, that diff-tool would still need to exist.

The fugitive plugin appears simply inoperative (e.g. the Gdiff command is "not an editor command"). I'm not sure what it would take to fix this, but it seems like something that should be possible. For now, "inoperative" is a better place to be in than "broken".

I am using the py:percent format converting from .ipynb. The syntax highlighting is working well, but I don't get any linting with ALE as I normally do with .py files.

NVIM v0.5.0-nightly-1445-g1ff5b60cb
Build type: Release
LuaJIT 2.0.5
with Python 3.8.0 (default, Nov 8 2019, 09:53:30)

Trying to load a blank ipynb notebook throws an error since it is not of a valid JSON format. Trying to load a template on 'BufNewFile' gets overwritten by jupytext.

" Create .ipynb with skeleton
autocmd BufNewFile *.ipynb 0r ~/.vim/skeletons/skeleton.ipynb

One alternate way that I tried to explore is to disable the auto conversion of ipynb to allow vim templates to load, saving and reloading with jupytext. Disabling auto conversion can be done by unsetting let g:jupytext_enable = 1 but being someone relatively new to vim, I am not sure on how to manually call the jupytext extension to do its thing.

Any help would be appreciated. Thanks in advance!

I love this plugin, using it quite often, however I find it failing for plugins (LSP servers) that need/use files on disk (not vim buffers).

I think when opening an ipynb file, the name of the vim buffer (foo.ipynb) is misleading, I am not editing the original .ipynb file, but the converted plain file .py.
Is there an option to rename the vim buffer to foo.py? The automatic write back to the ipynb is great, as well as deleting the py file if I opened a .ipynb.

Describe the bug
It looks nice when open the file with neovim. However, when starting edit the buffer, the file format changed from python to json. All syntax highlight lost. Edits are saved back to the notebook.

I set the format to python, but the problem happens with md as well.
Diagnostics

• vim --version: NVIM 0.9.4

• Operating system information (e.g. uname -a): Darwin MBP

• python --VV: unknown option --VV

• Are you using Anaconda?
  NO. miniconda

• Put let g:jupytext_print_debug_msgs = 1 in your ~/.vimrc. What is the output of :messages when reproducing the problem?
  NO output

• Does converting the notebook to/from ipynb with jupytext on the command line work?
  YES

• Does it work when you set g:jupytext_command in ~/.vimrc to be the exact some jupytext that you used manually, with the exact same version of Python?

• YES

Hello,
I am using the following options let g:jupytext_fmt = 'R:spin' as allowed by jupyter --help
As a matter of fact the following works perfectly

jupytext --set-formats R:spin,ipynb,Rmd --sync test.R
[jupytext] Reading test.R
[jupytext] Updating notebook metadata with '{"jupytext": {"formats": "R:spin,ipynb,Rmd"}}'
[jupytext] Updating 'test.ipynb'
[jupytext] Updating 'test.Rmd'
[jupytext] Updating 'test.R'

But when I try to vim test.ipynb, when I save the file the conversion fails because you jupytext.vim try to run jupytext --to=R:spin --output='/home/statquant/CodeProjects/jupyter/workflow_4/test.r' '/home/statquant/CodeProjects/jupyter/workflow_4/test.ipynb' if it was .R all would be fine

With an empty vimrc, on either Ubuntu or Windows, jupytext.vim quickly displays this error message:

Error detected while processing function <SNR>11_read_from_ipynb:
line   40:
E121: Undefined variable: g:jupytext_filetype_map
E116: Invalid arguments for function get(g:jupytext_filetype_map, g:jupytext_fmt,              s:jupytext_filetype_map[g:jupytext_fmt])
E15: Invalid expression: get(g:jupytext_filetype_map, g:jupytext_fmt,              s:jupytext_filetype_map[g:jupytext_fmt])
line   42:
E121: Undefined variable: l:ft
E116: Invalid arguments for function <SNR>11_debugmsg
line   43:
E121: Undefined variable: l:ft
E15: Invalid expression: "set ft=".l:ft

Describe the bug

When trying to make a new copy of the notebook I'm working on with :saveas, the buffer name changes, the message "two.ipynb saved via jupytext" appears, but no new file is created, while the previous one is overwritten. Steps to reproduce:

1. :ed one.ipynb
2. Make edits, :w
3. More edits
4. :saveas two.ipynb

Diagnostics

• vim --version:

centi@centi-Latitude-3540:~$ nvim --version
NVIM v0.10.0-dev
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Run "nvim -V1 -v" for more info

• Operating system information (e.g. uname -a):

centi@centi-Latitude-3540:~$ uname -a
Linux centi-Latitude-3540 6.5.0-18-generic #18~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb  7 11:40:03 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

• python --VV:

centi@centi-Latitude-3540:~$ python -VV
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]

• Are you using Anaconda? No
• Put let g:jupytext_print_debug_msgs = 1 in your ~/.vimrc. What is the output of :messages when reproducing the problem?

// :ed one.ipynb
DBG: filename: /home/centi/one.ipynb
DBG: filename exists: 0
DBG: jupytext_file: /home/centi/one.py
DBG: jupytext_file exists: 0
DBG: autocmd jupytext_ipynb BufUnload <buffer> call s:cleanup("/home/centi/one.py", 1)
DBG: autocmd jupytext_ipynb BufWriteCmd,FileWriteCmd <buffer> call s:write_to_ipynb()
DBG: filetype: python
// :w
DBG: overwriting /home/centi/one.py
DBG: Updating notebook from /home/centi/one.py
DBG: cmd: jupytext --from=py --to=ipynb --update '/home/centi/one.py'
DBG: [jupytext] Reading /home/centi/one.py in format py^@[jupytext] Writing /home/centi/one.ipynb
// :saveas two.ipynb
DBG: overwriting /home/centi/one.py
DBG: Updating notebook from /home/centi/one.py
DBG: cmd: jupytext --from=py --to=ipynb --update '/home/centi/one.py'
DBG: [jupytext] Reading /home/centi/one.py in format py^@[jupytext] Writing /home/centi/one.ipynb (destination file updated)

• Does converting the notebook to/from ipynb with jupytext on the command line work? Not tested, conversion works correctly in vim.

• Does it work when you set g:jupytext_command in ~/.vimrc to be the exact some jupytext that you used manually, with the exact same version of Python? Not tested, conversion works correctly in vim.

Probably not the best place to ask this question, but I'm quite new to Jupyter Notebook, so I'll give it a try. I do all my notebook editing in vim through this plugin, but any changes made are not reflected in Jupyter Notebook until a manual reload is done in JN; I can verify that both the .ipynb and .py files have changed on disk. Is there any way to automatically trigger a reload of the .ipynb file within JN?

I've tried something like this, but as far as I can tell it applies to submodules that the current file relies on, and not the current file itself.

Describe the bug

I used let g:jupytext_fmt = 'py' but there is no python syntax highlighting.

Diagnostics

• vim --version: VIM - Vi IMproved 8.1 (2018 May 18

• Operating system information (e.g. uname -a): macOS 10.14.3

• python --VV: Python 2.7.10

• Are you using Anaconda? No

• Put let g:jupytext_print_debug_msgs = 1 in your ~/.vimrc. What is the output of :messages when reproducing the problem?

Messages maintainer: Mike Williams <[email protected]>                                                                                                                                
DBG: filename: /Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.ipynb                                                                                                                                                                                          
DBG: filename exists: 1                                                                                                                                                                                                                                                         
DBG: jupytext_file: /Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py                                                                                                                                                                                        
DBG: jupytext_file exists: 0                                                                                                                                                                                                                                                    
DBG: Generate file /Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py                                                                                                                                                                                         
DBG: cmd: !jupytext --to=py --output='/Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py' '/Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.ipynb'                                                                                           
DBG: read /Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py                                                                                                                                                                                                  
DBG: autocmd BufUnload <buffer> call s:cleanup("/Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py", 1)                                                                                                                                                       
DBG: filetype: py 

A notebook with spaces in the filename results in an empty buffer