Similar to .pyc files for .py files.

[ ] Investigate how .py -> .pyc compilation works
[ ] Investigate how much of a perf improvement it really is

Thanks for this great project! However, when I did pip install ipynb I got version 0.4, which does not correctly comment out markdown cells when converting into code (fixed in commit b266e35, which lies between 0.4 and 0.5). Currently one must do pip install -e git+https://github.com/ipython/ipynb#egg=ipynb to get the fixed version.

NB, the PyPI page also has an old project URL http://github.com/yuvipanda/ipynb instead of http://github.com/ipython/ipynb .

I'm guessing we'll have user that want to import ..anipynbmodule not sure how to do that explicitly.

Likely on Travis-CI.

Hi,

I found that under certain circumstances __package__ is not defined and hence FSFinder has no package_prefix set. This leads to an error when trying to import a notebook.

Can it be that this is because __package__ is expected to be the same as spec.parent beginning with Python 3.6?
https://docs.python.org/3/reference/import.html#__package__

Anyway, relying on __package__ seems to be a risk.

Best,
Jens Henrik

...python3.6/site-packages/ipynb-0.5.1-py3.6.egg/ipynb/fs/finder.py in find_spec(self=<ipynb.fs.finder.FSFinder object>, fullname='ipynb.fs.full.job_submit', path=['...site-packages/ipynb-0.5.1-py3.6.egg/ipynb/fs/full'], target=None)
     41         Claims modules that are under ipynb.fs
     42         """
---> 43         if fullname.startswith(self.package_prefix):
        fullname.startswith = <built-in method startswith of str object at 0x7f41805a6580>
        self.package_prefix = None
     44             for path in self._get_paths(fullname):
     45                 if os.path.exists(path):

TypeError: startswith first arg must be str or a tuple of str, not NoneType

Let us suppose my main notebook (main.ipynb) is next to a folder otherpath containing another notebook to be imported (to_be_imported.ipynb).

If I do
import ipynb.fs.defs.otherpath.to_be_imported

I get
ModuleNotFoundError: No module named 'ipynb.fs.defs.otherpath'

If I move to_be_imported.ipynb in the same folder of the main file, everything works.

Following https://stackoverflow.com/questions/44116194/import-a-function-from-another-ipynb-file
Python/Sagemath9.4/Jupyter error?
Using jupyter with kernel selections:sagemath9.4, python3
I found that the import works if the file pyt.ipynb is written with the Jupyter kernel Python3 selected, but can't be read if it's written from Sagemath9.4 kernel.
Is there a more correct way or is it a bug?
File pyt.ipynb to be imported

def factorial(n):
if n == 0:
    return 1
else:
    return n * factorial(n-1)

File for importing

from ipynb.fs.full.pyt import *
testing = factorial(5)
testing

Here is the diff output, comparing the python3 save and the sagemath9.4 save
diff pyt.ipynb pyt-python.ipynb

< "display_name": "SageMath 9.4",
< "language": "sage",
< "name": "sagemath"

'> "display_name": "Python 3",
'> "language": "python",
'> "name": "python3"

Assuming there is a function files I want to import: project/helper_fun.ipynb
My current working ipynb file is: project/branch/work.py
when I write from ipynb.fs.defs.helper_fun import fun1 in mywork.p, it saysthere is no module called helper_fun

So my question is how to import the ipynb functions that are in a different folder.

In non-interactive python 2, we can use from __future__ import absolute_import to force explicit relative import. This is not working when, for example, we write some vanilla python files and relative-importing them from a notebook.
I tried pip install git+https://github.com/ipython/ipynb but it gives that it requires python 3. Is there any workaround about this?

It's really wierd that when the source notebook has the magic cell funcction %matplotlib inline statement,
the from ipynb.fs.defs.one_eda import guess_age throws an error citing the same.

1. You can not import functions that have doc strings in them.

Here is the error msg

File "", line 13
"""uses Gauss's method for summing integers
http://wmueller.com/precalculus/advanced/hint4_3_6.html"""
^
IndentationError: unexpected indent

For some imported notebook, say demo1
import ipynb.fs.defs.demo1 is OK, but import ipynb.fs.defs.demo1 as d1 FAIL

and for other imported notebook, say demo2
import ipynb.fs.defs.demo2 as d2 is OK, but import ipynb.fs.defs.demo2 FAIL

I've something like

import os
os.environ["KERAS_BACKEND"] = "tensorflow"

on top of my code (I know I can do that in some other way…) and ipynb.fs.defs fails to import this into another module. I like to fix this myself but honestly don't have good clue what should be done, excepting it from defs or accepting it? As this case, my favor is to accepting it but I am not sure if that would be what all want.

Hi,
newbie here - I have tried to use ipynb in google colab with no success... Here is my code:

!pip install ipynb --upgrade

from ipynb.fs.defs.testingnb import *

and in the same folder I have a testingnb.ipynb file. I get the error:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)

<ipython-input-7-8088143b2d48> in <module>()
----> 1 import ipynb.fs.defs.testingnb

ModuleNotFoundError: No module named 'ipynb.fs.defs.testingnb'

Am i missing some setup step? thanks!
Luigi

I dont' have access to webhooks to set that up, but registered ipynb.readthedocs.org.

Hi, I am posting this issue up here after having received zero help from Stackoverflow. I would greatly appreciate anyone's help on this!

Cryptic TypeError: 'decimal.Decimal' object cannot be interpreted as an integer

Hi and thanks for this great package.
I getting an error while opening a symbolic link notebook to the working directory. Do you have any recommendations how to over come that?

Thanks, HM

Hi

I fail to load a book due to an image in a MD cell.

I have a small notebook mylib.ipynb
[Cell 1 - Python]

# <api>
var = 1

[Cell 2 - MD]

![alt text](images/system-map.png "Title")

... the below also does not work...
<!-- <img src="images/system-map.png" width=1000  align="left"/> -->

And a notebook importing:

import ipynb.fs.defs.mylib

When running this, I get

File "<unknown>", line 11
    ![alt text](images/system-map.png "Title")
    ^
SyntaxError: invalid syntax

This is OSX. Python 3.6.1

Packages:

appdirs==1.4.3
appnope==0.1.0
attrs==16.3.0
Automat==0.5.0
awscli==1.11.87
bleach==2.0.0
botocore==1.5.50
cffi==1.9.1
click==6.7
colorama==0.3.7
constantly==15.1.0
cryptography==1.7.2
cssselect==1.0.1
cycler==0.10.0
decorator==4.0.11
docutils==0.13.1
entrypoints==0.2.2
et-xmlfile==1.0.1
excel-modelling-helper==0.3.0
Flask==0.12
Flask-SocketIO==2.8.4
google-api-python-client==1.6.2
html5lib==0.999999999
httplib2==0.10.3
idna==2.2
incremental==16.10.1
ipykernel==4.6.0
ipynb==0.4
ipython==5.3.0
ipython-genutils==0.2.0
ipywidgets==6.0.0
itsdangerous==0.24
jdcal==1.3
Jinja2==2.9.6
jmespath==0.9.2
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.0.1
jupyter-cms==0.7.0
jupyter-console==5.1.0
jupyter-core==4.3.0
lxml==3.7.2
MarkupSafe==1.0
matplotlib==2.0.0
mistune==0.7.4
mpmath==0.19
nbconvert==5.1.1
nbformat==4.3.0
networkx==1.11
nose==1.3.7
notebook==5.0.0
numpy==1.12.0
oauth2client==4.1.0
olefile==0.44
onedrivesdk==1.1.8
openpyxl==2.4.2
packaging==16.8
pandas==0.19.2
pandoc==1.0.0b2
pandocfilters==1.4.1
parsel==1.1.0
patsy==0.4.1
pbr==1.10.0
pexpect==4.2.1
pickleshare==0.7.4
Pillow==4.1.0
ply==3.10
prompt-toolkit==1.0.14
ptyprocess==0.5.1
pyasn1==0.2.3
pyasn1-modules==0.0.8
pycparser==2.17
PyDispatcher==2.0.5
pydot-ng==1.0.0
pydotplus==2.0.2
Pygments==2.2.0
pygraphviz==1.3.1
pyOpenSSL==16.2.0
pypandoc==1.3.3
pyparsing==2.2.0
PyQt5==5.7.1
pyspark==2.1.0+hadoop2.7
python-dateutil==2.6.0
python-engineio==1.2.3
python-socketio==1.7.1
pytz==2016.10
PyYAML==3.12
pyzmq==16.0.2
qtconsole==4.3.0
queuelib==1.4.2
requests==2.13.0
rsa==3.4.2
s3transfer==0.1.10
scipy==0.18.1
Scrapy==1.3.2
service-identity==16.0.0
simplegeneric==0.8.1
simplejson==3.10.0
sip==4.19
six==1.10.0
statsmodels==0.8.0
stevedore==1.20.0
sympy==1.0
tabulate==0.7.7
terminado==0.6
testpath==0.3
tornado==4.4.3
traitlets==4.3.2
Twisted==17.1.0
tzlocal==1.4
uritemplate==3.0.0
virtualenv==15.1.0
virtualenv-clone==0.2.6
virtualenvwrapper==4.7.2
w3lib==1.17.0
wcwidth==0.1.7
webencodings==0.5
Werkzeug==0.11.15
Whoosh==2.7.4
widgetsnbextension==2.0.0
xarray==0.8.2
xlrd==1.0.0
xlwt==1.2.0
zope.interface==4.3.3

Many thanks

I want to be able to import things straight out of github with the following format:

from ipynb.github.someuser.somerepo.branch_or_tag_or_sha.file_path import some_function

and have that work. Relative imports should also be supported for this so you can put a package of ipynb files on github.

Also investigate how to do this without killing github's servers and have them ban us.

Make sure they're py3 and nbformat 4

Ideally these will not require the ipynb package at all at runtime. We could do some amount of pre-processing to make sure we ship .py files rather than .ipynb files, and dependency on IPython itself will depend on #6.

This totally requires us to have a solid documented understanding of how relative imports work (see #4)

week1.ipynb.txt

Note sure how to debug the following error. Attached is the notebook I tried to import. They are in the same directory

Any ideas?

Thanks

Andy

#from ipynb.fs.defs.week1 import patternToNumber
import ipynb.fs.defs.week1 as wk1
​
#import ipynb.fs.defs.week1 as week1

import ipynb.fs from .defs.week2 \

import foo from .defs.week1 \

import patternToNumber

​

import ipynb.fs

​

from ../week2/week2 import patternToNumber from .defs.week1

​

# base 4 to base 10

# TODO copied from week1 notebook

def patternToNumber(pattern) :

exponent = 0

ret = 0

for nucleotide in reversed(pattern):

digit = 0

if nucleotide == "C" :

digit = 1

elif nucleotide == "G" :

digit = 2

elif nucleotide == "T" :

digit = 3

ret = ret + digit * 4**exponent

#print("ret:{0}, digit:{1}, exponent:{2} nucleotide:{3}"

# .format(ret, digit, exponent, nucleotide))

exponent = exponent + 1

return ret

AttributeError Traceback (most recent call last)
in ()
----> 1 from ipynb.fs.defs.week1 import patternToNumber
2 #import ipynb.fs.defs.week1 as wk1
3
4 #import ipynb.fs.defs.week1 as week1
5 # import ipynb.fs from .defs.week2 \

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py in find_and_load(name, import)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py in find_and_load_unlocked(name, import)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py in _load_unlocked(spec)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap_external.py in exec_module(self, module)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in get_code(self, fullname)
60 ))
61 return self.source_to_code(
---> 62 self._get_filtered_ast(code_from_ipynb(nb)),
63 self.path
64 )

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in _get_filtered_ast(self, source)
47 """
48 tree = ast.parse(source)
---> 49 tree.body = [n for n in tree.body if self._filter_ast_node(n)]
50 return tree
51

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in (.0)
47 """
48 tree = ast.parse(source)
---> 49 tree.body = [n for n in tree.body if self._filter_ast_node(n)]
50 return tree
51

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in _filter_ast_node(self, node)
38
39 if isinstance(node, ast.Assign):
---> 40 return all([t.id.isupper() for t in node.targets])
41
42 return False

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in (.0)
38
39 if isinstance(node, ast.Assign):
---> 40 return all([t.id.isupper() for t in node.targets])
41
42 return False

AttributeError: 'Tuple' object has no attribute 'id'

When cells in a notebook contain magic commands, ipynb cannot import the notebook.

For example, a cell in a notebook contains %matplotlib inline. To import this notebook, ipynb throws an exception, SyntaxError: invalid syntax.

How about

with ipynb.importer(full=True): # or defs=True
  import my_notebook

For example, it would allow to specify options like

with ipynb.importer(ignore_errors=True):
  import my_notebook

Also, we may add the ability to write

import ipynb.finder
import nb1
import nb2

I can make a PR if it's ok

In one notebook (en2ru.ipynb) I create function:
def ruchar(): return 'Б'
In another notebook I import first notebook and call ruchar()

from ipynb.fs.full.en2ru import *
ruchar()

and I get 'Р‘'

importnb is a newer and better version of this package, that does the same things - but is more maintained.

I spoke to @tonyfast and there is interest in making this happen. It will help adoption, and keep this project moving forward.

Things to do:

• Import contents of that git repo to here, preserving all histories
• Add maintainers of importnb to this repo
• Add maintainers of importnb to pypi
• Make an announcement

When using import ipynb.fs.defs.myipynb, I get an error:

 File "<unknown>", line 81
    alpha_dict[key] = alpha_dict[key].ewm(span=20,adjust=True,min_periods=0,ignore_na=False).mean()
    ^
IndentationError: unexpected indent

The cell is:

for key in alpha_dict:
    alpha_dict[key] = alpha_dict[key].ewm(span=20,adjust=True,min_periods=0,ignore_na=False).mean()

I don' t know why would it try to import a for loop like this.

I was testing the following import:
import ipynb.fs.defs.utils.testfoo

and got the following exception:

... ipynb/fs/defs/__init__.py in <listcomp>(.0)
     39         if isinstance(node, ast.Assign):
---> 40             return all([t.id.isupper() for t in node.targets])

AttributeError: 'Subscript' object has no attribute 'id'

So I modified the _filter_ast_node to include a couple of try/excepts to see where it was failing:

        if isinstance(node, ast.Assign):
            try:
                return all([t.id.isupper() for t in node.targets])
            except AttributeError:
                print(ast.dump(node))
                for t in node.targets:
                    try:
                        t.id
                    except AttributeError:
                        print(ast.dump(t))
                raise

ast.dump(node):

Assign(targets=[Subscript(value=Name(id='df', ctx=Load()), slice=Index(value=Str(s='test')), ctx=Store())], value=Str(s='foo'))

ast.dump(t):

Subscript(value=Name(id='df', ctx=Load()), slice=Index(value=Str(s='test')), ctx=Store())

Looks like it's occurring on the following line of my notebook:

df['test'] = 'foo'

One possible solution:

    def _filter_ast_node(self, node):
        for an in ALLOWED_NODES:
            if isinstance(node, an):
                return True

        if isinstance(node, ast.Assign):
            return all(self._id_is_upper(t) for t in node.targets)

        return False

    @staticmethod
    def _id_is_upper(t):
        try:
            return t.id.isupper()
        except AttributeError:
            return False

I'm in JupyterLab, with two notebooks open, one of which is importing a function from the other (tools.ipynb):

import ipynb.fs.defs.tools

However, it doesn't seem to pick up changes from the other one, even when I re-run the cell containing the import. However, if I:

import importlib
importlib.reload(ipynb.fs.defs.tools)

Then it does indeed pick it up changes each time. Unfortunately, this doesn't work for the from X import Y style, since the reference to the module containing the imported item isn't in scope to pass to reload.

Is it possible that ipynb could watch timestamps on the imported files and handle this under the hood when the imported file changes? Thanks for a great project!

Right now, we only include assignments where the LHS is all caps. However, LHS all caps might have dependencies on RHS that's not all caps. These will not be evaluated, causing strange amounts of confusion.

Instead, we should attempt to track the dependencies on the RHS of such assignments, and eval those too (recursively). We should do this in such a way that the rules are very clear to explain to people.

This should still only be done for top level imports though.

I have strived to import an external ipynb, but I got the following error:

ImportError: Could not import /content/.ipynb for ipynb.fs.full.: incorrect version or language

I was wondering if I could know that you have ever experienced the same issue, please!

NB: I am running the codes on the Google Colab.

Thanks in advance

Currently, we don't support IPython Magics at all from noteboks. We initially supported them, but removed them because importing IPython was causing a massive startup time lag (>1s) that seemed pretty bad.

Ideally, I'd like to be able to do the following:

1. If IPython is already loaded (you are already in a notebook), support magics by default
2. If IPython is not loaded, and there are no magics in the imported notebooks, do not load it
3. Load IPython only if there are magics in the notebook

I'd also like to possibly consider implementing the ! action (which runs things in a shell) natively, so we don't have to import all of IPython to get just that behavior. Also investigate why startup takes so long!