jupyter-naas / naas Goto Github PK

Hi,

Installing NAAS with existing Jupyter notebook/lab installation will be a very useful use case as Scheduling & Jupyter as API are a very string value proposition. I raised this on twitter and was asked to raise an issue here by Maxime Jublou.

Please let us know if there are any easy way to do this.

Regards
Ganesh

Also remove Storage

As a user, I want my naas runner to not fail, or at least to have it being restarted automatically.

Today when the runner crashes, users are not directly aware of it. They receive an email when a job is failing but the only solution for them is to restart their instance, this should be handled automatically.

Make sure you see all features wherever you are in your repo

When we call webhook we should find a way to make kernel boot faster

https://naas.canny.io/requests/p/boost-response-time-of-webhook

When use the naas.run to test a notebook in prod but it returned an error message instead of outputs.

https://public.naas.ai/ZmxvcmVudC0yRXJhdmVuZWwtNDBjYXNoc3RvcnktMkVjb20=/asset/2fe80da9e44cf1bf34e641ad534f4238616af4c76a533500c6b2ef2ebdbc

https://naas.canny.io/incident-report/p/naas-run-error-cannot-get-your-file-none

I tried to setup a domain but I have the following message :
module 'naas' has no attribute 'domain'
Will send the setup video in private.

As a user, I used to receive a daily email of my credits consumption. I don't receive it anymore.

Today we have an issue when user are running:
pip install --user naas
pip install --user naas-drivers

This is pulling and installing naas and/or naas-drivers in /home/ftp/.local/lib/python3.8/site-packages which takes over /opt/conda/lib/python3.8/site-packages (where we install new naas and naas-drivers versions).

We should run rm -rf /home/ftp/.local/lib/python3.8/site-packages/naas* to make sure the user is always using latest versions.

Following the documentation, I tried to use the formulas but there is a weird conflict (see pitcture) https://naas.gitbook.io/drivers/more-info/changelog#to-get-the-latest-version

Sometimes, in a non-linear manner, we can see crashes inside our Kubernetes/AWS cloud infra.

It's an issue related to Kubernetes spawner that does not create deployment but just a pod.

What we should expect is an immediate reboot of the machine anytime it crashes. #chaosstrategy

Community support
Contributing guide
Features
Quick start
Mission

Inspired by
https://github.com/airbytehq/airbyte/

I have tried to install torch a few times but it's not working.

https://naas.canny.io/admin/board/incident-report/p/pytorch-install-not-working

ML Librairies =
Numpy
Scipy
Scikit-learn
Theano
TensorFlow
Keras
PyTorch
Pandas
Matplotlib
https://www.geeksforgeeks.org/best-python-libraries-for-machine-learning/

https://naas.canny.io/admin/board/requests/p/add-ml-librairies-to-installed-naas-librairies

There is a problem installing NAAS locally. It says, installation build dependencies error and attribute error. < attached screenshots.
https://naas.canny.io/incident-report/p/installation-error-on-local-system

Package Version

absl-py 0.10.0
alabaster 0.7.12
albumentations 0.1.12
altair 4.1.0
appdirs 1.4.4
argon2-cffi 20.1.0
asgiref 3.3.1
astor 0.8.1
astropy 4.1
astunparse 1.6.3
async-generator 1.10
atari-py 0.2.6
atomicwrites 1.4.0
attrs 20.3.0
audioread 2.1.9
autograd 1.3
Babel 2.9.0
backcall 0.2.0
beautifulsoup4 4.6.3
bleach 3.3.0
blis 0.4.1
bokeh 2.1.1
Bottleneck 1.3.2
branca 0.4.2
bs4 0.0.1
CacheControl 0.12.6
cachetools 4.2.1
catalogue 1.0.0
certifi 2020.12.5
cffi 1.14.4
chainer 7.4.0
chardet 3.0.4
click 7.1.2
cloudpickle 1.3.0
cmake 3.12.0
cmdstanpy 0.9.5
colorlover 0.3.0
community 1.0.0b1
contextlib2 0.5.5
convertdate 2.3.0
coverage 3.7.1
coveralls 0.5
crcmod 1.7
cufflinks 0.17.3
cvxopt 1.2.5
cvxpy 1.0.31
cycler 0.10.0
cymem 2.0.5
Cython 0.29.21
daft 0.0.4
dask 2.12.0
dataclasses 0.8
datascience 0.10.6
debugpy 1.0.0
decorator 4.4.2
defusedxml 0.6.0
descartes 1.1.0
dill 0.3.3
distributed 1.25.3
Django 3.1.6
dlib 19.18.0
dm-tree 0.1.5
docopt 0.6.2
docutils 0.16
dopamine-rl 1.0.5
earthengine-api 0.1.238
easydict 1.9
ecos 2.0.7.post1
editdistance 0.5.3
en-core-web-sm 2.2.5
entrypoints 0.3
ephem 3.7.7.1
et-xmlfile 1.0.1
fa2 0.3.5
fancyimpute 0.4.3
fastai 1.0.61
fastdtw 0.3.4
fastprogress 1.0.0
fastrlock 0.5
fbprophet 0.7.1
feather-format 0.4.1
filelock 3.0.12
firebase-admin 4.4.0
fix-yahoo-finance 0.0.22
Flask 1.1.2
flatbuffers 1.12
folium 0.8.3
future 0.16.0
gast 0.3.3
GDAL 2.2.2
gdown 3.6.4
gensim 3.6.0
geographiclib 1.50
geopy 1.17.0
gin-config 0.4.0
glob2 0.7
google 2.0.3
google-api-core 1.16.0
google-api-python-client 1.7.12
google-auth 1.25.0
google-auth-httplib2 0.0.4
google-auth-oauthlib 0.4.2
google-cloud-bigquery 1.21.0
google-cloud-bigquery-storage 1.1.0
google-cloud-core 1.0.3
google-cloud-datastore 1.8.0
google-cloud-firestore 1.7.0
google-cloud-language 1.2.0
google-cloud-storage 1.18.1
google-cloud-translate 1.5.0
google-colab 1.0.0
google-pasta 0.2.0
google-resumable-media 0.4.1
googleapis-common-protos 1.52.0
googledrivedownloader 0.4
graphviz 0.10.1
grpcio 1.32.0
gspread 3.0.1
gspread-dataframe 3.0.8
gym 0.17.3
h5py 2.10.0
HeapDict 1.0.1
hijri-converter 2.1.1
holidays 0.10.5.2
holoviews 1.13.5
html5lib 1.0.1
httpimport 0.5.18
httplib2 0.17.4
httplib2shim 0.0.3
humanize 0.5.1
hyperopt 0.1.2
ideep4py 2.0.0.post3
idna 2.10
image 1.5.33
imageio 2.4.1
imagesize 1.2.0
imbalanced-learn 0.4.3
imblearn 0.0
imgaug 0.2.9
importlib-metadata 3.4.0
importlib-resources 5.1.0
imutils 0.5.4
inflect 2.1.0
iniconfig 1.1.1
intel-openmp 2021.1.2
intervaltree 2.1.0
ipykernel 4.10.1
ipython 5.5.0
ipython-genutils 0.2.0
ipython-sql 0.3.9
ipywidgets 7.6.3
itsdangerous 1.1.0
jax 0.2.9
jaxlib 0.1.60+cuda101
jdcal 1.4.1
jedi 0.18.0
jieba 0.42.1
Jinja2 2.11.3
joblib 1.0.0
jpeg4py 0.1.4
jsonschema 2.6.0
jupyter 1.0.0
jupyter-client 5.3.5
jupyter-console 5.2.0
jupyter-core 4.7.1
jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.0.0
kaggle 1.5.10
kapre 0.1.3.1
Keras 2.4.3
Keras-Preprocessing 1.1.2
keras-vis 0.4.1
kiwisolver 1.3.1
knnimpute 0.1.0
korean-lunar-calendar 0.2.1
librosa 0.8.0
lightgbm 2.2.3
llvmlite 0.34.0
lmdb 0.99
lucid 0.3.8
LunarCalendar 0.0.9
lxml 4.2.6
Markdown 3.3.3
MarkupSafe 1.1.1
matplotlib 3.2.2
matplotlib-venn 0.11.6
missingno 0.4.2
mistune 0.8.4
mizani 0.6.0
mkl 2019.0
mlxtend 0.14.0
more-itertools 8.6.0
moviepy 0.2.3.5
mpmath 1.1.0
msgpack 1.0.2
multiprocess 0.70.11.1
multitasking 0.0.9
murmurhash 1.0.5
music21 5.5.0
natsort 5.5.0
nbclient 0.5.1
nbconvert 5.6.1
nbformat 5.1.2
nest-asyncio 1.5.1
networkx 2.5
nibabel 3.0.2
nltk 3.2.5
notebook 5.3.1
np-utils 0.5.12.1
numba 0.51.2
numexpr 2.7.2
numpy 1.19.5
nvidia-ml-py3 7.352.0
oauth2client 4.1.3
oauthlib 3.1.0
okgrade 0.4.3
opencv-contrib-python 4.1.2.30
opencv-python 4.1.2.30
openpyxl 2.5.9
opt-einsum 3.3.0
osqp 0.6.2.post0
packaging 20.9
palettable 3.3.0
pandas 1.1.5
pandas-datareader 0.9.0
pandas-gbq 0.13.3
pandas-profiling 1.4.1
pandocfilters 1.4.3
panel 0.9.7
param 1.10.1
parso 0.8.1
pathlib 1.0.1
patsy 0.5.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 7.0.0
pip 19.3.1
pip-tools 4.5.1
plac 1.1.3
plotly 4.4.1
plotnine 0.6.0
pluggy 0.7.1
pooch 1.3.0
portpicker 1.3.1
prefetch-generator 1.0.1
preshed 3.0.5
prettytable 2.0.0
progressbar2 3.38.0
prometheus-client 0.9.0
promise 2.3
prompt-toolkit 1.0.18
protobuf 3.12.4
psutil 5.4.8
psycopg2 2.7.6.1
ptyprocess 0.7.0
py 1.10.0
pyarrow 0.14.1
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycocotools 2.0.2
pycparser 2.20
pyct 0.4.8
pydata-google-auth 1.1.0
pydot 1.3.0
pydot-ng 2.0.0
pydotplus 2.0.2
PyDrive 1.3.1
pyemd 0.5.1
pyglet 1.5.0
Pygments 2.6.1
pygobject 3.26.1
pymc3 3.7
PyMeeus 0.3.7
pymongo 3.11.3
pymystem3 0.2.0
pynndescent 0.5.1
PyOpenGL 3.1.5
pyparsing 2.4.7
pyrsistent 0.17.3
pysndfile 1.3.8
PySocks 1.7.1
pystan 2.19.1.1
pytest 3.6.4
python-apt 1.6.5+ubuntu0.5
python-chess 0.23.11
python-dateutil 2.8.1
python-louvain 0.15
python-slugify 4.0.1
python-utils 2.5.6
pytz 2018.9
pyviz-comms 2.0.1
PyWavelets 1.1.1
PyYAML 3.13
pyzmq 22.0.2
qdldl 0.1.5.post0
qtconsole 5.0.2
QtPy 1.9.0
regex 2019.12.20
requests 2.23.0
requests-oauthlib 1.3.0
resampy 0.2.2
retrying 1.3.3
rpy2 3.2.7
rsa 4.7
scikit-image 0.16.2
scikit-learn 0.22.2.post1
scipy 1.4.1
screen-resolution-extra 0.0.0
scs 2.1.2
seaborn 0.11.1
Send2Trash 1.5.0
setuptools 53.0.0
setuptools-git 1.2
Shapely 1.7.1
simplegeneric 0.8.1
six 1.15.0
sklearn 0.0
sklearn-pandas 1.8.0
smart-open 4.1.2
snowballstemmer 2.1.0
sortedcontainers 2.3.0
SoundFile 0.10.3.post1
spacy 2.2.4
Sphinx 1.8.5
sphinxcontrib-serializinghtml 1.1.4
sphinxcontrib-websupport 1.2.4
SQLAlchemy 1.3.23
sqlparse 0.4.1
srsly 1.0.5
statsmodels 0.10.2
sympy 1.1.1
tables 3.4.4
tabulate 0.8.7
tblib 1.7.0
tensorboard 2.4.1
tensorboard-plugin-wit 1.8.0
tensorboardcolab 0.0.22
tensorflow 2.4.1
tensorflow-addons 0.8.3
tensorflow-datasets 4.0.1
tensorflow-estimator 2.4.0
tensorflow-gcs-config 2.4.0
tensorflow-hub 0.11.0
tensorflow-metadata 0.27.0
tensorflow-privacy 0.2.2
tensorflow-probability 0.12.1
termcolor 1.1.0
terminado 0.9.2
testpath 0.4.4
text-unidecode 1.3
textblob 0.15.3
textgenrnn 1.4.1
Theano 1.0.5
thinc 7.4.0
tifffile 2020.9.3
toml 0.10.2
toolz 0.11.1
torch 1.7.0+cu101
torchsummary 1.5.1
torchtext 0.3.1
torchvision 0.8.1+cu101
tornado 5.1.1
tqdm 4.41.1
traitlets 4.3.3
tweepy 3.6.0
typeguard 2.7.1
typing-extensions 3.7.4.3
tzlocal 1.5.1
umap-learn 0.5.0
uritemplate 3.0.1
urllib3 1.24.3
vega-datasets 0.9.0
wasabi 0.8.2
wcwidth 0.2.5
webencodings 0.5.1
Werkzeug 1.0.1
wheel 0.36.2
widgetsnbextension 3.5.1
wordcloud 1.5.0
wrapt 1.12.1
xarray 0.15.1
xgboost 0.90
xkit 0.0.0
xlrd 1.1.0
xlwt 1.3.0
yellowbrick 0.9.1
zict 2.0.0
zipp 3.4.0

https://naas.canny.io/admin/board/requests/p/sync-naas-with-google-colab-pre-installed-packages

Allow user to choose the version he wants to open when
naas.scheduler.get_output()

Evolution of : https://docs.naas.ai/features/scheduler#get-output

Users should know that running every min a process is to be used with Caution.

Discussion with MJ 25/05/2021

Should increase acquisition rate if people only have to click "connect with google"

To be tested and documented

Since install of Jupyter lab 3 the search module / extension is KO.

Martin Donadieu
marked this post as
IN PROGRESS
It work in local but not behind jupyter hub, have report it to the plugin creator and wait for an answer
0
·
April 6, 2021

https://naas.canny.io/admin/feedback/incident-report/p/search-not-working?boards=incident-report

I have noticed that naas.scheduler.list() is not working.

Created. 13 May 2021
https://naas.canny.io/incident-report/p/naaslist-ko

As a user, I want to see be able to access from the launcher :

• creation of a new python notebook
• access naas manager
• open doc naas
• open doc naas_drivers
• access all the templates available in git = https://github.com/jupyter-naas/awesome-notebooks

Now. => PS: to remove the "Asset sharing" feature, just replace .add by .delete
Should be => PS: to remove the "Notebook as API" feature, just replace .add by .delete

Use case of ugomesa, has received 5 emails in a row 2 days ago.

When we share assets 2 times, it breaks the link.

import requests
from os import environ

jup_token = environ.get('JUPYTERHUB_API_TOKEN')
headers = {"Authorization": f"token {jup_token}"}
url = f'https://app.naas.ai/user/[email protected]/proxy/5000/job'
res = requests.get(url, headers=headers)

data = res.json()
for job in data:
status = job.get("status")
path = job.get("path")
if path == "admin_naas/usage/script/update_credits_spent.ipynb":
print(job)

As a user I cannot iframe my pages properly.

naas.scheduler

naas.assets

naas.api

When I have an email “scheduler down” I click on the button and it goes to a new page with the manager and search for ID of event. But it goes seems to go into an infinite loop.

https://naas.canny.io/incident-report/p/manager-loop-when-click-on-scheduler-down-button

May 13, 2021

As per as discussion on Slack with Martin

When I use df = naas.callback.list() I have the following error message : 'Connection aborted.', RemoteDisconnected('Remote end closed connection without response')

=> Solution : enable pagination in callback microservice

JupyterLab recently implemented a collaborative model.

This PR implements collaborative editing in JupyterLab using the Yjs shared editing framework. Yjs is an open-source framework to build collaborative applications using data structures (CRDTs) that sync automatically.

jupyterlab/jupyterlab#10118

Correct me if I'm wrong, but you're running the flake8 and black in a GH action, then not throwing an error or committing the changes if the code is badly formatted?

An example of this is shown in #100; although 2 lines have changed, running black shows formatting errors.

When I try to use prophet lib I have an error : Building wheel for prophet (setup.py) ... error
ERROR: Command errored out with exit status 1:

ModuleNotFoundError: No module named 'convertdate'

ERROR: Failed building wheel for prophet
=> as per as the error I tried to install convertdate but it does no do anything.
Here the script I'm trying to replicate in naas.
https://towardsdatascience.com/time-series-analysis-and-climate-change-7bb4371021e

Multiple popup stack up and come back as of today when opening notebooks, impacting the UX

Might be linked to Jupyter single user

Will be a component of the Naas API.

Objective :

• Improve security
• Better integration with JupyterLab

Key result :

• Allow removing jobs directly from the interface/page
• Show naas credits everywhere

To do :

• POC JupyterLab module (design and implementation)
• Validation of POC
• Implementation of final design

As a user, I tried to get my webhook output with the formula "naas.webhook.get_output(path=SCRIPT_PATH)" but it returned the error "❌ Cannot get your file ../script/email_Naas_unsubscribe.ipynb"

When I click on the Naas manager storage button, a modal opens but no value is inside.

Today we have no message in prompt when .delete is done

Suggestions for messaging:

=> naas.assets
✔️ The previous asset URL has been deleted successfully.

=> naas.scheduler
✔️ The previous scheduler recurrence has been deleted successfully.

=> naas.api
✔️ The previous APi endpoint has been deleted successfully.