Git Product home page Git Product logo

jupyter-naas / naas Goto Github PK

View Code? Open in Web Editor NEW
282.0 4.0 25.0 2.67 MB

Low-code Python library to safely use notebooks in production: schedule workflows, generate assets, trigger webhooks, send notifications, build pipelines, manage secrets (Cloud-only)

Home Page: https://app.naas.ai/

License: GNU Affero General Public License v3.0

Python 55.07% HTML 8.11% Shell 1.30% Jupyter Notebook 32.81% Dockerfile 0.52% Makefile 0.17% Batchfile 0.02% JavaScript 0.21% TypeScript 1.78% CSS 0.01%
engine notebooks open-source data-science data integration pipeline etl data-transformation jupyter jupyterlab binder ai

naas's Introduction

Naas Logo

✨ Welcome to Naas!

Notebooks as a service (Naas) is an open source platform that allows anyone touching data (business analysts, scientists and engineers) to create powerful data engines combining automation, analytics and AI from the comfort of their Jupyter notebooks.

Naas is an attempt to propose an alternative to Google Colab, powered by the community.

In addition to Google Colab, Naas platform upgrade notebooks with with 3 low-code layers: features, drivers, templates.

  • Templates enable the user to create automated data jobs and reports in minutes.
  • Drivers act as connectors to push and/or pull data from databases, APIs, and Machine Learning algorithms and more.
  • Features transform Jupyter in a production ready environment with scheduling, asset sharing, and notifications.

🚀 Quick Start

Try all of Naas's features for free using -- Naas cloud -- a stable environment, without having to install anything.

⚙️ Installation

Check out our step by step guide on how to set up Naas locally.

❤️ Contributing

We value all kinds of contributions - not just code. We are paticularly motivated to support new contributors and people who are looking to learn and develop their skills.

Please read our contibuting guidelines on how to get started.

🤔 Community Support

The naas documentation is a great place to start and to get answers for general questions.

📃 License

The project is licensed under AGPL-3.0

naas's People

Contributors

actions-user avatar bobcashstory avatar cbevilaqua avatar dependabot[bot] avatar dr0p42 avatar florentlvr avatar gilsonbridge avatar github-actions[bot] avatar imasdekar avatar l-loic avatar minurapunchihewa avatar quentinklebaur avatar riderx avatar shloka-gupta avatar thomasparenteau avatar tomhamiltonstubber avatar xja avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

naas's Issues

Naas - As a user I want that naas and naas-driver force update even if I tried to run `pip install naas{-drivers} --user`

Today we have an issue when user are running:
pip install --user naas
pip install --user naas-drivers

This is pulling and installing naas and/or naas-drivers in /home/ftp/.local/lib/python3.8/site-packages which takes over /opt/conda/lib/python3.8/site-packages (where we install new naas and naas-drivers versions).

We should run rm -rf /home/ftp/.local/lib/python3.8/site-packages/naas* to make sure the user is always using latest versions.

Convert Naas manager into a JupyterLab module

Objective :

  • Improve security
  • Better integration with JupyterLab

Key result :

  • Allow removing jobs directly from the interface/page
  • Show naas credits everywhere

To do :

  • POC JupyterLab module (design and implementation)
  • Validation of POC
  • Implementation of final design

Installing naas with existing Jupyter installation

Hi,

Installing NAAS with existing Jupyter notebook/lab installation will be a very useful use case as Scheduling & Jupyter as API are a very string value proposition. I raised this on twitter and was asked to raise an issue here by Maxime Jublou.

Please let us know if there are any easy way to do this.

Regards
Ganesh

PS error message on naas.asset is KO

image

Now. => PS: to remove the "Asset sharing" feature, just replace .add by .delete
Should be => PS: to remove the "Notebook as API" feature, just replace .add by .delete

Add message when .delete is done on naas.assets; naas.scheduler, naas.api

Today we have no message in prompt when .delete is done

Suggestions for messaging:

=> naas.assets
✔️ The previous asset URL has been deleted successfully.

=> naas.scheduler
✔️ The previous scheduler recurrence has been deleted successfully.

=> naas.api
✔️ The previous APi endpoint has been deleted successfully.

Naas - Webhook can not get output in prod

As a user, I tried to get my webhook output with the formula "naas.webhook.get_output(path=SCRIPT_PATH)" but it returned the error "❌ Cannot get your file ../script/email_Naas_unsubscribe.ipynb"

image

image

Remove automated callbacks for credits

When I use df = naas.callback.list() I have the following error message : 'Connection aborted.', RemoteDisconnected('Remote end closed connection without response')

=> Solution : enable pagination in callback microservice

Naas - Improve naas runner resiliency

As a user, I want my naas runner to not fail, or at least to have it being restarted automatically.

Today when the runner crashes, users are not directly aware of it. They receive an email when a job is failing but the only solution for them is to restart their instance, this should be handled automatically.

Enable shared editing in notebooks

JupyterLab recently implemented a collaborative model.

This PR implements collaborative editing in JupyterLab using the Yjs shared editing framework. Yjs is an open-source framework to build collaborative applications using data structures (CRDTs) that sync automatically.

jupyterlab/jupyterlab#10118

Naas with Colab/Data science pre-installed packages

Package Version


absl-py 0.10.0
alabaster 0.7.12
albumentations 0.1.12
altair 4.1.0
appdirs 1.4.4
argon2-cffi 20.1.0
asgiref 3.3.1
astor 0.8.1
astropy 4.1
astunparse 1.6.3
async-generator 1.10
atari-py 0.2.6
atomicwrites 1.4.0
attrs 20.3.0
audioread 2.1.9
autograd 1.3
Babel 2.9.0
backcall 0.2.0
beautifulsoup4 4.6.3
bleach 3.3.0
blis 0.4.1
bokeh 2.1.1
Bottleneck 1.3.2
branca 0.4.2
bs4 0.0.1
CacheControl 0.12.6
cachetools 4.2.1
catalogue 1.0.0
certifi 2020.12.5
cffi 1.14.4
chainer 7.4.0
chardet 3.0.4
click 7.1.2
cloudpickle 1.3.0
cmake 3.12.0
cmdstanpy 0.9.5
colorlover 0.3.0
community 1.0.0b1
contextlib2 0.5.5
convertdate 2.3.0
coverage 3.7.1
coveralls 0.5
crcmod 1.7
cufflinks 0.17.3
cvxopt 1.2.5
cvxpy 1.0.31
cycler 0.10.0
cymem 2.0.5
Cython 0.29.21
daft 0.0.4
dask 2.12.0
dataclasses 0.8
datascience 0.10.6
debugpy 1.0.0
decorator 4.4.2
defusedxml 0.6.0
descartes 1.1.0
dill 0.3.3
distributed 1.25.3
Django 3.1.6
dlib 19.18.0
dm-tree 0.1.5
docopt 0.6.2
docutils 0.16
dopamine-rl 1.0.5
earthengine-api 0.1.238
easydict 1.9
ecos 2.0.7.post1
editdistance 0.5.3
en-core-web-sm 2.2.5
entrypoints 0.3
ephem 3.7.7.1
et-xmlfile 1.0.1
fa2 0.3.5
fancyimpute 0.4.3
fastai 1.0.61
fastdtw 0.3.4
fastprogress 1.0.0
fastrlock 0.5
fbprophet 0.7.1
feather-format 0.4.1
filelock 3.0.12
firebase-admin 4.4.0
fix-yahoo-finance 0.0.22
Flask 1.1.2
flatbuffers 1.12
folium 0.8.3
future 0.16.0
gast 0.3.3
GDAL 2.2.2
gdown 3.6.4
gensim 3.6.0
geographiclib 1.50
geopy 1.17.0
gin-config 0.4.0
glob2 0.7
google 2.0.3
google-api-core 1.16.0
google-api-python-client 1.7.12
google-auth 1.25.0
google-auth-httplib2 0.0.4
google-auth-oauthlib 0.4.2
google-cloud-bigquery 1.21.0
google-cloud-bigquery-storage 1.1.0
google-cloud-core 1.0.3
google-cloud-datastore 1.8.0
google-cloud-firestore 1.7.0
google-cloud-language 1.2.0
google-cloud-storage 1.18.1
google-cloud-translate 1.5.0
google-colab 1.0.0
google-pasta 0.2.0
google-resumable-media 0.4.1
googleapis-common-protos 1.52.0
googledrivedownloader 0.4
graphviz 0.10.1
grpcio 1.32.0
gspread 3.0.1
gspread-dataframe 3.0.8
gym 0.17.3
h5py 2.10.0
HeapDict 1.0.1
hijri-converter 2.1.1
holidays 0.10.5.2
holoviews 1.13.5
html5lib 1.0.1
httpimport 0.5.18
httplib2 0.17.4
httplib2shim 0.0.3
humanize 0.5.1
hyperopt 0.1.2
ideep4py 2.0.0.post3
idna 2.10
image 1.5.33
imageio 2.4.1
imagesize 1.2.0
imbalanced-learn 0.4.3
imblearn 0.0
imgaug 0.2.9
importlib-metadata 3.4.0
importlib-resources 5.1.0
imutils 0.5.4
inflect 2.1.0
iniconfig 1.1.1
intel-openmp 2021.1.2
intervaltree 2.1.0
ipykernel 4.10.1
ipython 5.5.0
ipython-genutils 0.2.0
ipython-sql 0.3.9
ipywidgets 7.6.3
itsdangerous 1.1.0
jax 0.2.9
jaxlib 0.1.60+cuda101
jdcal 1.4.1
jedi 0.18.0
jieba 0.42.1
Jinja2 2.11.3
joblib 1.0.0
jpeg4py 0.1.4
jsonschema 2.6.0
jupyter 1.0.0
jupyter-client 5.3.5
jupyter-console 5.2.0
jupyter-core 4.7.1
jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.0.0
kaggle 1.5.10
kapre 0.1.3.1
Keras 2.4.3
Keras-Preprocessing 1.1.2
keras-vis 0.4.1
kiwisolver 1.3.1
knnimpute 0.1.0
korean-lunar-calendar 0.2.1
librosa 0.8.0
lightgbm 2.2.3
llvmlite 0.34.0
lmdb 0.99
lucid 0.3.8
LunarCalendar 0.0.9
lxml 4.2.6
Markdown 3.3.3
MarkupSafe 1.1.1
matplotlib 3.2.2
matplotlib-venn 0.11.6
missingno 0.4.2
mistune 0.8.4
mizani 0.6.0
mkl 2019.0
mlxtend 0.14.0
more-itertools 8.6.0
moviepy 0.2.3.5
mpmath 1.1.0
msgpack 1.0.2
multiprocess 0.70.11.1
multitasking 0.0.9
murmurhash 1.0.5
music21 5.5.0
natsort 5.5.0
nbclient 0.5.1
nbconvert 5.6.1
nbformat 5.1.2
nest-asyncio 1.5.1
networkx 2.5
nibabel 3.0.2
nltk 3.2.5
notebook 5.3.1
np-utils 0.5.12.1
numba 0.51.2
numexpr 2.7.2
numpy 1.19.5
nvidia-ml-py3 7.352.0
oauth2client 4.1.3
oauthlib 3.1.0
okgrade 0.4.3
opencv-contrib-python 4.1.2.30
opencv-python 4.1.2.30
openpyxl 2.5.9
opt-einsum 3.3.0
osqp 0.6.2.post0
packaging 20.9
palettable 3.3.0
pandas 1.1.5
pandas-datareader 0.9.0
pandas-gbq 0.13.3
pandas-profiling 1.4.1
pandocfilters 1.4.3
panel 0.9.7
param 1.10.1
parso 0.8.1
pathlib 1.0.1
patsy 0.5.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 7.0.0
pip 19.3.1
pip-tools 4.5.1
plac 1.1.3
plotly 4.4.1
plotnine 0.6.0
pluggy 0.7.1
pooch 1.3.0
portpicker 1.3.1
prefetch-generator 1.0.1
preshed 3.0.5
prettytable 2.0.0
progressbar2 3.38.0
prometheus-client 0.9.0
promise 2.3
prompt-toolkit 1.0.18
protobuf 3.12.4
psutil 5.4.8
psycopg2 2.7.6.1
ptyprocess 0.7.0
py 1.10.0
pyarrow 0.14.1
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycocotools 2.0.2
pycparser 2.20
pyct 0.4.8
pydata-google-auth 1.1.0
pydot 1.3.0
pydot-ng 2.0.0
pydotplus 2.0.2
PyDrive 1.3.1
pyemd 0.5.1
pyglet 1.5.0
Pygments 2.6.1
pygobject 3.26.1
pymc3 3.7
PyMeeus 0.3.7
pymongo 3.11.3
pymystem3 0.2.0
pynndescent 0.5.1
PyOpenGL 3.1.5
pyparsing 2.4.7
pyrsistent 0.17.3
pysndfile 1.3.8
PySocks 1.7.1
pystan 2.19.1.1
pytest 3.6.4
python-apt 1.6.5+ubuntu0.5
python-chess 0.23.11
python-dateutil 2.8.1
python-louvain 0.15
python-slugify 4.0.1
python-utils 2.5.6
pytz 2018.9
pyviz-comms 2.0.1
PyWavelets 1.1.1
PyYAML 3.13
pyzmq 22.0.2
qdldl 0.1.5.post0
qtconsole 5.0.2
QtPy 1.9.0
regex 2019.12.20
requests 2.23.0
requests-oauthlib 1.3.0
resampy 0.2.2
retrying 1.3.3
rpy2 3.2.7
rsa 4.7
scikit-image 0.16.2
scikit-learn 0.22.2.post1
scipy 1.4.1
screen-resolution-extra 0.0.0
scs 2.1.2
seaborn 0.11.1
Send2Trash 1.5.0
setuptools 53.0.0
setuptools-git 1.2
Shapely 1.7.1
simplegeneric 0.8.1
six 1.15.0
sklearn 0.0
sklearn-pandas 1.8.0
smart-open 4.1.2
snowballstemmer 2.1.0
sortedcontainers 2.3.0
SoundFile 0.10.3.post1
spacy 2.2.4
Sphinx 1.8.5
sphinxcontrib-serializinghtml 1.1.4
sphinxcontrib-websupport 1.2.4
SQLAlchemy 1.3.23
sqlparse 0.4.1
srsly 1.0.5
statsmodels 0.10.2
sympy 1.1.1
tables 3.4.4
tabulate 0.8.7
tblib 1.7.0
tensorboard 2.4.1
tensorboard-plugin-wit 1.8.0
tensorboardcolab 0.0.22
tensorflow 2.4.1
tensorflow-addons 0.8.3
tensorflow-datasets 4.0.1
tensorflow-estimator 2.4.0
tensorflow-gcs-config 2.4.0
tensorflow-hub 0.11.0
tensorflow-metadata 0.27.0
tensorflow-privacy 0.2.2
tensorflow-probability 0.12.1
termcolor 1.1.0
terminado 0.9.2
testpath 0.4.4
text-unidecode 1.3
textblob 0.15.3
textgenrnn 1.4.1
Theano 1.0.5
thinc 7.4.0
tifffile 2020.9.3
toml 0.10.2
toolz 0.11.1
torch 1.7.0+cu101
torchsummary 1.5.1
torchtext 0.3.1
torchvision 0.8.1+cu101
tornado 5.1.1
tqdm 4.41.1
traitlets 4.3.3
tweepy 3.6.0
typeguard 2.7.1
typing-extensions 3.7.4.3
tzlocal 1.5.1
umap-learn 0.5.0
uritemplate 3.0.1
urllib3 1.24.3
vega-datasets 0.9.0
wasabi 0.8.2
wcwidth 0.2.5
webencodings 0.5.1
Werkzeug 1.0.1
wheel 0.36.2
widgetsnbextension 3.5.1
wordcloud 1.5.0
wrapt 1.12.1
xarray 0.15.1
xgboost 0.90
xkit 0.0.0
xlrd 1.1.0
xlwt 1.3.0
yellowbrick 0.9.1
zict 2.0.0
zipp 3.4.0

https://naas.canny.io/admin/board/requests/p/sync-naas-with-google-colab-pre-installed-packages

JupyterLab restart when down.

Sometimes, in a non-linear manner, we can see crashes inside our Kubernetes/AWS cloud infra.

It's an issue related to Kubernetes spawner that does not create deployment but just a pod.

What we should expect is an immediate reboot of the machine anytime it crashes. #chaosstrategy

Linter in GH Action won't commit anything

Correct me if I'm wrong, but you're running the flake8 and black in a GH action, then not throwing an error or committing the changes if the code is badly formatted?

An example of this is shown in #100; although 2 lines have changed, running black shows formatting errors.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.