Git Product home page Git Product logo

openshift-ml-workflows-workshop's Introduction

Machine learning workflows for developers

This repository has materials from a hands-on tutorial on machine learning workflows and using OpenShift for these. The goal of this workshop is to get application developers comfortable with some of the habits, concerns, and practices necessary to effectively incorporate machine learning workflows into their general engineering discipline. One notable aspect of the lab is that we offer students the opportunity to solve a real problem with machine learning in five fundamentally different ways -- and these approaches are quick enough to evaluate that attendees will be able to try out several and see which performs the best!

Our slides from presenting the lab at Red Hat Summit 2019 are available as a PDF or as a movie.

This repository is also intended to demonstrate the new reproducible workshops standards that are currently under development. We've intentionally let it fall short of the standard in places and have filed issues about these shortcomings. Feedback is welcome.

openshift-ml-workflows-workshop's People

Contributors

sophwats avatar willb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

openshift-ml-workflows-workshop's Issues

deploy.sh fails with "No package python-openshift available." error.

TASK [ocp4-workload-rhte-analytics_data_ocp_infra : Verify python-openshift installed] ***
Wednesday 03 March 2021 20:04:39 -0500 (0:00:00.126) 0:00:00.162 *******
fatal: [bastion.49f9.sandbox1817.opentlc.com]: FAILED! => {"ansible_facts": {"pkg_mgr": "dnf"}, "changed": false, "failures": ["No package python-openshift available."], "msg": "Failed to install some of the specified packages", "rc": 1, "results": []}

PLAY RECAP *******************************************************************************
bastion.49f9.sandbox1817.opentlc.com : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

Security

How security is done? As pickle files with ML models can execute a random code...In the new OpenShift versions, there are no ip-tables, just network policies, how to isolate pods from each other?

Trouble loading training.parquet dataset

When attempting to run the 01-vectors-and-visualization notebook, I ran into an issue on my environment where the parquet engine did not appear to be loading correctly.

Steps to reproduce:

  1. Using macOS 10.15.5, Zsh
  2. Followed the pre-install instructions in the readme: brew install python which installed python-3.7.7.catalina.bottle.tar.gz and brew install pipenv which installed pipenv-2020.6.2.catalina.bottle.tar.gz
  3. Cloned this repo, and changed current directory to repo root folder
  4. Ran the initialization ipenv install --skip-lock:
~/Development/ml-workflows-notebook  develop ✔                                                                                                                                                                     21d22h  
▶ pipenv install --skip-lock
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Creating a virtualenv for this project…
Pipfile: /Users/carl/Development/ml-workflows-notebook/Pipfile
Using /usr/local/bin/python3.7m (3.7.7) to create virtualenv…
⠇ Creating virtual environment...created virtual environment CPython3.7.7.final.0-64 in 571ms
  creator CPython3Posix(dest=/Users/carl/.local/share/virtualenvs/ml-workflows-notebook-B_cU6XbN, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/carl/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment! 
Virtualenv location: /Users/carl/.local/share/virtualenvs/ml-workflows-notebook-B_cU6XbN
Installing dependencies from Pipfile…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 10/10 — 00:00:24
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
  1. Ran Jupyter:
▶ pipenv run jupyter notebook
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
[I 16:40:40.780 NotebookApp] Serving notebooks from local directory: /Users/carl/Development/ml-workflows-notebook
[I 16:40:40.780 NotebookApp] The Jupyter Notebook is running at:
[I 16:40:40.780 NotebookApp] http://localhost:8888/?token=174bd2b70cb82569facbbd57a47bbae470967f29952e63a1
[I 16:40:40.780 NotebookApp]  or http://127.0.0.1:8888/?token=174bd2b70cb82569facbbd57a47bbae470967f29952e63a1
[I 16:40:40.780 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 16:40:40.786 NotebookApp] 
    
    To access the notebook, open this file in a browser:
        file:///Users/carl/Library/Jupyter/runtime/nbserver-93704-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=174bd2b70cb82569facbbd57a47bbae470967f29952e63a1
     or http://127.0.0.1:8888/?token=174bd2b70cb82569facbbd57a47bbae470967f29952e63a1
  1. Open the 01-vectors-and-visualization notebook, select the first code block and run it (shift + enter):
    image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.