Git Product home page Git Product logo

conda-trick-42's Introduction

Conda-Trick-42

How to install conda / python and avoid space problemss, the install wil be saved to sgoinfre and moved back to goinfre for speed.


WARNING

This is just a fix of mine and i have no clue if this will work as intended, be carefull.


Explanation

As you know sgoinfre and goinfre have space, and we need space, but what's the difference ?
goinfre is on the physical computer, and will disapear when you change places. sgoinfre is on the servers and will not dissapear.



Why goinfre ?

You can install conda on sgoinfre and never have problems but IT WILL BE SLOOOOOW.
My solution ? copy the files from sgoinfre to goinfre every day (go have a coffee i guess :) ) and have the path point to ~/goinfre/miniconda so it will work.



How to ?

First install miniconda in goinfre (say yes to everything, including conda init):

cd ~/goinfre
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -p $(pwd)/miniconda



Now we need to add conda to the path, so we add the following line to .zshrc (or.bashrc) above the # >>> conda initialize >>> line: change YOUR_USERNAME with your real username!!

export PATH=$PATH:/mnt/nfs/homes/YOUR_USERNAME/goinfre/miniconda/bin

Then install the python libraries you need:
JUST INSTALL WHAT YOU NEED, THIS IS A DATASCIENCE AI INSTALL FOR EXAMPLE

conda install pytorch torchvision torchaudio cpuonly -c pytorch
conda install -c conda-forge jupyterlab
conda install -c conda-forge numpy
pip   install    pillow

If you get a conda command not found either the install failed, or you forgot to restart your terminal.

Now we copy the install files to sgoinfre so they dont get deleted. The progress % is wacky but it should go to 2-4GB and take about 10 minutes.
If you install new packages later and dont want them to dissapear when u change computers you will have to run this again, it will only copy modified files and wont take long the second time.

rsync -ah --info=progress2 ~/goinfre/miniconda ~/sgoinfre

and now you can work as you like. If you go to a new pc copy your files from sgoinfre to goinfre again with;

rsync -ah --info=progress2 ~/sgoinfre/miniconda ~/goinfre

this should take about 3 minutes

Sanity Check ?

If your install is good and running which python should output blabla/goinfre/blabla

conda-trick-42's People

Contributors

jbarmentlo avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.