xenonnt / env_starter Goto Github PK
View Code? Open in Web Editor NEWScript to start the XENONnT environment and jupyter server jobs on dali/midway
Script to start the XENONnT environment and jupyter server jobs on dali/midway
Now we have three shell scripts to start the notebook on midway2, midway3 and dali, which is a dirty solution. We should use the same script but just modify the settings according to the partition.
Nicely pointed out by @DonNabla. The issue is that we are not binding X509_USER_PROXY
when we are on dali compute node, which is supposed to locate at /project2/lgrandi/grid_proxy/xenon_service_proxy
. A quick solution would be just to bind the X509_USER_PROXY
copy on dali manually.
Not sure this is a problem within env_starter
-- | -- | -- | -- | --
python | 3.9.16 | /opt/XENONnT/anaconda/envs/XENONnT_development... | None
strax | 1.5.1 | /opt/XENONnT/anaconda/envs/XENONnT_development... | None
straxen | 2.1.0 | /opt/XENONnT/anaconda/envs/XENONnT_development... | None
cutax | 1.15.0 | /dali/lgrandi/xenonnt/software/cutax/v1.15.0/c... | None```
[ershockley@dali-login1 ~]$ ln -s start_jupyter.py /home/ershockley/nt/computing/env_starter/start_jupyter.py
[ershockley@dali-login1 ~]$ ln -s /home/ershockley/nt/computing/env_starter/start_jupyter.py start_jupyter.py
ssh [email protected] /dali/lgrandi/strax/miniconda3/bin/python start_jupyter.py
Starting jupyter job
/tmp/slurmd/job12221635/slurm_script: line 18: /start_notebook.sh: No such file or directory
Traceback (most recent call last):
File "start_jupyter.py", line 310, in <module>
main()
File "start_jupyter.py", line 275, in main
raise RuntimeError("Jupyter did not start inside your job!")
RuntimeError: Jupyter did not start inside your job!
start_notebook.sh
.If one want to start more than one job for testing different container if would be handy to have various log files.
env_starter is doing a similar thing as batchq
, and the only difference is that the job string here is always jupyter .......
So here is no reason to make env_starter an independent package, which increases the complexity of maintenance.
Job started. Logfile is displayed below; we're looking for the jupyter URL.
Starting jupyter job
Using singularity image: /project2/lgrandi/xenonnt/singularity-images/xenonnt-development.simg
WARNING: skipping mount of /project: no such file or directory
WARNING: skipping mount of /scratch/midway3/yuanlq: no such file or directory
Currently, starting a new start_jupyter
job when an old one is already running results in the extension of that old job and does not start a new one.
The idea suggested in this issue is to give the user a way to launch parallel jobs, which can be useful to run code on, e.g., the main and backup environments at the same time, thus allowing one to test new features in slightly different scopes (for example, with different strax/straxen versions which may not be deployed simultaneously on both environments).
As suggested by @JelleAalbers on the XENON1T/analysis Gitter Room, this new feature could be implemented as a flag set by the user when running start_jupyter
.
Deleting a file from within the Jupyter environment can give a permission error, under certain conditions. The error is as follows:
This happens both for deleting newly created files as well as existing files. I have checked the permissions of these files from the command line when I am logged into dali. I can delete files directly from the terminal.
I start the Jupyter environment by calling the "start_jupyter.py" script on the dali cluster. I am using the latest version from the master branch. I am calling the script from within my own folder on dali (/dali/lgrandi/bderonde) with the following command:
python3 /dali/lgrandi/bderonde/env_starter/start_jupyter.py --tag 2023.11.1 --partition dali --cpu 6 --ram 45000 --exclude_node dali001
Using the stable env_starter branch located in "/project2/lgrandi/xenonnt/development/env_starter/start_jupyter.py" gives the same problem.
Supposedly, the issue arises from the fact that the Jupyter environment tries to move a file to the dali trash folder, which I don't have permission to, rather than actually deleting the file. A similar issue is discussed here: (jupyter-server/jupyter_server#1338).
A solution to the problem was suggested on the linked github page as well: adding the "--FileContentsManager.delete_to_trash=False" option to the line of code that calls Jupyter changes the behaviour of Jupyter, where files are now directly deleted instead of moved to a trash folder. This line of code is in the "start_notebook_dali.sh" shell script; the suggested change is highlighted in the picture below:
I tested this solution myself on a local copy of the scripts and this resolved the issue.
It's already implemented for utilix, but not for env_starter yet
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.