nsls-ii-esm-2 / profile_collection Goto Github PK

Make sure that the current latest environments are pushed to all the bl machines
Skim the IPython startup files in profile_collection
- Check if it's safe to start bsui remotely (does it touch any hardware?) - use git grep .put or git grep caput
Check for outstanding PRs and issues on beamline repos
Branch https://github.com/NSLS-II/playbooks for the purpose of updating the ESM-2 current_env_tag in the production file

Housekeeping

Update conda

$ su - <your-controls-account>
$ sudo su -
# conda activate base  # should be activated by default, but just in case.
# conda install conda -c https://repo.anaconda.com/pkgs/main/

Add BL staff to the BL GitHub organization as owners
Work with BL staff to commit any un-committed changes to their profiles
Tag the profile_collection repository as-found as 2020C2.1
```
$ git tag -a 2020C2.1
```
Enter a message such as Before 2021C1.0 deployment.
Discuss with BL staff which conda envs they want to keep/delete and perform the cleanup
Check/update the beamline's workstation inventory with the BL staff (in https://github.com/NSLS-II/playbooks/blob/master/production)

In ~/.bashrc, if necessary, update the logging environment variables to use directory /var/log/bluesky/... and add umask 0002.

umask u=rwx,g=rwx,o=rx  # 0002
export BLUESKY_LOG_FILE=/var/log/bluesky/bluesky.log
export BLUESKY_IPYTHON_LOG_FILE=/var/log/bluesky/bluesky_ipython.log

and (if needed) create that directory with the following permissions and ownership:

sudo mkdir /var/log/bluesky
sudo chown -Rv xf21id2:xf21id2 /var/log/bluesky
sudo chmod -Rv g+rws /var/log/bluesky

Remember to source ~/.bashrc.

Remove explicit setting of ophyd logging level from the first startup script
Double-check that if there is an open PR removing handlers, it is merged (and tested).
Update vendored copy of PersistentDict to bug-fixed version (see the updated gist snippet: https://gist.github.com/jklynch/a4366b8900ec0c03883403455ae711b2).
Make sure all repos have the BSD-3-Clause licenses (see NSLS-II-SRX/profile_collection#16 as an example)

Test deployment

Run profile against new environment

 $ BS_ENV=collection-2021-1.0 bsui

It may be necessary to disable jedi in ipython. Add this to the ipython command line arguments inbsui:

--IPCompleter.use_jedi=False

Update the profile as needed to run

Test databroker v2

from databroker import catalog
cat = catalog['xpeem']
cat[-1].primary.read()

Run acceptance tests (if any) and run a representative set of scans for the beamline

Finish

Tag the updated profile as 2021C1.0
```
$ git tag -a 2021C1.0
```
Enter a message such as After 2021C1.0 deployment.
Open and merge a PR against https://github.com/NSLS-II/playbooks setting current_env_tag="2021-1.0" for the ESM-2 beamline
Update BSUI via ansible to use the new environment (http://nsls-ii.github.io/deployment/upgrading-beamlines.html#change-bsui-s-default-environment)
```
$ ansible-playbook -i production update_default_vars.yml --limit=ESM-2 -kK
```

Deployment 2020-2.0 checklist

Previsit

Make sure that the current latest environments are pushed to all the bl machines
Skim the IPython startup files in profile_collection
- Check if it's safe to start bsui remotely (does it touch any hardware?) - use git grep .put or git grep caput
tested RE(Eph.move_to(100, grating='600', EPU='105', shutter='open', branch='B'))
Check for outstanding PRs and issues on beamline repos
Check if beamline specific packages pushed to the BL machines
(Optional) Enable CI

Housekeeping

Add BL staff to the BL GitHub organization as owners
Work with BL staff to commit any un-committed changes to their profiles
Tag the profile as-found as 2019C3.x
Discuss with BL staff which conda envs they want to keep/delete and perform the cleanup
Check/update the beamline's inventory with the BL staff (in https://github.com/NSLS-II/playbooks/blob/master/production)

Test deployment

In ~/.bashrc add the following and remember to source ~/.bashrc afterward.
```
export BLUESKY_LOG_FILE=/var/log/bluesky/bluesky.log
export BLUESKY_IPYTHON_LOG_FILE=/var/log/bluesky/bluesky_ipython.log
umask 0002
```
and (if needed) create those directories, making them group-read/writeable-able:
chown xf21id2:xf21id2 /var/log/bluesky/
chmod g+rws /var/log/bluesky/
removed proxies and path=/opt/conda... from '~/.bashrc, replaced with /etc/profile.d/conda.sh, /etc/profile.d/proxy.sh`
Append to 00-startup.py https://gist.github.com/jklynch/a4366b8900ec0c03883403455ae711b2
Run profile against new environment
- BS_ENV=collection-2020-2.0rc7-1 bsui
  - need suitcase 0.7.0: conda install suitcase=0.7.0
Update the profile as needed to run:
- Replace .value -> .get() if it's in a Device, or with yield from bps.read() if in a plan
- history.sqlite issue - see https://blueskyproject.io/bluesky/metadata.html#persistence-between-sessions
- potential issue with images in databroker V2
Run acceptance tests (if any) and run a representative set of scans for the beamline
- RE(LEEM_plan(Eph.move_to(100, grating='300', EPU='105', shutter='open', branch='B'))
- RE(LEEM_plan(Eph.move_to(100, grating='300', EPU=None, E_start=110, E_stop=112, E_step=0.5))
Tag the updated profile as 2020C2.0
Update BSUI via ansible to use the new environment (http://nsls-ii.github.io/deployment/upgrading-beamlines.html#change-bsui-s-default-environment)