The My Jobs app requires the sqlite3 gem, and this requires the sqlite3-devel package. This doesn't seem to be installed by default on CentOS 7. We need to add it:

yum install sqlite-devel

Ric Anderson pointed out that we should mention something about firewalld for the CentOS 7 users.

Whether we point to documentation on how to open ports in it or just recommend they blow away firewalld with the given commands and install iptables.

Wherever we decide to keep our check list for OOD releases, we should add to run through a spell checker.

I did a poor man's version and resulted in fixing 9 different misspellings: 688c65d and bf90b32 and 192b24c

I made a list of words to ignore and ran https://www.internetmarketingninjas.com/online-spell-checker.php and entered https://osc.github.io/ood-documentation/release-1.0/ into the website and selected to craw up to 100 pages.

I added these words to words to ignore:

abaqus
addr
admin
ansys
apps
auth
authn
authnz
awesim
backend
bashrc
bgohar
bindir
binstub
blackslashes
bmcmichael
centos
chmod
cilogon
comsol
conda
conf
config
configs
cpus
cron
devgrp
diff
dropdown
efranz
envdir
erubi
expr
fallback
fastcgi
favicon
filesystem
fontawesome
frontend
github
grep
hostname
hterm
htop
htpasswd
httpd
includedir
init
iptables
javascript
jnicklas
json
jupyter
ldap
ldaps
libdir
lifecycle
logroot
lsof
mapdn
mapfile
matlab
metadata
minitest
mkdir
namespace
namespaces
navbar
ncpus
nginx
nodejs
nonexistant
oauth
oidc
ondemand
ood
openid
openidc
openldap
params
paraview
phishing
phusion
plugin
plugins
prepend
procs
proxied
proxying
qstat
qsub
redir
regex
repo
repos
requiretty
rnode
rstudio
rubydoc
sbatch
sbin
scgi
searchable
sendfile
serverdir
servername
shib
shibsession
slurm
stderr
stdin
stdout
struct
structs
stylesheet
sudo
sudoers
symlink
systemctl
systemd
templating
turbovnc
userinfo
username
uuid
uwsgi
versioned
webdev
websocket
websockets
websockify
wget
whitelist
whitelisted
wildcard
workflow
wsgi
wstunnel
xzvf

It incorrectly flags words in URLs so I had to ignore them. When using a spell checker, ideally it would ignore checking paths and URLs first; otherwise we might be attempted to add "words" like myapps and activejobs to the list of words that are not misspellings. Ignoring code blocks (but not code comments) might be nice. We want to check code comments in documentation too.

┆Issue is synchronized with this Asana task by Unito

In the release notes we do not list anywhere the versions of all the components during the release.

The only way to find out is to look at https://github.com/OSC/ood-apps-installer/blob/master/config.json and/or the latest versions of the components.

The negative of this is that every time we update https://github.com/OSC/ood-apps-installer/blob/master/config.json for patch releases of apps we would need to update this list too.

To replace "Development" section, with far more information like:

1. directions on check out and edit and build (make sure Docker is running, execute this command to build, open this html file to test and verify, commit changes in source directory
2. describe the branching model, how it is loosely based on Gitflow
3. describe Travis CI usage for building master, branches, etc. and comitting to gh-pages
4. describe the convention of master always matching the latest branch
5. describe the new OOD release workflow (following git flow, create a new release-X.Y branch, add commits, when ready for release, merge into master so it is up to date)
6. describe the updated fix (fix on release branch, merge to master and develop)

┆Issue is synchronized with this Asana task by Unito

Check all apps to ensure their release tags are up-to-date.

How about we follow Bootstrap's example.

Code licensed MIT, docs CC BY 3.0.

https://creativecommons.org/licenses/by/3.0/

I believe we should remove the step-by-step instructions on opening ports 80/443 using iptables as it can be insecure since the rules are inserted at the very top of the chain. Also the last command to save the new rules doesn't work on RHEL7.

Instead we should replace it with documentation such as:

https://wiki.centos.org/HowTos/Network/IPTables

This is more of a question than an issue, but I noticed that the configuration files we are using on step 7 are v1, I know v2 is available now.

Per @nickjer we're not using v1 on OOD yet, so I'm not sure what the difference is and when we're making the cutover.

The steps necessary to disable this warning are in the Dashboard's README.md, but should also be included here in the documentation. This has caused confusion I believe at least twice already, so this should be bumped to higher priority.

Note: The Dashboard's documentation cites .env.local, but this has changed to the global configuration path /etc/ood/config/apps/dashboard/env.

Do this: 4722676
This is the safest solution, instead of doing some type of remote. Also you can just git cherry-pick 4722676a704a40c4304ebe5483140afd2f713d51.

But then generate a new auth token to use with Travis so Travis can commit to this repo AND the ood-documentation-test repo.

This page was updated after I ran through the latest run.

In doing so, the iptables commands were removed and replaced with a link to the CentOS documentation.

While it's probably appropriate that we send people to documentation, removing the quick and easy instructions make this step a bottleneck in the install procedure and increase the complexity of the install.

I'd recommend that we keep the existing iptables commands (since firewalld also uses these commands) as an example, and then leave it up to the user whether they want to perform them verbatim or use something more secure.

Many of the installations at this point are going to be for evaluations, so we want to streamline this process and get people up and running.

To my knowledge the overriding of systemd is not common knowledge and finding out how do to do the overrides is somewhat painful and may change based on OS. For CentOS / RedHat it may be worth telling people to drop the override contents into a .conf file in /etc/systemd/system/httpd24-httpd.service.d.

Add short tutorial on installing the Jupyter app, as a main section under "Install Desktops", by making use of the app developer views.

Also, address other concerns.

See requested changes in #53 (review)

Instructions for automated app installation can be added if we want. This allows for unattended retrieval of files. After installation, I deleted the deployed apps and re-ran the app installation with the installer. Everything tested ok.

The commands to install apps are:

cd ~/ood/src
wget https://raw.githubusercontent.com/OSC/ood-apps-installer/master/Rakefile
scl enable rh-ruby22 git19 nodejs010 -- rake
sudo scl enable rh-ruby22 -- rake install

Once OSC/ood-apps-installer#1 is implemented we won't need to worry about sync'ing the version numbers to the Rakefile either, so the same script can be used for updating.

See https://github.com/OSC/ood-documentation/blob/develop/source/install-ihpc-apps.rst https://osc.github.io/ood-documentation/develop/install-ihpc-apps.html

The table should be updated similar to how https://github.com/OSC/Open-OnDemand/issues/42 updated the similar table in the README so VMD is listed.

Follow GitHub Flow off of develop branch for this. We actually use Gitflow (mostly) which is where you use the GitHub Flow on the develop branch for developing, but when we go to create a new release thats when we create release branches and eventually merge releases to the master branch. But Jeremy or I will take care of the updating the release and master branches when the time is appropriate.

Follow https://guides.github.com/introduction/flow/ to create branch, add modification, and make a PR

We are using VMD now in OSC OnDemand in production but have not advertised it as an Interactive app we run at OSC. Add this to the table of "Applications: OSC's Interactive Apps" in the README, where the name is VMD and the repo is https://github.com/OSC/bc_osc_vmd.

This is possible with https://robpol86.github.io/sphinxcontrib-versioning/index.html

For example, what version of Sphinx, and what plugins are required? Example of using Docker to set this up for you. Etc.

We are lacking documentation on how to configure the cluster config for PBS Pro here:

https://osc.github.io/ood-documentation/master/installation/add-cluster-config.html#example-configs

This can lead to confusion for sys admins when installing OOD:

It isn't obvious the above is describing how to change a line.

The ldap URI for osc does not work correctly.

Updating to:

  - 'AuthLDAPURL "ldap://cts06.osc.edu/ou=People,ou=hpc,o=osc?uid"'

fixed the issue.

Add section on Dashboard announcements to Components => Applications => Dashboard App => Announcements

We might also consider adding sections for all the other configurations that can be done to the Dashboard and apps in each section. Maybe we should make separate sub pages like before so we can link to them from an uber Customization section under "Authentication". For that we can open separate issues and address.

We need a new section called "Interactive Apps" under the "Applications" section.

I am too lazy to do pull request, so, I report this small error this way.

In https://osc.github.io/ood-documentation/master/app-development/tutorials-interactive-apps/add-custom-queue/local-dynamic-list.html, section 4, the code should be
...
output, status = Open3.capture2e(cmd)
if status.success?
queues = output.split("\n").map(&:strip).reject(&:blank?).sort
else
...
not
...
queues = out.split("\n").map(&:strip).reject(&:blank?).sort
...

Related to https://github.com/OSC/Open-OnDemand/issues/27

Currently OnDemand is made up of many individual packages in GitHub, each with their own version. We advertise the various versions of the repos of OnDemand version 1.0, 1.1, 1.2, etc. in the Release Notes section of the documentation. Also, the version of the main OnDemand rpm and the version of the ood-apps-installer corresponds to the OnDemand version (i.e. 1.1, 1.2, 1.3) and those act as a manifest for the versions of the other components installed.

This appears to be difficult to find, however. We may want to find a more prominent location for these two tables:

https://osc.github.io/ood-documentation/release-1.2/release-notes/v1.2-release-notes.html#infrastructure

Also, we have discussed merging repos into one repo. We could do this for most of the core repos, but still may have the need for some separate repos. So after merging repos there still may be a need to clearly document what individual components make up a version of OnDemand.

MVP is to add logo, link to website, and use OH-TECH colors (red or the OH-TECH grey or the "jazzy" background that communications suggested - I'll share that in an email to WIAG.

Also, keep in sync loosely with https://github.com/OSC/Open-OnDemand/issues/27

https://osc.github.io/ood-documentation/master/installation/modify-system-security.html

A question was raised:

The Open OnDemand website says you have to disable SE Linux. Is
that just on the web front end node? Or is it on the compute nodes as
well?

Troy answered with note:

I would think that's only on the web node, unless there's something that
OOD jobs run on a compute node that requires it. In any case, since SE
Linux is a security feature, it might be good to a) be specific as to
whether that means on web or compute nodes and b) document briefly why
it's not a supported configuration (e.g. "it can cause permission
problems with user-installed apps" or whatever) rather than just making
a blanket statement of "turn it off". I imagine that security officers
at sites with stronger security requirements than ours might looks
askance at user-facing software that requires SE Linux to be disabled
without explaining why.

(Also, there are plenty of other things that could force you to disable
SE Linux on compute nodes... Lustre, for example.)

It was mentioned that we should explore sub-projects as a way to reduce our left-hand navigation menu's size.

I notice when I navigate between the subsections of add-custom-queue the navigation on the left disappears because on a new page you are scrolled to the top and you have to scroll down to see. The readthedocs theme I think supports "subprojects" https://docs.readthedocs.io/en/latest/faq.html#how-do-i-host-multiple-projects-on-one-cname (or that might just be the readthedocs site itself...). My guess is this would result in the left navigation being devoted entirely to that subproject, which could be "App Development" or even "Interactive App Development". This is outside the scope of this PR though, maybe worth opening another issue.

Following an example for inspriation: https://community.box.com/t5/Managing-Your-Content/Box-Notes-Notices-and-Known-Limitations/ta-p/340

Are there others?

We would call out important issues and limitations:

1. Browsers supported (across all apps)
2. Lack of support for Job Arrays
3. Current issue with cat-ing very large files in the shell app

Currently configuration information is sparingly on the dashboard wiki, and in email exchanges with external OOD sites. See the email exchanged titled "OOD on Bridges" to see a good example of this.

All of this should be in a config section for the Dashboard app, and linked to as "something to do after installation".

The documentation on nginx_stage needs to be updated with the new configuration options:

• user_regex (added in 0.3.0)

Although running:

sudo service httpd24-httpd restart

works for RHEL7, that above command is becoming deprecated.

Instead we should use:

sudo systemctl restart httpd24-httpd

@brianmcmichael should confirm the above command is correct when he reinstalls OOD on our test RHEL7 server.

I need to update the example Jupyter app and the Desktop app documentation to use the new Batch Connect style.

By default, you can just do mkdir -p ~/ondemand/dev in order to create a new dev directory. So that should be a. in https://osc.github.io/ood-documentation/master/app-development/enabling-development-mode.html

The upcoming nginx_stage will display a warning message when an app is first accessed. This will be true for all the system apps right after a fresh installation.

A temporary workaround is documenting at the end of the installation directions to generate new app configs for our system apps, e.g.:

sudo /opt/ood/nginx_stage/sbin/nginx_stage --user "$SUDO_USER" --sub-uri '/pun' --sub-request '/sys/dashboard'
...

The above needs to be verified as well, since I am pulling it off the top of my head.

The App Development overview page has this description:

Passenger apps are rack based Ruby apps, wsgi based Python web apps, or Node.js apps that follow a convention for the app’s startup file.

• https://osc.github.io/ood-documentation/master/app-development.html

Unfortunately, this is not prominent enough. The Tutorials: Passenger Apps section should have a better description added to it as well: https://osc.github.io/ood-documentation/master/app-development/tutorials-passenger-apps.html

Perhaps even a page "What is a Passenger App?", maybe added to an FAQ section of the App Development section and wherever we mention "Passenger App" link to this FAQ.

A fuller description:

Rack based ruby app, WSGI based Python app, or Node.js app that has a “startup file” following the convention https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_startup_file that Passenger Web Application Server uses to start the process. OnDemand uses the NGINX Integration mode https://www.phusionpassenger.com/library/walkthroughs/basics/ruby/fundamental_concepts.html#multiple-integration-modes for Passenger, where NGINX and Passenger and individual web app processes all communicate using unix domain sockets.

1. Use console for everything
2. Take advantage of the fact that the lexer puts <span class="gp">$</span> around the $ and apply custom style user-select: none and padding-right: 5px to the .highlight-console span.gp class
3. Remove the proceeding whitespace to the command, so it is typed in as $tree /etc/ood/config

The result is this:

Original full issue description

Using "shell" format without preceding $ for commands users have to enter to do things makes it easier to copy a line: just click 3 times to highlight the line. This does copy the newline so if your shell isn't configured to not auto-execute the command when a newline is placed in (i.e. bash vs zsh) it could be bad if you want to paste, then inspect, then execute. But otherwise it is very convenient to follow through a set of directions.

One approach is to use console format with preceeding $ for read only with lots of output. For example, in https://osc.github.io/ood-documentation/master/release-notes/v1.3-release-notes.html#upgrading-from-v1-2 we use shell formatting for all the commands you will copy and execute, where as we use the console formatting for $ tree /etc/ood/config because it conveniently formats the code below it.

Two problems with this approach:

1. One problem is that this is inconsistent and if you are copying and pasting verbatim, as in testing an install in a VM, you might accidentally enter this full string $ tree /etc/ood/config instead of tree /etc/ood/config.
2. In the installation instructions there are many places where users enter commands AND we want to display the output.

What do other sites do? They are all over the place.

1. https://docs.pipenv.org/ just puts $ in the example code
2. https://theforeman.org/manuals/1.1/index.html#3.2.1Installation displays commands shell like, without displaying output
3. Digital Ocean displays commands and output in separate boxes which is an option https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-16-04
4. Heroku uses a cool approach where the styling looks like "console" with the $ but the doubleclick actually highlights only the first line. This is because they are using https://developer.mozilla.org/en-US/docs/Web/CSS/user-select on the $ . Example: https://devcenter.heroku.com/articles/exec

The ideal might be to:

1. Use console for everything
2. Take advantage of the fact that the lexer puts <span class="gp">$</span> around the $ and apply custom style user-select: none and padding-right: 5px to the .highlight-console span.gp class
3. Remove the proceeding whitespace to the command, so it is typed in as $tree /etc/ood/config

The result is this:

Copyright date can be found in source/conf.py and needs to be updated on the develop, master, and release-1.3 branches.

We have our first client walking through the Jupyter tutorial and he found step 2 confusing due to its terseness.

I need to explain further the following topics:

• what is so special about ~/ondemand/dev (why not use ~/ood/dev)
• why is there no installation step
• how do you send a GET request

Currently directions for enabling Interactive Apps in the infrastructure are only located in the "Install Desktops" section. This should be factored out and probably placed in the "App Development / Interactive Apps" section.

Then links should be added to "Install Desktops" and the corresponding Interactive App tutorials (e.g., Jupyter).

It was suggested that we add some command line commands that can be called to test ldap server connectivity as well as attributes.

Example:

ldapsearch -x -H ldaps://ldap.domain.com:636 cn=username

┆Issue is synchronized with this Asana task by Unito

Add a section discussing how to contribute to the Documentation (maybe to the other projects?).

1. update https://osc.github.io/ood-documentation/installation/modify-system-security.html?highlight=iptables to include firewalld
2. ???

The latest nginx_stage 0.3.0 now has configuration in a global location. The installation docs should reflect this.

Update this section:

https://github.com/OSC/ood-documentation/blob/27c241d70501e415c891b51537a7a94fdbf4e9e4/source/installation/install-pun-utility.rst

Use substitution in the reST documents for OOD infrastructure/app versions, for easier updating.