The daily or preview labels behave like "latest"

However, forcing a build of a version with override / by specifying a version will also replace these labels, making them potentially defunct / not referencing the "latest daily" or "latest preview."

This is especially pernicious for "preview," since that will get replaced by a daily version, for instance.

Similar to what we did for product versions, it will probably be worth adding a Makefile helper for when we have to update the R versions (i.e. related to #17). Locations:

• in each Dockerfile
• in rstudio-pm.gcfg

• make sure directories exist
• ensure launcher startup problems are detected in logs
• see about better ways to funnel logs to stdout...?
• see about segregating launcher from rsp

Our current implementation is a bit confusing, so we should do a better job explaining!

[email protected] is currently inaccessible to outside traffic (i.e. no customer can send an email to that address). Should we use a different email address for the maintainer? We could create a public google group dedicated to these images. That is a pattern that I know is in use elsewhere.

[email protected]?

This repo's README.md contains the following:

Licensing
Using the RStudio Server Pro Docker image requires to have a valid License. You can set the RSP license to use this in three ways:

1. Setting the RSP_LICENSE environment variable to a valid license key
2. Setting the RSP_LICENSE_SERVER environment variable to a valid license server / port
3. Mounting a /etc/rstudio-server/license.lic single file that contains a valid license for RStudio Server Pro

While the above implies that Offline Activation is not supported, this could be stated explicitly.

The above is for RSP, but the same applies to RSPM and RSC.

(See 53862.)

We might also consider adding a comment to this effect, in the Dockerfiles:

• RSP
• RSPM
• Connect

The RStudio Server Pro product has been rebranded to RStudio Workbench. As a result, we should:

• change the server-pro directory to workbench
• update all CI paths to function
• change RSP_ prefixed env vars to RSW_ prefixes
• Add NEWS.md entries to note the breaking change (this one is going to be a bit pernicious. Helm charts will be affected, users will be affected on next container pull, etc.). Maybe we should see how close the RSW preview release is to see if we can coordinate
• Trace through docs and update any server-pro / Server Pro / etc. naming
• Update Makefile targets and related docs
• Update docker-compose.yaml examples and naming

We are insulated a bit because the workbench image is not in wide circulation / publicly documented yet. However, the helm chart is using it, so Kubernetes nodes could pull a new image and change licensing / etc. behavior.

Related to #123 b/c better tagging would help protect us from breaking changes like these (i.e. those who are concerned about breaking changes should pin to a particular build)

Public sector organizations use centos7 / RHEL 7 as their default images. They'd like to rely on our docker images rather than building their own. Making docker images for this distro would be a big plus.

RStudio Server Pro handles the license file differently than the other products, and I believe this command disregards the license file location

We should fix that!

We should also consider using a dedicated license directory, that way a Kubernetes secret volume could "automatically update." (And bonus points for consistency across products! Unfortunately a breaking change though...)

Note: A container using a ConfigMap as a subPath volume will not receive ConfigMap updates.

From here

There are a few things we can do in all Dockerfiles right now that are general security best practices. And which as an additional benefit will also slim down the size of the images.

I assumed multi-stage builds would be helpful here, based on past experience. But given we aren't actually compiling anything in here, I don't see it being helpful. Another look can be done by reading through
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#use-multi-stage-builds and similar pages. You can also see if I missed anything from https://sysdig.com/blog/dockerfile-best-practices/ or similar resources

• Ensure connect, rspm, and server-pro containers work before and after all of the changes.
• Ensure we are always cleaning up in the same layer that we do work. For instance, always run apt-clean in the same layer as an install, or removing a file we download after we install
• Wherever any explicit versions are mentioned in Dockerfiles, see if we are at the latest version. If not, make sure to understand why (as there are some reasons, like for the R version) or upgrade.
• The above may all be good and we have nothing to do! If not, after the PR is merged in, understand how publishing works and make sure this happens

The biggest problem here is for RStudio Connect. If we change R / Python versions, users who are using the image stand to experience all of their content breaking at once.

We should think through how to make that switch. Should we be providing an image that will only do additive R / Python version additions for "LTS"? At least for very long periods of time? Will product changes make this easier? Actual user feedback preferred 😄

For security reasons the rocker/rstudio images force the user to provide a password, instead of the default passwords:

docker run --rm -p 8787:8787 -e PASSWORD=yourpasswordhere rocker/rstudio

IMO the rstudio provided images should do the same, to prevent the risk that people stand up open containers using the default password.

It would be nice to test that services actually run successfully / properly in goss. However, this is hard because goss tests are parallelized.

We need to add licenses into secrets for GitHub Actions and make easy make targets / etc. for testing locally (i.e. - RSC_LICENSE in docker-compose.test.yml will use an env var that is defined in the user's environment variables without needing to put them in a .env file / etc.). You can look at how licenses are handled here if that's helpful: https://github.com/rstudio/sol-eng-docker#getting-started

A couple of possible patterns:

1. Run the product inside of "timeout" so that we get an expected output if the product runs successfully for X seconds
   

   rstudio-docker-products/connect/test/goss.yaml

   Line 35 in de5f9c3

   "timeout --signal=SIGINT 7 /opt/rstudio-connect/bin/connect --config /etc/rstudio-connect/rstudio-connect.gcfg":

   • If we use this pattern, we should probably test / document the different behaviors
   • Downside: quite confusing setup, not using the startup script (in this example)

2. Run the product interactively:

   • i.e. https://github.com/nahara7/rstudio-docker-products/blob/08fd9e03e0456d2908ac213f37d1ab5979e06f32/package-manager/test/goss.yaml#L45
   • Downside: if the product is successful, then it will not return / the goss test will timeout
   • Also, we should look into why this is exiting 0 when "unhappy" (i.e. the product failed)
   • Downside: not using the startup script (in this example)

3. Run the product as a part of the testing script, and then check for listening in goss:

   • We could probably use tini -- /usr/local/bin/startup.sh consistently because that is the CMD/ENTRYPOINT for all of the product images. We will need to run into the background, maybe write to a log file, and then cat the log file if the tests exit unhappily? Getting logs is a secondary matter to testing that the product functions 😄
   • could inject into https://github.com/rstudio/rstudio-docker-products/blob/main/server-pro/test/run_tests.sh
   • Downside: we are (potentially) more limited in what we can test in the goss.yaml because the service will already be running. This should be mostly academic / not a real downside though.

I lean towards option 3. What are your thoughts!?

A couple of notes:

• we will likely need to add the privileged: true flag to docker-compose.test.yml
• we will need to take a look at GitHub Secrets and see how they behave across forks, etc. @nahara7 if you'd like to work on this repo so that we can get "true" CI, then we can add your access and use a branch here rather than your fork!
• I would recommend testing some of these things interactively so you get a sense for what the goss tests are doing
  • i.e. start the container with bash i.e. docker run -it --rm -e RSC_LICENSE --privileged rstudio/rstudio-connect:latest bash
  • then run the command tini -- /usr/local/bin/startup.sh, etc.

Acceptance criteria:

• goss tests for rstudio-workbench check that the service runs properly
• goss tests for rstudio-workbench fail if there is misconfiguration of the product that prevents it from starting (i.e. add an invalid line like hello=friend to the rserver.conf file)
• goss tests for rstudio-connect check that the service runs properly
• goss tests for rstudio-connect fail if there is a misconfiguration of the product that prevents it from starting (i.e. add an invalid line like Chicken=Little to the config)
• goss tests for rstudio-package-manager check that the service runs properly
• goss tests for rstudio-package-manager fail if there is a misconfiguration of the product that prevents it from starting (i.e. add an invalid line like Chicken=Little to the config)

The current rsp-session tarball originates from a xenial base.

Given that Xenial soon reaches its end of support and that the libssl1.0 requirement blocks using focal as the FROM: layer (because libssl1.0 cannot be installed on focal), I was wondering if there are plans to provide newer sources for the rsp-session tarball?

We've tried other Ubuntu refs than Xenial in the download URL but it seems no resources exist.

We are building our own images right now (for use in RSP + Kubernetes) from the Dockerfile of this repo (thanks for the initial work!).

cc @klrmlr

Hi,
We are trying to use your image FROM rstudio/rstudio-connect:1.8.6.1 in Dockerfile. However after deep analyses this image consist of many critical vulnerabilities mostly connected with python 3.6.5:

• https://nvd.nist.gov/vuln/detail/CVE-2019-10160
• https://nvd.nist.gov/vuln/detail/CVE-2019-9636
• https://nvd.nist.gov/vuln/detail/CVE-2018-18074
• https://nvd.nist.gov/vuln/detail/CVE-2019-10160
  ...
  On the other hand latest stable python is 3.9.0 https://www.python.org/downloads/release/python-390/ where most of that vulnerabilities are solved.

Questions

• Is there a chance to update it in base image ?
• Another point, I know that a docker image with many versions of python can be really huge, is there a chance to have a rstudio-connect image without a python at all?

Thank you.

In our startup scripts, we send deactivation to /dev/null with > /dev/null 2>&1. Is there output that we want to hide from our logs...? Why do we do this?

We should confirm and then add to developer documentation

An image should not get pushed if the tests fail...

Dear R-Studio/Github,

Using image rstudio/rstudio-server-pro on an Ubuntu 18.04 LTS virtual machine I can start a container instance without persistent storage Home and configurations:

$ docker run --name rstudiopro --privileged -it -p 8787:8787 -p 5559:5559 -d rstudio/rstudio-server-pro:latest

I also can start a stable running container instance making the home directories persistent:

$ docker run --privileged -it -p 8787:8787 -v $PWD/data/rsp:/home -d rstudio/rstudio-server-pro:latest

But when I start an instance with persistent configuration:

$ docker run --privileged -it -p 8787:8787 -v $PWD/data/rsp:/home -v $PWD/server-pro/conf/:/etc/rstudio -d rstudio/rstudio-server-pro:latest

the container immideately stops running giving an exited code 124. This code is nowhere to be found or explained on the internet and I can not explain why this last command is giving this much trouble. Can someone help me out?

Thanks in advance for thinking along and reacting and kind regards,

Aad Dijksman (the Netherlands)

RStudio Server has a helpful --verify-installation that is very hard to use in docker. It would be nice if our entrypoint pivoted on an environment variable or something to know that it is in "diagnose" mode and runs the --verify-installation to a provided directory (ideally a shared / persistent directory...)

For the postgres docker image, they have the notion of an "init" directory that you can put scripts into to be executed.

https://github.com/docker-library/postgres/blob/0d0485cb02e526f5a240b7740b46c35404aaf13f/12/docker-entrypoint.sh#L128

I think having such a directory (for RStudio Server Pro, at least) may be helpful for making the images more extensible. It also may be another way of starting the launcher 🤷‍♂(Related to #9)

The use case I have presently is I want to symlink the /etc/rstudio/load-balancer file to somewhere else (workaround for this: https://github.com/rstudio/rstudio-pro/issues/1380), and extending our entrypoint script is challenging.

Maybe the Makefile becomes the source of truth for version numbers? And then make update-versions updates the Dockerfile defaults and the *_VERSION information that we use in the build?

This goes for both Docker Hub and GHCR.

When we push new images to Docker Hub and GHCR, we need to ensure that the updated READMEs are submitted that go along with the release via API calls.

Just wanted to throw this one out there since dailies are pretty commonly used by the community for RStudio Server. It could also be useful for internal testing purposes.

Thoughts:

• Maybe a daily tag... we could also tag version numbers? Or not, as that may get confusing for users
• We could also make this a completely separate image... rstudio/rstudio-server-pro-daily and just tweak the version numbers somehow / keep the same codebase
  • Random idea. A pre_build hook that overwrites the version number with the latest version available only if the tag or image being built is daily
• Drop the idea and do this elsewhere 😄

RStudio Workbench currently has a very complicated startup.sh / entrypoint script and logging structure that:

• writes logs to several directories (i.e. /var/log/, /var/lib/rstudio-server/monitor, /var/lib/rstudio-launcher, etc.)
• tails these logs to stdout (which interleaves output together in potentially confusing ways, but only works for files that are initialized with proper permissions by the startup script itself)
• tails the logs again at container shutdown (potentially duplicating messages) to ensure that no log messages are missed (if the container dies before another "tail" polll)

In daily builds for RStudio Workbench (see rstudio/rstudio-workbench-preview or ghcr.io/rstudio/rstudio-workbench-preview images), we purportedly have some logging improvements! We should:

• Edit logging.conf so that all logging output (ideally for workbench AND launcher) comes to stdout / stderr
• Ensure that no log files are being written to disk
• Ensure that log level changes (i.e. info -> debug) maintain stdout/stderr and no files being written to disk
• Change the startup.sh script to remove all of the "log file tailing, initializing, etc." stuff

A few notes:

• We should check on the status of "workbench running launcher as a subprocess" - so we do not have to start the process ourselves
• It is possible to start the container interactively, mount config files, and then run the different services manually so that iteration is quicker

Acceptance Criteria

• RSW container prints all logs to stdout and stderr
• RSW container does not use any of the default file logging locations
• When the RSW log level is changed, no files are used

The current set of packages that get installed have version incompatabilties:

ERROR: typer 0.3.2 has requirement click<7.2.0,>=7.1.1, but you'll have click 8.0.0 which is incompatible.
ERROR: pathy 0.5.2 has requirement smart-open<4.0.0,>=2.2.0, but you'll have smart-open 5.0.0 which is incompatible.
ERROR: tensorflow 2.5.0 has requirement h5py~=3.1.0, but you'll have h5py 3.2.1 which is incompatible.
ERROR: tensorflow 2.5.0 has requirement numpy~=1.19.2, but you'll have numpy 1.20.3 which is incompatible.
ERROR: tensorflow 2.5.0 has requirement six~=1.15.0, but you'll have six 1.16.0 which is incompatible.
ERROR: tensorflow 2.5.0 has requirement typing-extensions~=3.7.4, but you'll have typing-extensions 3.10.0.0 which is incompatible.

We should set specific versions in the pip install list to resolve these messages.

The problem appears to be mainly that tensorflow requires older versions of some packages. This list installs without warnings:

pip install \
    altair \
    beautifulsoup4 \
    bokeh \
    cloudpickle \
    cython \
    dash \
    dask \
    flask \
    gensim \
    h5py~=3.1.0 \
    keras \
    matplotlib \
    nltk \
    numpy~=1.19.2 \
    pandas \
    pillow \
    plotly \
    pyarrow \
    requests \
    scipy \
    scikit-image \
    scikit-learn \
    scrapy \
    seaborn \
    six~=1.15.0 \
    spacy \
    sqlalchemy \
    statsmodels \
    streamlit \
    tensorflow \
    typing-extensions~=3.7.4 \
    xgboost

Hi All,

I have added Server and SMTP details in rstudio-connect.gcfg, while creating container License get deactivated due below yellow color highlighted errors.

Error details:

• unset RSC_LICENSE
• unset RSC_LICENSE_SERVER
• /opt/rstudio-connect/bin/connect --config /etc/rstudio-connect/rstudio-connect.gcfg
  2021/07/15 14:06:31 Resource limits: {"nofile":{"soft":1048576,"hard":1048576}}
  2021/07/15 14:06:31 RStudio Connect running with PID 18
  2021/07/15 14:06:31 Starting RStudio Connect v1.8.8.2
  2021/07/15 14:06:31 Loaded server migration from '/etc/rstudio-connect/rstudio-connect-migration.gcfg'
  2021/07/15 14:06:31 Error: Couldn't read configuration file '/etc/rstudio-connect/rstudio-connect.gcfg': warnings:
  can't store data at section "SMTP", variable "DataDir"
  can't store data at section "SMTP", variable "Address"
  Deactivating license ...
• deactivate
• echo 'Deactivating license ...'
• /opt/rstudio-connect/bin/license-manager deactivate

Email Setup and SMTP configuration added in rstudi connect config file and building the docker image: I have provided credentials and senderEmail in place of User, Password and Host.

[Server]
Address = https://rstudio-connect-pks.dell.com
EmailProvider = SMTP
SenderEmail =
EmailSubjectPrefix = Rstudio
ViewerKiosk = false
HideEmailAddresses = false
MailAll = false

[SMTP]
Host =
Port = 587
SSL = true
User =
Password =

Noticed a minor error in the licensing instructions on the readme:

Licensing
Using the RStudio Server Pro Docker image requires to have a valid License. You can set the RSP license to use this in two ways:

"two" should be "three" as we then describe the three methods (regular, floating, and file)

https://github.com/rstudio/docker-rstudio-products/blob/a54ac2c499d0469a274e04c8977e908d20bb8c9d/connect/startup.sh#L15

Otherwise, this license variable will be inherited by all child processes

There is a risk in leaving "old" images around too long. We should evaluate having a "janitor" to clean up old and defunct images

We should explore whether setting a default repository (i.e. https://packagemanager.rstudio.com) would be worthwhile

Right now image testing is limited (no licensing) and does not gate a docker push. We should fix that!

• Explore and attempt each of the items listed below in the comments
• For items that don't look possible now, explain why in a comment or create a follow-up ticket

In particular, the docs are hard to read from being one giant list. It would be great to have a more concise presentation focused on each image.

Hi All,

Please find the below error logs and how to rectify this Unable to launch a sandbox with Applications.RunAs as 'rstudio-connect'; may not be a real user or may not have sufficient permission in docker file. Could you please help us.

2021/06/21 16:51:03 Configuring licensing to use a local license...
2021/06/21 16:51:03 Unable to obtain a valid license: Your RStudio Connect license has expired. Please contact your Customer Success representative or email [email protected] to obtain a current license.
2021/06/21 16:51:03 Using the normalized Server.Address: http://localhost:3939/
2021/06/21 16:51:03 Error: Unable to launch a sandbox with Applications.RunAs as 'rstudio-connect'; may not be a real user or may not have sufficient permission: exit status 1; output: 21 Jun 2021 16:51:03 [rsandbox] ERROR system error 1 (Operation not permitted); OCCURRED AT int main(int, char* const*) src/cpp/server/sandbox/SandboxMain.cpp:158; LOGGED FROM: int main(int, char* const*) src/cpp/server/sandbox/SandboxMain.cpp:159

• deactivate
• echo 'Deactivating license ...'
• /opt/rstudio-connect/bin/license-manager deactivate
  Deactivating license ...

DockerHub needs to link to the source on GitHub - add such links to each README!

I.e. the readme at connect/README.md should link to https://github.com/rstudio/rstudio-docker-products/tree/main/connect, perhaps? Or the root of the repo, to be a bit more fault-tolerant 🤷‍♂️

Also would be good to link to the repository source as well... (i.e. the connect/README.md should link to docker hub).

Customer request from internal ticketing system, ticket 51638:

Regarding tags - one of tags we had in conf file was removed (rstudio/r-session-complete:centos7-1.3.842-1) and made confusion among our users.
Therefor I would suggest to use some tags like centos7-latest or similar that would always point to latest version for specific base image.

Hi All,

I want to run shinyapp in rstudio connect and finally need to deploy in Kubernetes. For that I am writing single Dockerfile, but unable to process the same.

I have seen your repo for RstudioConnect having two dockerfiles(Connect and Connect-content-init) and I tried to merged in single docker of Connect folder and build the same with docker compose yaml file but Image is not working.

Could you please share some single Dockerfile for Rstudio Connect to host shinyapp application.

We should evaluate supervisord (in use by https://github.com/rstudio/sol-eng-demo-server) and S6 (used by rocker and here: https://www.google.com/url?q=https://github.com/just-containers/s6-overlay&sa=D&source=editors&ust=1620667214417000&usg=AOvVaw2aD27QL-1VKAbNr9xWCDD1)

This should also be extensible for things like sssd / etc. (And we should consider adding sssd to the container anyways). Ideally with a way to stop services as well

Is it currently possible to connect to a license server rather than passing a License Key? If so, it would be great if the instructions could be added to the readme.

Also, not sure if this would make sense, but it might be useful to have a docker image added here that could be used to run the license server.

https://github.com/rstudio/docker-r-session-complete/blob/5d458fefb7ff0ec9865498e383c28a79aa0022f2/r-session-complete/centos7/Dockerfile#L10

The r-session-complete images for both centos and ubuntu utilize py v3.7.7 whereas the Docker images we provide utilize 3.6.5. These are out of sync.

Can we make these in sync?

For reference: dockerfiles

Hello,

I am using base image in docker:

• rstudio/rstudio-connect:1.8.4.2-2

$ /opt/rstudio-connect/bin/connect
2020/11/30 10:33:19 Resource limits: {"nofile":{"soft":1048576,"hard":1048576}}
2020/11/30 10:33:19 Starting RStudio Connect v1.8.4.2-2

I would like to restart rstudio-connect application inside of container (I would like to keep container running) for migration script and also for getting logs/backup. All this scripts required to have stopped application, but restart of app is not available to use it. I was able to install systemd, but not able to restart app:

$ systemctl status rstudio-connect
System has not been booted with systemd as init system (PID 1). Can't operate.
$ systemctl stop rstudio-connect
System has not been booted with systemd as init system (PID 1). Can't operate.

I also tried https://stackoverflow.com/questions/52197246/system-has-not-been-booted-with-systemd-as-init-system-pid-1-cant-operate but without success.

Is there any command for restart application itself ?

Do you have any other suggestion ?
If yes, please add also in documentation.
Thanks

I think to make this container ideal, we should make launcher "optional." A few reasons for this:

• If you want to use legacy behavior without the launcher (i.e. project sharing, load balancing local sessions, etc.)
• If you want to run the launcher in a separate container from RSP. This pattern isn't common yet, but it is something we may want to think about as an option longer term 😄

Dear people,

Using the RStudio-server-pro image I understand that creating the image several configuration files are put in the /etc/rstudio directory of the container. Starting a container via docker-compose-up -d with persistent user and configuration data the container crashes immideately due to missing configuration files in the persistent directory.
Inspecting the container I see:

    "HostConfig": {
        "Binds": [
            "/data/docker/docker-data/users:/home:rw",
            "/data/docker/docker-data/etc/rstudio:/etc/rstudio:rw"

Running a container instance without persistent configuration I find several config files in the /etc/rstudio directory within the container.

As I suppose, but correct me if I am wrong, should mounting the container config directory /etc/rstudio/ with a persistent location /data/docker/docker-data/etc/rstudio/ create a situation in which I can find those config files in the persistent directory also. Or am I wrong? And if I am wrong (which happes a lot), how can I get those configuration files into that persistent location without crashing containers?

Thanks in advance and kind regards,

Aad Dijksman (The Netherlands)

In the RSP image in particular

Feedback from a customer (in private ticket 58975):

there's a script on the rstudio docker image at /usr/local/bin/startup.sh that tails a bunch of log files. As of now, it pulls from /var/lib/rstudio-launcher/Local, in addition to other common logs. It'd be useful if that script figured out how to pull in the log file of actual launcher profile in use (in my case this is Kubernetes).

I'm trying to build a pipeline that extends these rsp and rsc images. However, I have to build from source since I need to use a custom Ubuntu OS layer (instead of rstudio/r-base). I was hoping to pull from tagged versions of the source (instead of master) so that my pipeline can maintain version re-producibility. Do you have any plans to add source code tagging in your git hooks? Or do you think there is some better way I can extended these base images and maintain source versioning?

r-session-complete uses R v4.x whereas the RSC image uses R v3.6.2.

This causes confusion for folks using our Docker images and potential issues with differences b/w major versions.

r-session-complete:

rsc docker image:

The Package Manager container has R installed but is not configured to use it. I propose we should have the default configuration support R / building packages from git / etc.

Hi,

I am using docker destop to download rstudio connect and I can see below path successfully.
root@565ad3ca8b23:/opt/rstudio-connect/bin# ls
configdigest connect feature-usage license-manager migrate rscadmin usermanager

But when I am going to create my own dockerfile with your provided dockerfile. Where I am unable to see the path like above one. I can see only /opt/rstudio-connect-runtime only

I think we should add the rstudio-server stop command to the SIGKILL trap implanted in the startup.sh file. https://github.com/rstudio/rstudio-docker-products/blob/master/server-pro/startup.sh#L7

I noticed in the readme that there is a misspelling: the "--priviliged" flag should be "privileged". FYI @danielfrg