cis548 / docker-env Goto Github PK

View Code? Open in Web Editor NEW

5.0 8.0 7.0 45 KB

A dockerized development environment for MCIT-595

Dockerfile 27.03% Shell 72.97%

environment

docker-env's Introduction

Docker-env

Usage

Preparations

Install Docker Desktop (version 3.0+). For Windows, we recommend setting up WSL2.
Copy docker-compose.yml to your course projects folder.

First-time Setup

For AMD64 Architecture

If your CPU is in AMD64 Architecture, in other words, if your CPU is not Apple M1, then do the following.

Run docker pull cis548/docker-env to will pull the docker image from Docker Hub repository.
Run docker-compose run --rm mcit to launch a Docker Container.

--rm will clean up the container when it is finished.
The home directory /home/cit595, also known as ~, is synchornized with the folder, where you put the docker-compose.yml, in your Host System.

You should now be inside your docker image, with current folder at /home/cit595. You can use pwd to check.

You will log into the container as user cit595 with password mcit. The password is used just in case you occationally need sudo previlege.

For ARM64 Architecture

If you are using a machine with Apple M1 CPU, you need to do a few extra steps to compile the docker image locally.

Copy Dockerfile to your course projects folder along with docker-compose.yml
Run docker-compose build mcit to compile the docker image locally.
Follow For AMD64 Architecture section, starting from the second step. In other words, run docker-compose run --rm mcit to launch a Docker Container.

All Future Log-ins

Just open your course projects folder, and run docker-compose run --rm mcit.

Multiple Sessions

To establish multiple sesssion to the same container, call docker exec -it [container_id] bash after booting the first window with docker-compose run mcit.

About `docker-compose`

The following is the offical definition of docker-compose:

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

In CIT 595, we only use docker-compose to start one service which is your virtual machine. But docker-compose make the command consice and convient because we can predefine the configuration in docker-compose.yml.

If we use the bare-mental docker to start the container with same configuration, you have to run

docker run -it --rm -v `pwd`:/home/cit595 cis548/docker-env

If we use docker-compose, it runs container with -it by default. The volume mounting is configed in docker-compose.yml under mcit service. So we only need to call docker-compose run --rm mcit.

This is why we choose to use docker-compose, but the overhead is you have to prepare a docker-compose.yml file.

Set up Vim

Vim is installed in the Docker container. We prepared a Vim configure file for students who would like to explore. The configuration file helps to install plug-ins and theme to make Vim more user-friendly and powerful. To use the configuration file, first use docker-compose run --rm mcit to get into the container. Then do the following two steps.

# Step 1: Download the configuration file from CIS548
wget -O .vimrc https://raw.githubusercontent.com/CIS548/gists/master/example_vimrc.txt

# Step2: Update Vim based on the configuration files
vim -es -u vimrc -i NONE -c "PlugInstall" -c "qa"

Now you are ready for Vim!

(For TAs) Push Image to DockerHub

Docker Image in DockerHub (Link)

We use docker buildx to generate images for AMD64 and ARM64 CPUs. The steps are as follows.

Create a new builder. The default docker builder does not support multi-platform compiling, so we need to create a new one. Here we name it mybuilder.

docker buildx create --name mybuilder

Set mybuilder as the new default builder and double-check the change is in effect.

docker buildx use mybuilder
docker buildx inspect --bootstrap

Build and push the images to cis548/docker-env. To push images to the Docker Hub reposiotry, make sure you have the right permissons to the repo.

docker buildx build . --platform=linux/amd64,linux/arm64/v8 -t cis548/docker-env:latest --push
# change :latest to another tag if necessary

To learn more, the following sections in this YouTube video gives a step-by-step demo.

05:40 - Buildx setup
06:46 - Dockerhub repo setup
06:59 - Building the images!

An example of running docker image with a specific platform

docker run -it --rm --platform linux/arm64 cis548/docker-env

Resources

docker-env's People

Contributors

Stargazers

Watchers

Forkers

bexxmodd conmeconte narwhalish asidach alesparza kuang-da

docker-env's Issues

Mount the Docker home directory to $PWD

Update "Project 0: Description"

The PDF instructions in Project 0: Description will need to be replaced with a updated instructions.

The current version (v5) is saved in Google Drive here: https://docs.google.com/document/d/1nb8pSyI3lWWMtjzwzGUlQuqskM8aH_x73qBxwaVqFcc/edit#

I see that @kuang-da already started working on a version of Project 0 description for Docker in August 2021:
https://docs.google.com/document/d/1Se_ncw_QvQluBo9k4gnIQqzWdACz8gHTIpG1zLlPaGk/edit#

I have compared it with v5 and have so far made some minor edits (list to be expanded in follow up comments to this issue):

Update the introduction to mention Docker and the available resources for completing this project.
Update Step 1 to point to resources in Week 1, rather than Resources section of Coursera (these need to be moved, see the related issue)

There are more edits that need to be made - removing a few references to Vagrant, and making sure that instructions are consistent with the GitHub README , which I believe is the current, most tested set of instructions. @kuang-da - can you confirm this?

Add automated test on Dockerfile

Make Vim plugins come out-of-box

Starting from a503785, I removed the PlugInstall from installation script. It is because now we mount the host folder to /home/cit595 and it will reset the Vim configuration. To bypass that, I added a section about Set up Vim in README.md but it would be nice if we could have all the plugins ready out-of-box.

Remove Vagrant-specific Coursera pages

Pages to be removed (or hidden from students):

In Week 1:
Vagrant FAQs

TA Demo: Course VM Environment

In Resources:
Project 0 VagrantFile

Do we need to maintain a collection of Vagrant resources to be made available to any students for whom there is some insurmountable roadblock to using Docker, and form whom Vagrant may work? (Is that even likely?)

Cannot create ssh key within Docker container

Reproduce the problem

OS: Window 10

When creating the ssh key, it says Could not create directory '/home/cit595/.ssh': Permission denied.

cit595@ba27ad01f3a3:~$ ssh-keygen -t ed25519 -C "[email protected]"
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/cit595/.ssh/id_ed25519): 
Could not create directory '/home/cit595/.ssh': Permission denied
Enter passphrase (empty for no passphrase):
Enter same passphrase again: 
Saving key "/home/cit595/.ssh/id_ed25519" failed: No such file or directory

Reason

We did not change the ownership of cit595 to new user cit595 so /home/cit595 still belongs to root

cit595@ba27ad01f3a3:/home$ ls -al
total 8
drwxr-xr-x 1 root root 4096 May 25 20:15 .
drwxr-xr-x 1 root root 4096 Sep  2 03:21 ..
drwxr-xr-x 2 root root   40 Sep  2 03:21 cit595

Potential Solution

Update the dockerfile by adding

RUN chown cit595:cit595 /home/cit595

What is the best practice for Docker in Windows?

Add `zip` to docker environment

Please add zip to the docker environment. So we can use the deploy script

Update Windows Instruction with PowerShell

Based on #7, we should make it more clear in the instruction to use the PowerShell launch Docker container, not WSL2.

The Windows instruction is here.

update to Ubuntu 22

VSCode will stop supporting Ubuntu 18 in 2025, so we should just upgrade now

How to handle previously recorded Recitations for Project 0

Doing a search for "Vagrant" in Coursera brought up prior semesters' recorded Project 0 Recitations.

Would it make sense to remove previous Recitations? - To answer this later question, I would like to watch some of these videos and determine if they are still valuable in a post-Vagrant 595.

If we decide to keep them, it might make sense to add a sentence to Project 0 Description to the effect that recorded Recitations for Project 0 from Fall 2021 are for a different Virtual Machine setup.

Instuction based on Bookdown

Create an open-source, step-by-step tutorial to set up Docker on Windows and Mac OS by Bookdown.

https://bookdown.org/yihui/bookdown/rstudio-connect.html

Add `net-tools` to docker environment

We suggest to students to use netstat which requires installing net-tools with sudo permissions. Some students are scared to modify the docker environment, so we should just install this for them from the get-go.

The demo is showing how to use "netstat" to see ports in use, etc. When I tried to run this, I received a message that "netstat: command not found."

Based on an internet search, I tried installing netstat with

$ sudo apt update
$ sudo apt install net-tools

The terminal then asked for a password for cit595. I didn't recall setting any ubuntu password, and was nervous about trying to reset docker.

Any suggestions?

Add python3

Unless I'm reading docker-setup.sh wrong, we don't give the students python? Can anyone confirm?

Repository: Change master branch to main

please change the master branch to main, and also be sure to update the branch protection rule

Redo Command Line and Github TA demos using Docker

We currently have multiple resources for Git and Github, some mentioning/using Vagrant.

In Week 1:

Code Version Control - this is Boon's lecture and is high level and no mention of Vagrant. Definitely keep this.

TA Demo: Project 0 Git - this one explicitly uses Vagrant.

TA Demo: Command Line - this also explicitly uses Vagrant.

Best to replace the above with a newly demo or demos before Spring 2022.

In Resources:

Introduction to GIT

A useful guide to GIT. Would it be worth keeping/ moving this to Week 1, and group it together with other Version Control related content?

Pre-install `ping` in container

Ideally students will never have to install anything in the containers. https://edstem.org/us/courses/58581/discussion/4979122 required (or at least used for debugging) ping so we should add it

Move "Docker Environment Tutorials" from Resources to Week 1

In Coursera, we should move the Docker Environment Tutorials from Resources to immediately following Project 0 Description in Week 1.

Multi architecture build for amd64 and arm64 (Apple M1 Chip)

Recently, I've encountered issues when building the docker image from the Dockerfile and docker-setup.sh file.

Following the instructions in the README for ARM64 Architecture, when running "docker-compose build mcit", the build does not complete because of hash sums not matching for some installations in docker-setup.sh.

If I comment out all parts of the docker-setup.sh file after the line:

wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -

then the build is able to complete.

I believe the issue may have to do with my current internet, as I was able to build without problem in other locations.

I am not sure if others will encounter this issue, but I believe it would be best to do a multi-architecture build of the image for amd64 and arm64 so that Apple M1 users can also Pull the docker image and will not need to build it.

I am unable to build the multi-arch image because of the above issue.

@kuang-da , would you mind trying to build the multi-arch image and push it the course Dockerhub? I was able to follow tutorials online, such as this one, using my own Docker hub account and some test Dockerfiles.

colourize the prompt

There were some requests last semester to make the docker prompt have colour (similar to how vagrant looked). What is the possibility of modifying the bash profile to include colors for username and current directory?

Here is a basic resource: change bash colours

Update "Week 1 Lesson 3 Video: What is a Virtual Machine?"

Coursera Link

From 00:00 to 05:10, we talked about what is Virtual machine and why it is important for CIT 595. This part looks good.
From 05:10 to 06:17, we give a conceptual example of setting up Vagrant in the context of project 0. We could just trim this part out.
From 06:17 to the end, we answer the question why not just use "Codio". This part looks good.

Build image for ARM64

It would be nice if we could prepare the Docker image for Apple M1 students. But currently, Docker Hub is not able to automated build images for ARM64. Link

Docker Buildx can be used for cross-platform building.