scottslowe / learning-tools Goto Github PK
View Code? Open in Web Editor NEWA collection of tools and files for learning new technologies
License: MIT License
A collection of tools and files for learning new technologies
License: MIT License
Add aws.instance_type=instances['instance_type'] to vagrant-aws/Vagrantfile because if not it will always use default values... (m3.medium) which is not the cheapest node to test ;)
Thnks,
Javier R.
Add a learning environment for testing VTEP (VXLAN Tunnel End Point) connectivity between Cumulus Linux (via Cumulus VX) and Linux instances running Open vSwitch (OVS).
Running VirtualBox as my hypervisor (on CentOS 7.5), I've found that when the K8s VMs boot up and the services are installed, they bind to the first NIC (NAT connection to the outside world) rather than the second NIC (the 192.168.100.0/24 private network). This applies to all services except the API server, and broke networking for me (all nodes use the same IP address for the first NIC, and there is no connectivity on that network between nodes anyway). It's fixable by adding the --node-ip
argument to the kubelet process at startup, telling it to bind to the second NIC - e.g.
sed -i "s/KUBELET_EXTRA_ARGS=/KUBELET_EXTRA_ARGS=--node-ip=<correct-ip-address-here>/" /etc/default/kubelet
Also, even with the config changed to use the internal network, I couldn't get Calico v3 working by following step 9 - neither the Calico nor the CoreDNS pods would fire up. However, after installing v3.3 following instructions at https://docs.projectcalico.org/v3.3/getting-started/kubernetes/, all pods run fine.
I'm presuming this is the same with other providers, but haven't tested with them. If this is an acceptable solution, I'm happy to create a pull request to update the instruction steps as per the contribution guidelines, as that would also be a useful learning experience for me!
The box you're attempting to add doesn't support the provider
you requested. Please find an alternate box or use an alternate
provider. Double-check your requested provider to verify you didn't
simply misspell it.
If you're adding a box from HashiCorp's Atlas, make sure the box is
released.
Name: slowe/ubuntu-trusty-x64
Address: https://atlas.hashicorp.com/slowe/ubuntu-trusty-x64
Requested provider: [:virtualbox]
Some Ubuntu "Wily" 15.10 Vagrant boxes may use the ensX
naming convention for network interfaces, while others will use the (older) ethX
naming convention. For example, "ubuntu/wily64" uses the ethX
naming style, but "slowe/ubuntu-15.10-server-amd64" uses ensX
. I need to address these differences in interface naming conventions in the Docker/IPVLAN learning environment.
Pulumi offers a GitHub provider that can be used for managing issue labels across repositories. I'd like to add an example of such a Pulumi program, written in Go, for others to use and/or learn from.
In order to make it easier for others to contribute to this repository, the repository needs some contribution guidelines.
As it is currently written, the Ansible role for provisioning a Pulumi environment won't upgrade to newer versions of Pulumi. The role needs to be fixed to enable this functionality.
The OVN+Docker learning environment experiences intermittent errors with the Ansible playbooks.
Currently the KVM learning environment (in the kvm
folder) only supports the VMware provider for Vagrant (VirtualBox does not support nested virtualization). The Libvirt provider for Vagrant also supports nested virtualization, so I should update the KVM environment to also support the Libvirt provider.
Hi,
going through your instructions, I receive this error.
A Docker command executed by Vagrant didn't complete successfully!
The command run along with the output from the command is shown
below.
Command: "docker" "ps" "-a" "-q" "--no-trunc"
Stderr: Cannot connect to the Docker daemon. Is the docker daemon running on this host?
Stdout:
If I then run vagrant up again, I receive this...
A Docker command executed by Vagrant didn't complete successfully!
The command run along with the output from the command is shown
below.
Command: "docker" "run" "--name" "nginx-container" "-d" "-p" "80:80" "-p" "443:443" "nginx"
Stderr: Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx
6a5a5368e0c2: Pulling fs layer
4aceccff346f: Pulling fs layer
c8967f302193: Pulling fs layer
c8967f302193: Verifying Checksum
c8967f302193: Download complete
4aceccff346f: Verifying Checksum
4aceccff346f: Download complete
6a5a5368e0c2: Verifying Checksum
6a5a5368e0c2: Download complete
6a5a5368e0c2: Pull complete
4aceccff346f: Pull complete
c8967f302193: Pull complete
Digest: sha256:1ebfe348d131e9657872de9881fe736612b2e8e1630e0508c354acb0350a4566
Status: Downloaded newer image for nginx:latest
docker: Error response from daemon: driver failed programming external connectivity on endpoint nginx-container (8ade69b7e43443ebe5fdb52368b6cf3f9017dfb46c992cd6ed257bea095ec705): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use.
Stdout: 78d3ec7da7a4031c212ea7a0fbbfed4214dadffacc0f7f6ca0842b7861bd7c42
Any insight would be helpful. I have been using Vagrant for over a year with a custom vagrant file for multiple vhosts, but am just now playing around with Docker. I am on a windows machine...I know I know haha.
Thanks!
Of the learning environments that leverage Ansible, very few are using roles. This can make porting Ansible playbooks across learning environments more difficult. To fix this, Ansible-using learning environments need to be refactored (where possible) to use an Ansible role.
I'd like to add an example of using Vagrant triggers to create/destroy Docker contexts automatically when a Vagrant machine is created or destroyed.
I donβt see boostrap.json file. Will you be able to commit that file?
On mac osx.
Command: "docker" "ps" "-a" "-q" "--no-trunc"
Stderr: Get http:///var/run/docker.sock/v1.19/containers/json?all=1: dial unix /var/run/docker.sock: permission denied. Are you trying to connect to a TLS-enabled daemon without TLS?
Any ideas?
The Terraform+AWS environments need to be cleaned up a bit:
variables.tf
everywhere)*.tfvars
files to isolate variables and make environments more portableThe simple-ec2
environment has already been cleaned up, but the bastion-aws
and new-vpc
environments need to be reviewed. simple-module
should adhere to this standards already.
Now that Consul 1.0 is available, all the Consul-related learning tools/environments should be updated to use the 1.0 release.
Per a reader, there are some potential issues with the article on using Ansible through an SSH bastion host (http://blog.scottlowe.org/2015/12/24/running-ansible-through-ssh-bastion-host/):
ProxyCommand
and ForwardAgent
in an SSH configuration file is not required/recommended. The reader suggests that the commands "compete" with each other.I need to review these potential errors and correct them.
The Pulumi program found in debian/debian-pulumi-aws
relies on a configuration value for the SSH keypair to be used. This required configuration value can be removed by updating the program with two new resources (an SSH key and an AWS keypair).
Add a learning environment for using multi-NIC networking with Vagrant and Ubuntu "Wily Werewolf". Vagrant currently has a problem with Ubuntu Wily, per this blog post:
http://blog.scottlowe.org/2016/03/21/vagrant-ubuntu-wily-networking-problem/
This learning environment will provide a workaround, either via Ansible, Jinja2 templates, ERB templates, or shell scripts.
The VMware Photon-Ansible learning environment (found in photon-ansible
) does not have a README file.
Is it possible to communicate between containers on separate nodes? Can mpi applications be executed on such virtual cluster?
Hi
I am trying to modify the Vagrant file of dnf-2016 for VirtualBox provider but I stumble across the
metadata of https://app.vagrantup.com/slowe/boxes/ubuntu-trusty-x64 which applies only for the vmware case. How can I overcome this ?
The README.md
file for the Docker Swarm-etcd-Photon OS environment is out of date and incorrect, and needs to be updated to reflect the current state of the learning environment.
Currently, most of the Terraform+AWS environments (found in terraform/aws
) have no README.md
to explain how they can be used.
Options for the NAT instance include https://fck-nat.dev/stable/ (GH repo: https://github.com/AndrewGuenther/fck-nat)
I need to add CAPI+kustomize
content to accompany a blog post I'm writing on combining CAPI with kustomize
. This content will use the fictional scenario in the blog post.
Hi Scott
I have a little question about your blog
http://blog.scottlowe.org/2015/03/06/running-own-docker-swarm-cluster/
I understand the swarm setup, and it's supposed to listen on 2375 port instead of unix socket
But talking about the registrator, I find the --volumn option mapping the unix socket inside container, so when it comes with swarm nodes, how to map the :2375 to /tmp/docker.sock
Seems not working if you just use --volumn /var/run/docker.sock:/tmp/docker.sock
The Pulumi program for running Debian on AWS (found in the debian/debian-pulumi-aws
folder) has support for both AMD64- as well as ARM64-based instances. The Azure version (found in debian/debian-pulumi-azure
) lacks this functionality.
As a companion to flatcar/flatcar-pulumi-aws
, this Pulumi program would deploy Flatcar Linux to Azure, using a dedicated resource group, VNet, subnets, etc.
Hey Scott,
This seems like a really cool idea! π What do you think about linking to other peoples "tutorials" for distributed systems tools?
e.g.
The Pulumi program found at flatcar/flatcar-pulumi-aws
currently has a required configuration value that specifies the name of the AWS keypair. By modifying the program to create an SSH key and an AWS keypair, that required configuration value can be eliminated.
The ubuntu/ubuntu-pulumi-aws
program already includes these resources and can be used as a model for the changes needed here.
I need to add a learning environment that demonstrates using cloud-init
to customize the behavior of the Docker daemon on CentOS Atomic Host.
Add a learning environment that will enable users to work with/learn about runc (the future Docker container runtime) and containerd (the daemon that will manage runc).
Hey @scottslowe , loving all the content here π―
Have been working on couple of Terraform projects and digging around to find + learn from cool resources such as yours. Was going through the Terraform code for aws at terraform/aws
and figured we could possibly contribute more resources. To name a few:
1. s3
2. elasticache
3. dynamodb
4. beanstalk
5. waf
Let me know if such contribution will add value to the existing set of learning tools, if so I'll initiate a PR :)
Regards,
Udhav
Would you mind adding packer to your folders?
I forked a repo (2016-dnf-materials) to create a testbed scenario to play with docker networking modes: https://github.com/chadell/2016-demo-net-linux-networking
I tried to reproduce the same macvlan scenario you had there (and here) using Docker v.1.12.1 instead of the plugin I get that the container can't ping a remote host through the eth1 which is connected to the internal bridge created by Vagrant for all the VM hosts.
Debugging it using tcpdump on the container, docker-01 (host) and remote-01 (a remote host), I realize that I get ARP request and replies either in the remote-01 and the container, but when comes to ICMP I see the remote-01 receiving ICMP request from the container and replying but not ICMP requests coming into eth1 in the docker-01 host (before sending to the container), and neither works the ICMP request from the remote-01.
Config used in docker-01 host:
sudo ip link set eth1 promisc on
sudo docker network create -d macvlan --subnet=192.168.100.0/24 -o parent=eth1 pub_net
sudo docker run -td alpine /bin/sh
sudo docker network connect pub_net 2bab90874f2f
Tcpdump in remote-01 (192.168.100.110), the container has 192.168.100.2:
09:30:45.279510 ARP, Request who-has 192.168.100.2 tell 192.168.100.110, length 28
09:30:45.279812 ARP, Reply 192.168.100.2 is-at 02:42:c0:a8:64:02 (oui Unknown), length 46
09:30:45.279819 IP 192.168.100.110 > 192.168.100.2: ICMP echo request, id 1849, seq 1, length 64
Since the default Ubuntu 16.04 AMIs don't come with a Python interpreter, you can't use Ansible straight "out of the box." However, it's possible to use the raw
module to bootstrap Ansible. This learning environment would show how that is done (and would serve as a building block for other environments).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.