Git Product home page Git Product logo

kocho's Introduction

Kocho

Build Status Go Report Card IRC Channel

Kocho provides a set of mechanisms to bootstrap AWS nodes that must follow a specific configuration with CoreOS. It sets up fleet meta-data, and patched versions of fleet, etcd, and docker when using Yochu.

Getting Kocho

Download the latest release: https://github.com/giantswarm/kocho/releases/latest

Clone the git repository: https://github.com/giantswarm/kocho.git

Download the latest docker image from here: https://hub.docker.com/r/giantswarm/kocho/

Running Kocho

./kocho help

Further Steps

Check more detailed documentation: docs

Check code documentation: godoc

Contact

Contributing & Reporting Bugs

See CONTRIBUTING for details on submitting patches, the contribution workflow as well as reporting bugs.

License

Kocho is under the Apache 2.0 license. See the LICENSE file for details.

Origin of the Name

kocho (こちょう[蝴蝶] pronounced "ko-cho") is Japanese for butterfly.

kocho's People

Contributors

hectorj2f avatar josephsalisbury avatar marians avatar puja108 avatar taylorbot avatar zeisss avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

mnjstwins nkhine

kocho's Issues

enh: Do not rely on discovery.etcd.io, use AWS metadata to bootstrap

Inspired from https://github.com/MonsantoCo/etcd-aws-cluster/blob/develop/etcd-aws-cluster

We use the following cloud-init to provision our etcd clusters and we find it much better to rely on AWS metadata rather than a public service with no guarantees on.

#cloud-config

coreos:
  etcd2:
    advertise-client-urls: http://$private_ipv4:2379
    initial-advertise-peer-urls: http://$private_ipv4:2380
    listen-client-urls: http://0.0.0.0:2379
    listen-peer-urls: http://$private_ipv4:2380
  fleet:
    public-ip: $private_ipv4
    etcd_servers: "${peers}"
  update:
    group: ${channel}
    reboot-strategy: ${reboot-strategy}
  units:
    - name: docker.service
      runtime: true
      command: start

    # etcd and fleet are initially stopped. We'll start them when the peers have been calculated
    - name: etcd2.service
      command: stop
    - name: fleet.service
      command: stop

    - name: etcd-peers.service
      command: start
      content: |
        [Unit]
        Description=Write a file with the etcd peers that we should bootstrap to
        After=docker.service
        [Service]
        Restart=on-failure
        RestartSec=10
        EnvironmentFile=/etc/environment
        ExecStartPre=/usr/bin/docker pull peopleperhour/etcd-aws-cluster:latest
        ExecStartPre=/usr/bin/docker run --rm=true -v /etc/sysconfig/:/etc/sysconfig/ peopleperhour/etcd-aws-cluster:latest
        ExecStart=/usr/bin/systemctl start etcd2

    # Query AWS metadata service when this EC2 instance starts to get info about instance's self. e.g. IP address.
    - name: fleet-metadata.service
      runtime: true
      command: start
      content: |-
        [Unit]
        Description=Inject EC2 Metadata into Fleet
        After=docker.service
        [Service]
        Type=oneshot
        EnvironmentFile=/etc/environment
        ExecStartPre=/usr/bin/docker pull peopleperhour/ec2-metadata
        ExecStart=/usr/bin/bash -c '\
          mkdir -p /run/systemd/system/fleet.service.d; \
          conf_file=/run/systemd/system/fleet.service.d/666-ec2-metadata.conf; \
          zone=$(docker run --rm peopleperhour/ec2-metadata -z   | sed "s/: /=/g"); \
          id=$(docker run --rm peopleperhour/ec2-metadata -i     | sed "s/: /=/g"); \
          ip=$(docker run --rm peopleperhour/ec2-metadata -v     | sed "s/: /=/g" | sed "s/ /_/g"); \
          type=$(docker run --rm peopleperhour/ec2-metadata -t   | sed "s/: /=/g"); \
          echo "[Service]" > $conf_file; \
          echo "Environment=FLEET_METADATA=role=${role},$ip,$id,$type,$zone"  >> $conf_file;'

    - name: fleet-reloaded.service
      command: start
      content: |
        [Unit]
        Description=Ensure fleet starts with updated metadata
        After=docker.service
        After=fleet-metadata.service
        [Service]
        Restart=on-failure
        RestartSec=10
        EnvironmentFile=/etc/environment
        ExecStartPre=/usr/bin/systemctl daemon-reload
        ExecStart=/usr/bin/systemctl start fleet
write_files:
    # Pre-configure the core user with authentication to our private Docker Hub repo (as the pphrobot hub user).
    - path: /home/core/.dockercfg
      owner: core:core
      permissions: 0644
      content: |
        {"https://index.docker.io/v1/":{"auth":"snipped","email":"[email protected]"}}
    - path: /etc/systemd/system/etcd2.service.d/30-etcd_peers.conf
      permissions: 0644
      content: |
        [Service]
        EnvironmentFile=/etc/sysconfig/etcd-peers

users:
  - name: pmoust
    coreos-ssh-import-github: pmoust
    groups:
      - sudo
      - docker
      - wheel
  - name: core
    coreos-ssh-import-github-users:
      - pmoust

manage_etc_hosts: localhost

Make Cloudflare optional

Me and @puja108 were chatting, and thought it would be good to make the Cloudflare features optional - that is, that no Cloudflare DNS would be set up at all.

Show Yochu version on `kocho instances`

It would be cool to have primer be able to show the provisioner version used on a machine. This could be displayed like that:

$ kocho list
Id          Image         Type       PublicDns             PrivateDns           Provisioner
i-71ba8194  ami-58e14e2f  t2.medium  ec2-54-194-36-194...  ip-172-31-27-35...   0.6.0
i-73ba8196  ami-58e14e2f  t2.medium  ec2-54-171-252-25...  ip-172-31-27-36...   0.5.0
i-321923d7  ami-58e14e2f  t2.medium  ec2-54-77-72-201...   ip-172-31-20-208...  0.6.0

This could be done by using yochu version.

`template-init` does not work in a bare directory

I was testing out initial user experience (so, not having the source in the directory), and it looks like the templates are not being copied correctly.

$ ls
total 36968
-rwxr-xr-x  1 joseph  staff    18M 11 Feb 16:22 kocho*
-rw-r--r--  1 joseph  staff   291B 11 Feb 16:22 kocho.yml
joseph on Josephs-MacBook-Pro at 16:24:14 in ~/Desktop/kocho-test
$ ./kocho template-init
couldn't read template: primary-cloudconfig.tmpl open default-templates/primary-cloudconfig.tmpl: no such file or directory
joseph on Josephs-MacBook-Pro at 16:24:18 in ~/Desktop/kocho-test
$ ls
total 36968
-rwxr-xr-x  1 joseph  staff    18M 11 Feb 16:22 kocho*
-rw-r--r--  1 joseph  staff   291B 11 Feb 16:22 kocho.yml
drwxr-xr-x  2 joseph  staff    68B 11 Feb 16:24 templates/
joseph on Josephs-MacBook-Pro at 16:24:20 in ~/Desktop/kocho-test
$ ls ./templates/
joseph on Josephs-MacBook-Pro at 16:24:23 in ~/Desktop/kocho-test
$

--debug does nothing useful

I tried to use --debug to see why a dns command does nothing. Unfortunately, no output at all, neither with or without --debug.
More information should be printed when --debug is enabled.

Add command to remove machine from etcd cluster

Currently removing a machine from the etcd cluster is not possible without killing the AWS instance. This is problematic when either a machine does not exist anymore or needs to be removed if it is still running.

We need a way to perform the removal without trying to kill the instance itself.

Something like remove-from-etcd would be good.

kocho complains about my AWS credentials when using ENV variables

I defined my AWS env variables following the README,

export AWS_ACCESS_KEY=cute
export AWS_SECRET_ACCESS_KEY=cute

getting the following error:

couldn't list swarms SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
'POST
/

content-type:application/x-www-form-urlencoded; charset=utf-8
host:cloudformation.eu-west-1.amazonaws.com
x-amz-date:20160401T111626Z

content-type;host;x-amz-date
1c0d327d16f37b15838ca07a3964664f2dcff8d7051d9f1d87552e7df79be01f'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20160401T111626Z
20160401/eu-west-1/cloudformation/aws4_request
7a536408aa84a3bdc89df94941076f9620c3d54a9067ed98458001038bcdae1c'

However, when I pass the credentials using the cli arguments, it WORKS.

Add ssh command

Being able to ssh straight into an instance from Kocho would be useful.
e.g:

kocho ssh cluster-1

Fetch VPC ID if empty

Currently one has to provide the VPC id when using Kocho. It should be possible to fetch the known VPCs in the configured region, and if there is only one (or only has a special kocho tag), we can use that one.

Same could be done with keypairname / certificates

kill-instance does not remove instance from etcd quorum

I created a cluster and over time remove two instances from the original cluster. as expected the AS added new instances, which seemed to have joined the etcd cluster. At least reading worked on all machines. After the second machine was removed I tried writing into etcd on the machine, but it failed with a 501: peers unavailable errors

Field Value
Template standalone
Cluster-Size 3
Instance Type t2.micro

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.