docker-archive / deploykit Goto Github PK
View Code? Open in Web Editor NEWA toolkit for creating and managing declarative, self-healing infrastructure.
License: Apache License 2.0
A toolkit for creating and managing declarative, self-healing infrastructure.
License: Apache License 2.0
Local machine state presents a challenge for users in terms of durability and consistency. It shouldn't be too difficult to implement a backing store that persists all writes to S3 for users that wish to enable it.
This command hangs, which is confusing:
$ build/infrakit group watch
I suspect this is an artifact of accepting input via stdin. Perhaps we should eliminate support for configurations via stdin, or do so only by a non-default use of the command.
Addresses this TODO in rollingupdate.go
:
// TODO(wfarner): Provide a mechanism to gracefully drain instances.
Draining would be a nice addition to make updates more graceful. Command line tooling would also be nice to perform manual service drains.
A few points on managing machine secrets
Implementation Notes
The Vanilla plugin maps its UserData
field into instance.Spec.Init
, and Labels
to instance.spec.Tags
. These fields should use the same names for consistency.
Currently in Docker Machine API credentials are managed by the drivers themselves and they work differently from driver to driver:
So different provisioners expect different input and we could consider a design where credentials are stored in an yaml file that is referenced by the user, along with the provisioner (provider/driver) name, to authenticate. For example, for AWS, suppose we need
myaws
and it can look like:region: us-east
access_id: xxxx
secret_key: yyyy
and the command line could be
docker node create -driver=aws -auth=myaws -template=medium-appserver -count=10
It is then the responsibility of the AWS provisioner to parse the file and perform the necessary auth against the AWS API.
Same would apply to Azure, except the Azure provisioner would expect a yaml of different format and can load that and perform API auth. The provisioner obviously would have to provide helpful error if the specified auth file isn't what it expects.
Right now logging is pretty ad-hoc. Other than using logrus here and there, there is no pattern or systematic support for provisioner developers. docker-machine
exposes logger for driver developers to use and libmachete should as well.
Currently the binaries built by infrakit have unhelpful names, eg cli
, file
(conflicts with common unix command), vagrant
(its not vagrant
) etc. The simplest solution is to just prefix them all infrakit-...
not sure about other options.
For the plugins maybe it doesn't matter, but then we should remove the make install
from the Makefile.
In plugin discovery, we should not instruct users to apply broad permissions to an absolute directory:
chmod 777 /run/infrakit/plugins
go version go1.6.2 darwin/amd64
make -k infrakit
+ clean
mkdir -p ./infrakit
rm -rf ./infrakit/*
+ vendor-sync
+ build
+ infrakit
# github.com/docker/infrakit/vendor/golang.org/x/text/unicode/norm
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x1f41ead02f9e pc=0xf0eb]
runtime stack:
runtime.throw(0x4971e0, 0x2a)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/panic.go:547 +0x90
runtime.sigpanic()
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/sigpanic_unix.go:12 +0x5a
runtime.unlock(0x984540)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/lock_sema.go:107 +0x14b
runtime.(*mheap).alloc_m(0x984540, 0x1, 0xf, 0xc820000180)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/mheap.go:492 +0x314
runtime.(*mheap).alloc.func1()
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/mheap.go:502 +0x41
runtime.systemstack(0xc820453e58)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/asm_amd64.s:307 +0xab
runtime.(*mheap).alloc(0x984540, 0x1, 0x1000000000f, 0xed8f)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/mheap.go:503 +0x63
runtime.(*mcentral).grow(0x985ea0, 0x0)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/mcentral.go:209 +0x93
runtime.(*mcentral).cacheSpan(0x985ea0, 0xc820442000)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/mcentral.go:89 +0x47d
runtime.(*mcache).refill(0xaf6e10, 0xc80000000f, 0xdf3070)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/mcache.go:119 +0xcc
runtime.mallocgc.func2()
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/malloc.go:642 +0x2b
runtime.systemstack(0xc820020000)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/asm_amd64.s:291 +0x79
runtime.mstart()
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/proc.go:1051
goroutine 1 [running]:
runtime.systemstack_switch()
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/asm_amd64.s:245 fp=0xc822bb50f8 sp=0xc822bb50f0
runtime.mallocgc(0xe0, 0x425e40, 0x1, 0xc8230d9ea0)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/malloc.go:643 +0x869 fp=0xc822bb51d0 sp=0xc822bb50f8
runtime.newobject(0x425e40, 0xc8230d9ea0)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/malloc.go:781 +0x42 fp=0xc822bb51f8 sp=0xc822bb51d0
cmd/compile/internal/gc.regopt.func1(0x0, 0x0)
/usr/local/Cellar/go/1.6.2/libexec/src/cmd/compile/internal/gc/reg.go:1064 +0x2f fp=0xc822bb5210 sp=0xc822bb51f8
cmd/compile/internal/gc.Flowstart(0xc8230e59e0, 0x4c4778, 0x20)
/usr/local/Cellar/go/1.6.2/libexec/src/cmd/compile/internal/gc/popt.go:288 +0x90c fp=0xc822bb5320 sp=0xc822bb5210
cmd/compile/internal/gc.regopt(0xc8230e59e0)
/usr/local/Cellar/go/1.6.2/libexec/src/cmd/compile/internal/gc/reg.go:1064 +0x3d0 fp=0xc822bb57d8 sp=0xc822bb5320
cmd/compile/internal/gc.compile(0xc8202eb710)
/usr/local/Cellar/go/1.6.2/libexec/src/cmd/compile/internal/gc/pgen.go:521 +0xe6d fp=0xc822bb5a48 sp=0xc822bb57d8
cmd/compile/internal/gc.funccompile(0xc8202eb710)
/usr/local/Cellar/go/1.6.2/libexec/src/cmd/compile/internal/gc/dcl.go:1450 +0x1c0 fp=0xc822bb5ac0 sp=0xc822bb5a48
cmd/compile/internal/gc.Main()
/usr/local/Cellar/go/1.6.2/libexec/src/cmd/compile/internal/gc/lex.go:472 +0x2116 fp=0xc822bb5de0 sp=0xc822bb5ac0
cmd/compile/internal/amd64.Main()
/usr/local/Cellar/go/1.6.2/libexec/src/cmd/compile/internal/amd64/galign.go:127 +0x58d fp=0xc822bb5e48 sp=0xc822bb5de0
main.main()
/usr/local/Cellar/go/1.6.2/libexec/src/cmd/compile/main.go:32 +0x395 fp=0xc822bb5f20 sp=0xc822bb5e48
runtime.main()
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/proc.go:188 +0x2b0 fp=0xc822bb5f70 sp=0xc822bb5f20
runtime.goexit()
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc822bb5f78 sp=0xc822bb5f70
make: *** [infrakit] Error 1
This is the top priority for supporting Cloud. Also seeking convergence with Docker4X (Editions). The requirements are
swarm init
and swarm join
for managers. On other platforms where there are no Docker-provided machine images, we must install, configure and launch the latest Docker Engines for this purpose.It is very odd to have infrakit/foo
everywhere. No need to have all the different packages. Just use cobra subcommands.
This should live in a separate repository (like github.com/docker/infrakit.aws), but keeping the issue here for tracking purposes.
We've found that gomock introduces more problems than it solves. I'd recommend removing it and replacing with simple interface embedding. The following provides the same functionality as gomock:
type myMock struct {
MockedInterface
}
// Override will intercept override.
func (m *myMock) Override(...) { }
Wish I could have warned you earlier.
I have been following the demo: https://github.com/docker/infrakit/blob/master/example/instance/terraform/cattle_demo.md
I saw the 3 instance-*.tf.json file created in my was-two-tier folder but I'm missing the docs on how do I configure my AWS to deploy these onto my AWS. I believe just the json plan file is generated but i don't see how to configure the deployment to AWS. Would be great to get this clarified.
As you may know Danny Trejo starred as the character Machete in a few Robert Rodriguez films as well as in the 2001 classic, "Spy Kids".
For daemons, this should be the first log line. At a bare minimum, we'll need the git SHA.
Some users have come across issues when building infrakit. We can improve the build UX by doing the build inside docker containers.
Some things to consider...
This gets into the topic of whether we should provide Docker containers to simplify experimentation and running of the binaries. It's outside the scope of this Issue unless people feel otherwise. Please comment here.
TCP-based plugin communication doesn't offer any built-in access controls (which we get with unix sockets, for example) and should be removed on that basis alone. TCP support is primarily offered as a lowest-common denominator for Windows, but we should look to named pipes for a closer analog.
Currently in the Editions / 1.12 TP2 the behavior of a node when provisioned is that it automatically joins the cluster via
docker swarm join <master_ip>:4500
which is executed via the user data for the instance when the Moby instance comes up (as part of an autoscaling group).
For other environments where autoscaling is absent (pretty much everyone else outside of AWS, Azure, GCE), machete needs to run the cluster join as a task in the provisioning workflow.
Assumptions:
Following the README on the project page, I did a go get for "kardianos/govendor" and "golang/lint/golint".
After doing that when I am running "make -k infrakit", I am getting the below error:
Makefile:88: *** Please install govendor: go get github.com/kardianos/govendor. Stop.
Also, doing "make ci" throws the error:
Makefile:42: *** Please install golint: go get -u github.com/golang/lint/golint. Stop.
Go version: go-1.7.1
OS: macOS 10.12
Kindly look into this problem or suggest if I am missing something.
Thanks,
Gaurav
SwarmKit just switched to vndr
for vendoring (moby/swarmkit#1546). Let's consider using that.
Don't switch right away: it would be good to get feedback.
Currently the CLI is very developer-facing and a free-for-all in that it reflects all the http endpoint for all the plugin types. It's handy for developers to develop and test the building blocks but confusing to end users / ops.
The Issue here tracks the design and development of a separate frontend (and likely a separate CLI) that will provide a unified UX that will be much better suited for end-users of this toolkit and not just for developers.
cli
tool.This "frontend" will likely to provide means to simplify the start up for plugins -- by examining the local filesystem holding the configs and determine the plugins that need to be run. Ability to do this will be environment specific (e.g. running the plugins as unix daemons or as docker containers or whatever), so a base implementation and mechanisms for naming/ typing of plugins will have to be worked out as part of this.
We need to implement a consistent way to install the Engine on a host after it's been created. This is the last stage of the processing pipeline that looks like:
ready
and can be added to a Swarm cluster (TBD later)Note
engine_ubuntu14_04
or engine_alpine
Spurring off from #214 which shed support for TCP in the plugin server and client libraries, we have had several requests for supporting communicating with plugins via TCP. The use case revolves around avoiding a separate body of code and service (the InfraKit plugin) to adapt a system to support InfraKit. For example, if AWS supported an InfraKit API, it would be a boon if users could interact directly with it without the need for a locally-running plugin.
Create the interface that will define the functions a provider may provide to provision machines. For the moment, this will only support creation of machines.
The component must receive a template schema (perhaps interface{}
), and a YAML document to marshal into the schema. It should additional receive a second YAML document to overlay onto the first. In this case, the first document represents the template and the second defines overrides.
Some thoughts on implementation:
I haven't had reseach enough for this ..and need some explanation ..you people put up something in the doc and and failed to tell us the dependecy.Probably it skips your tierd mind.
bhaskar@Fedora_07:14:39_Fri Oct 07:~/git-linux/infrakit>make ci
+ fmt
+ vet
can't load package: package _/home/bhaskar/git-linux/infrakit/cmd/cli: cannot find package "_/home/bhaskar/git-linux/infrakit/cmd/cli" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/cmd/cli (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/cmd/cli (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/cmd/group: cannot find package "_/home/bhaskar/git-linux/infrakit/cmd/group" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/cmd/group (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/cmd/group (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/discovery: cannot find package "_/home/bhaskar/git-linux/infrakit/discovery" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/discovery (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/discovery (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/example/flavor/swarm: cannot find package "_/home/bhaskar/git-linux/infrakit/example/flavor/swarm" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/example/flavor/swarm (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/example/flavor/swarm (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/example/flavor/vanilla: cannot find package "_/home/bhaskar/git-linux/infrakit/example/flavor/vanilla" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/example/flavor/vanilla (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/example/flavor/vanilla (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/example/flavor/zookeeper: cannot find package "_/home/bhaskar/git-linux/infrakit/example/flavor/zookeeper" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/example/flavor/zookeeper (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/example/flavor/zookeeper (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/example/instance/file: cannot find package "_/home/bhaskar/git-linux/infrakit/example/instance/file" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/example/instance/file (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/example/instance/file (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/example/instance/terraform: cannot find package "_/home/bhaskar/git-linux/infrakit/example/instance/terraform" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/example/instance/terraform (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/example/instance/terraform (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/example/instance/vagrant: cannot find package "_/home/bhaskar/git-linux/infrakit/example/instance/vagrant" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/example/instance/vagrant (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/example/instance/vagrant (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/plugin: cannot find package "_/home/bhaskar/git-linux/infrakit/plugin" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/plugin (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/plugin (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/plugin/flavor/swarm: cannot find package "_/home/bhaskar/git-linux/infrakit/plugin/flavor/swarm" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/plugin/flavor/swarm (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/plugin/flavor/swarm (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/plugin/flavor/vanilla: cannot find package "_/home/bhaskar/git-linux/infrakit/plugin/flavor/vanilla" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/plugin/flavor/vanilla (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/plugin/flavor/vanilla (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/plugin/flavor/zookeeper: cannot find package "_/home/bhaskar/git-linux/infrakit/plugin/flavor/zookeeper" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/plugin/flavor/zookeeper (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/plugin/flavor/zookeeper (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/plugin/group: cannot find package "_/home/bhaskar/git-linux/infrakit/plugin/group" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/plugin/group (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/plugin/group (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/plugin/group/types: cannot find package "_/home/bhaskar/git-linux/infrakit/plugin/group/types" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/plugin/group/types (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/plugin/group/types (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/plugin/group/util: cannot find package "_/home/bhaskar/git-linux/infrakit/plugin/group/util" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/plugin/group/util (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/plugin/group/util (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/plugin/instance/vagrant: cannot find package "_/home/bhaskar/git-linux/infrakit/plugin/instance/vagrant" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/plugin/instance/vagrant (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/plugin/instance/vagrant (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/plugin/util: cannot find package "_/home/bhaskar/git-linux/infrakit/plugin/util" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/plugin/util (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/plugin/util (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/spi/flavor: cannot find package "_/home/bhaskar/git-linux/infrakit/spi/flavor" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/spi/flavor (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/spi/flavor (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/spi/group: cannot find package "_/home/bhaskar/git-linux/infrakit/spi/group" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/spi/group (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/spi/group (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/spi/http/flavor: cannot find package "_/home/bhaskar/git-linux/infrakit/spi/http/flavor" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/spi/http/flavor (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/spi/http/flavor (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/spi/http/group: cannot find package "_/home/bhaskar/git-linux/infrakit/spi/http/group" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/spi/http/group (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/spi/http/group (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/spi/http/instance: cannot find package "_/home/bhaskar/git-linux/infrakit/spi/http/instance" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/spi/http/instance (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/spi/http/instance (from $GOPATH)
can't load package: package _/home/bhaskar/git-linux/infrakit/spi/instance: cannot find package "_/home/bhaskar/git-linux/infrakit/spi/instance" in any of:
/usr/lib/golang/src/_/home/bhaskar/git-linux/infrakit/spi/instance (from $GOROOT)
/home/bhaskar/go/src/_/home/bhaskar/git-linux/infrakit/spi/instance (from $GOPATH)
Makefile:29: recipe for target 'vet' failed
make: *** [vet] Error 1
Currently the provisioner interface contains functions that instead of using the error
interface type, return pointers to the concrete error type:
https://github.com/docker/libmachete/blob/master/spi/instance/spi.go#L8
This will actually cause problems when the spi is wrapped by others who follow golang convention and return error
in their functions. Here's a concrete example of this being a problem:
https://play.golang.org/p/6KM5W2mRT3
We should always return error
and not be returning pointers to concrete error type.
Background: https://speakerdeck.com/campoy/understanding-nil and https://github.com/gophercon/2016-talks/blob/master/DaveCheney-DontCheckErrorsHandleThemGracefully/GopherCon%202016.pdf
Instead of declaring a public error type with error code and message, we should make this type package private and provide helper functions to return an appropriate HTTP response code if the caller cares about HTTP. Or have helper functions that allow the caller to reason about the nature / context of the error. Inside the helper functions we then do type assertions.
Our scaffolding for communication already resembles the APIs to net/rpc
, which can be adapted to JSON-RPC with net/rpc/jsonrpc
. The switch would allow us to offload some custom code and move to a standard wire format as well.
Also weigh net/rpc/jsonrpc
against other libraries like github.com/gorilla/rpc/json
.
$ make ci
+ fmt
+ vet
+ lint
+ vendor-sync
# cd .; git clone https://go.googlesource.com/net /Users/aluther/code/go/.cache/govendor/golang.org/x/net
Cloning into '/Users/aluther/code/go/.cache/govendor/golang.org/x/net'...
error: RPC failed; curl 56 SSLRead() return error -36
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
Error: Remotes failed for:
Failed for "golang.org/x/net/context" (failed to clone repo): exit status 128
make: *** [vendor-sync] Error 2
Any ideas what could be wrong here?
This test appears to deadlock occasionally, causing CI to be flaky.
We need a subsystem to manage credentials (e.g. AWS access key/secret, Azure OAuth tokens). This subsystem will allow the user to
The storage of credentials will be separate from other system states such as inventory, SSH keys and TLS certs. This separation will make it possible for the use cases of backing up or sharing states without also backing up / sharing API secrets.
For example, the following operations should be possible:
$ machete credentials add aws production --access_key=...
$ machete credentials add aws production-payments-team --access_key=...
$ machete credentials add azure development --oauth_token_file=...
$ machete credentials list
production
production-payments-team
development
$machete create web1 --cred=production -instanceType=.....
$machete create batchHost1 --cred=production-payments -instanceType=.....
Hi all,
In running through the Terraform demo, I'm having trouble when I set InfraKit to start watching the group:
In window01 I launch the terraform plugin:
$ infrakit/terraform --log 5 --dir $(pwd)/example/instance/terraform/aws-two-tier/
INFO[0000] Starting plugin
INFO[0000] Listening on: unix:///run/infrakit/plugins/instance-terraform.sock
DEBU[0000] terraform instance plugin. dir= /Users/jesse/Code/learn/dkr_infrakit/src/github.com/docker/infrakit/example/instance/terraform/aws-two-tier/
INFO[0000] listener protocol= unix addr= /run/infrakit/plugins/instance-terraform.sock err= <nil>
Once I try to set InfraKit to watch that group:
$ infrakit/cli group watch example/instance/terraform/aws-two-tier/group.json
I get a bunch of errors in the original window01 that makes it seems as though InfraKit can't find the Terraform executable.
INFO[0322] Acquired lock. Applying
INFO[0322] Can't acquire lock. Wait.
INFO[0322] Error: unknown command "apply" for "terraform"
terraform=apply
INFO[0322] Run 'terraform --help' for usage.
terraform=apply
INFO[0322] time="2016-10-06T14:01:31-04:00" level=error msg="unknown command \"apply\" for \"terraform\""
terraform=apply
INFO[0322] Acquired lock. Applying
INFO[0322] Can't acquire lock. Wait.
INFO[0322] Error: unknown command "apply" for "terraform"
terraform=apply
INFO[0322] Run 'terraform --help' for usage.
terraform=apply
INFO[0322] time="2016-10-06T14:01:31-04:00" level=error msg="unknown command \"apply\" for \"terraform\""
terraform=apply
INFO[0322] Acquired lock. Applying
INFO[0322] Can't acquire lock. Wait.
INFO[0322] Error: unknown command "apply" for "terraform"
terraform=apply
INFO[0322] Run 'terraform --help' for usage.
Running terraform
outside of the directory gives me this the expected program output:
$ terraform
usage: terraform [--version] [--help] <command> [args]
The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.
...
While running it in the Go program directory starts the unix sock:
WOPR:infrakit jesse$ terraform
INFO[0000] Starting plugin
INFO[0000] Listening on: unix:///run/infrakit/plugins/instance-terraform.sock
Unsure of how to tell if InfraKit is calling Terraform appropriately, or how to set it. This is on Mac OSX Yosemite.
CI doesn't necessarily need to run end to end tests (yet), but we should at least have it compile and lint to make sure those sources are not completely broken
In InfraKit, plugin names don't relate to Docker Hub or registry in any way, so these comments need to be revised. We may even consider removing the PluginName
fields altogether as they're not currently used in any meaningful way.
$ ag 'Docker Hub'
cmd/group/main.go
24: // PluginName is the name of the plugin in the Docker Hub / registry
example/flavor/swarm/main.go
17: // PluginName is the name of the plugin in the Docker Hub / registry
example/flavor/vanilla/main.go
16: // PluginName is the name of the plugin in the Docker Hub / registry
example/flavor/zookeeper/main.go
16: // PluginName is the name of the plugin in the Docker Hub / registry
example/instance/file/main.go
15: // PluginName is the name of the plugin in the Docker Hub / registry
example/instance/terraform/main.go
16: // PluginName is the name of the plugin in the Docker Hub / registry
example/instance/vagrant/main.go
16: // PluginName is the name of the plugin in the Docker Hub / registry
Addresses this TODO in rollingupdate.go
:
// TODO(wfarner): Make the 'batch size' configurable.
We should accept a parallelism
flag for updates to allow multiple instances to be drained simultaneously.
This file covers both ELB and Azure loadbalancers but both comments, variable naming and logging output is AWS/ELB specific:
Addresses this TODO in rollingupdate.go
:
// TODO(wfarner): Poll ProvisionHelper.Healthy here.
A README in the style of the tutorial would be very nice.
Extra points for tying it into the Terraform example, showing how to deploy a swarm onto some terraformed instances.
TODO from types.go
:
// TODO(wfarner): This does not consider flavor plugin and properties. At present, since details like group
// size and logical IDs are extracted from opaque properties, there's no way to distinguish between a group size
// change and a change requiring a rolling update.
The storage interface and implementation can be primitive for now, possibly as little as an association from a machine ID and basic attributes.
@chungers suggests implementing the store on libkv as it is a basic API and there is a reasonable future where it is used as the longer-term distributed store API.
noticed some PRs were merged without a sign-off, and there's no contributing.md or maintainers file present in this repository. Some boilerplating can be found here https://github.com/docker/opensource/tree/master/project-template
We have 2 libs providing assert-like functionality. Let's converge to 1 until we have distinct requirements for both.
I am currently at the infrakit
BoF in Berlin, I am playing around with it for the first time.
I want to create a flavor plugin for a specific stack. I cannot find anywhere documentation for when you are writing a plugin.
I am checking the source code from other flavors trying to miss nothing for my implementation.
It could be great to document a plugin implementation documentation for (future) contributors (with the requirements for a group
or instance
or flavor
plugin).
We need a mechanism that can validate data structures in a simple, unified way without requiring too much work on the provisioner writers:
This will be important to account for some YAML parsing bugs that are known (e.g. skipping unrecognized fields without errors: [https://github.com/go-yaml/yaml/issues/136])
Some implementation ideas:
nil
or zero values.Example: We could introduce a tag check
that specifies requiredness of a field:
type MyExampleRequest struct {
Name *string `check:"required,no_zero"`
LastName *string `check:optional"`
Age *uint32 `check:"required,no_zero"`
}
Here the developer decrees that Name
is a required field and cannot be an empty string, while LastName
can be optional and Age
must exist and cannot be 0
.
With this specification, we should be able to do validation check with a helper function like this:
missing := []string{}
err := Validate(myStruct, func(fieldName string, fieldValue interface{}) {
// This is a callback when the field is either missing or have unacceptable zero value.
missing = append(missing, fieldName)
})
if err != nil {
panic(fmt.Errorf("Bad input %v. Missing fields: %v", err, missing))
}
There may be other validation rules. Please comment in this thread.
The default Group plugin currently only has in-memory state for the Groups being watched. Introduce a new plugin type that the Group plugin is configured with, which delegates state management.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.