remind101 / stacker_blueprints Goto Github PK
View Code? Open in Web Editor NEWDEPRECATED - moved to:
Home Page: https://github.com/cloudtools/stacker_blueprints
License: BSD 2-Clause "Simplified" License
DEPRECATED - moved to:
Home Page: https://github.com/cloudtools/stacker_blueprints
License: BSD 2-Clause "Simplified" License
Right now DockerImage
is set to master
which is doesn't do anything.
We should set this to a stable version of empire or make it a required parameter.
just a preference thing, i don't really like _
in repo names and always go with -
Reopen from Stacker issue:
I got this error using the latest released version and latest master version. Not exactly sure where to look to fix this issue. The same configuration and environment files worked two months ago.
[2017-09-17T12:09:25] INFO stacker.commands.stacker:32(configure): Using Default AWS Provider
[2017-09-17T12:09:25] DEBUG stacker.plan:288(outline): Plan "Create/Update stacks":
[2017-09-17T12:09:25] DEBUG stacker.plan:296(outline): - step: 1: target: "vdt-dev-vpc", action: "_launch_stack"
[2017-09-17T12:09:25] DEBUG stacker.plan:296(outline): - step: 2: target: "vdt-dev-empireDB", action: "_launch_stack"
[2017-09-17T12:09:25] DEBUG stacker.plan:296(outline): - step: 3: target: "vdt-dev-empireController", action: "_launch_stack"
[2017-09-17T12:09:25] DEBUG stacker.plan:296(outline): - step: 4: target: "vdt-dev-bastion", action: "_launch_stack"
[2017-09-17T12:09:25] DEBUG stacker.plan:296(outline): - step: 5: target: "vdt-dev-empireMinion", action: "_launch_stack"
[2017-09-17T12:09:25] DEBUG stacker.plan:296(outline): - step: 6: target: "vdt-dev-empireDaemon", action: "_launch_stack"
[2017-09-17T12:09:25] DEBUG stacker.actions.build:303(run): Launching stacks: vdt-dev-vpc, vdt-dev-empireDB, vdt-dev-empireController, vdt-dev-bastion, vdt-dev-empireMinion, vdt-dev-empireDaemon
[2017-09-17T12:09:25] INFO stacker.plan:358(_check_point): Plan Status:
[2017-09-17T12:09:25] INFO stacker.plan:380(_check_point): vdt-dev-vpc: pending
[2017-09-17T12:09:25] INFO stacker.plan:380(_check_point): vdt-dev-empireDB: pending
[2017-09-17T12:09:25] INFO stacker.plan:380(_check_point): vdt-dev-empireController: pending
[2017-09-17T12:09:25] INFO stacker.plan:380(_check_point): vdt-dev-bastion: pending
[2017-09-17T12:09:25] INFO stacker.plan:380(_check_point): vdt-dev-empireMinion: pending
[2017-09-17T12:09:25] INFO stacker.plan:380(_check_point): vdt-dev-empireDaemon: pending
[2017-09-17T12:09:25] DEBUG stacker.util:58(retry_with_backoff): Calling <bound method CloudFormation.describe_stacks of <botocore.client.CloudFormation object at 0x103c193d0>>, attempt 1.
[2017-09-17T12:09:25] DEBUG stacker.actions.build:231(_launch_stack): Resolving stack vdt-dev-vpc
[2017-09-17T12:09:25] DEBUG stacker.actions.build:234(_launch_stack): Launching stack vdt-dev-vpc now.
[2017-09-17T12:09:25] DEBUG stacker.blueprints.base:448(import_mappings): Adding mapping AmiMap.
[2017-09-17T12:09:25] DEBUG stacker.blueprints.base:369(setup_parameters): No parameters defined.
Traceback (most recent call last):
File "/Users//dev/projects/viditure/vt_env/bin/stacker", line 4, in
import('pkg_resources').run_script('stacker==1.0.4', 'stacker')
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/pkg_resources/init.py", line 748, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/pkg_resources/init.py", line 1524, in run_script
exec(script_code, namespace, namespace)
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/stacker-1.0.4-py2.7.egg/EGG-INFO/scripts/stacker", line 9, in
File "build/bdist.macosx-10.12-intel/egg/stacker/commands/stacker/build.py", line 52, in run
File "build/bdist.macosx-10.12-intel/egg/stacker/actions/base.py", line 127, in execute
File "build/bdist.macosx-10.12-intel/egg/stacker/actions/build.py", line 304, in run
File "build/bdist.macosx-10.12-intel/egg/stacker/plan.py", line 267, in execute
File "build/bdist.macosx-10.12-intel/egg/stacker/plan.py", line 227, in _single_run
File "build/bdist.macosx-10.12-intel/egg/stacker/plan.py", line 74, in run
File "build/bdist.macosx-10.12-intel/egg/stacker/actions/build.py", line 235, in _launch_stack
File "build/bdist.macosx-10.12-intel/egg/stacker/actions/base.py", line 101, in s3_stack_push
File "build/bdist.macosx-10.12-intel/egg/stacker/actions/base.py", line 25, in stack_template_key_name
File "build/bdist.macosx-10.12-intel/egg/stacker/blueprints/base.py", line 491, in version
File "build/bdist.macosx-10.12-intel/egg/stacker/blueprints/base.py", line 461, in render_template
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/troposphere-1.9.5-py2.7.egg/troposphere/init.py", line 609, in to_json
return json.dumps(self.to_dict(), indent=indent,
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/troposphere-1.9.5-py2.7.egg/troposphere/init.py", line 606, in to_dict
return encode_to_dict(t)
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/troposphere-1.9.5-py2.7.egg/troposphere/init.py", line 63, in encode_to_dict
props[name] = encode_to_dict(prop)
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/troposphere-1.9.5-py2.7.egg/troposphere/init.py", line 63, in encode_to_dict
props[name] = encode_to_dict(prop)
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/troposphere-1.9.5-py2.7.egg/troposphere/init.py", line 54, in encode_to_dict
return encode_to_dict(obj.to_dict())
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/troposphere-1.9.5-py2.7.egg/troposphere/init.py", line 226, in to_dict
self.validate()
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/troposphere-1.9.5-py2.7.egg/troposphere/ec2.py", line 344, in validate
exactly_one(self.class.name, self.properties, gateway_conds)
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/troposphere-1.9.5-py2.7.egg/troposphere/validators.py", line 201, in exactly_one
specified_count = mutually_exclusive(class_name, properties, conditionals)
File "/Users//dev/projects/viditure/vt_env/lib/python2.7/site-packages/troposphere-1.9.5-py2.7.egg/troposphere/validators.py", line 196, in mutually_exclusive
class_name, ', '.join(conditionals)))
ValueError: Route: only one of the following can be specified: EgressOnlyInternetGatewayId, GatewayId, InstanceId, NatGatewayId, NetworkInterfaceId, VpcPeeringConnectionId
Since cloudformation and troposphere support nat gateways now, should they be used instead of nat instances? I'm working on trying to modify vpc.py to support that, but I keep doing something wrong, so I don't have a pull request ready yet. Is this a direction in which you are moving, or are you planning on sticking with nat instances?
Also, thanks for doing some great work with this and empire!
It's just confusing at this point. Need to think about existing examples (empire
) that use the old rds.
I was going to create an s3 stack, but I don't see how to create the role that setting ReadWriteRoles would add a policy to.
I can work on a PR for this. Does it make more sense to have a blueprint that creates a bunch of ec2 and/or lambda roles, or just a single one.
Hello guys,
First of all, thank you for the excellent work.
Today I tried to change an old stack of mine and got the following error:
File "...github.com_remind101_stacker_blueprints/stacker_blueprints/sns.py", line 136, in create_topic
sqs_subs = [sub for sub in topic_subs if sub["Protocol"] == "sqs"]
TypeError: 'NoneType' object is not iterable
Here is my stack:
- name: sns
class_path: stacker_blueprints.sns.Topics
enabled: true
variables:
Topics:
LifeCycleHookTopic:
DisplayName: "cassandra-lifecycle-hook"
If I change the stack to define an empty Subscription list, it works
- name: sns
class_path: stacker_blueprints.sns.Topics
enabled: true
variables:
Topics:
LifeCycleHookTopic:
DisplayName: "cassandra-lifecycle-hook"
Subscription: []
I don't think Ref("PublicSubnets") is sufficiently configurable. I'm going to work on a patch for this too.
As an RDS user
I'd like to get io1 or gp2 disks depending on if iops is declared
So that I don't use slow spinning disks
Current:
When StorageType is not set, and iops == 0, magnetic disks are provisioned
Expected:
When StorageType is not set, and iops == 0, gp2 SSD disks are provisioned
I'm guessing this is RDS default behavior for Ref("AWS::NoValue") I'd recommend using something like StorageType = variables['StorageType'] or 'io1' if get_piops() > 0 else 'gp2'
When attempting to upgrade a Postgres RDS instance from 9.5.x to 9.6.1, the stack updates the DBParameterGroup and OptionGroup, but then proceeds to fail on AWS::RDS::DBInstance with the following error:
The Parameter Group stage-classchirp-com-shortymasterdb-parametergroup-z95umv2qmwhi with DBParameterGroupFamily postgres9.5 cannot be used for this instance. Please use a Parameter Group with DBParameterGroupFamily postgres9.6
This is despite having the family set to postgres9.6.
MariaDB is now supported as a DB engine, but there is no corresponding blueprint.
Should have the same rules as stacker
Looks like the elasticache package only supports redis
As a network admin
I want to expose the bastion security group as an output
So I can pass it into other stacks which add ingress rules allowing traffic from the bastions
The template conditionally does not create the ParameterGroup and OptionGroup based on whether the aforementioned variables are empty, but refers to them anyway and causes errors:
botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the CreateStack operation: Template format error: Unresolved resource dependencies [OptionGroup, ParameterGroup] in the Resources block of the template
Result:
API: ec2:disassociateAddress You do not have permission to access the specified resource.
--
Template ends up in a 'UPDATE_ROLLBACK_COMPLETE' state.
Dupe the issue.
Deploy VPC template with UseNatGateway: True, then attempt to do a stacker build with UseNatGateway: False
As a lazy admin
I want a predicable DNS name for my bastion box(s)
So that I have a reliable endpoint to connect to.
I'm not sure how to accomplish this, especially for users that choose to run more than one bastion. You'd likely have to add an ELB forwarding port 22, and deal with host keys.
The default cloudformation dynamodb stack is really rigid. Updating AttributeDefinitions
requires a replacement: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-table.html#cfn-dynamodb-table-attributedef and you can't have a global secondary index that references a non-attribute definition.
You CAN update a table and add attribute definitions/GSIs with the boto3 api: http://boto3.readthedocs.io/en/latest/reference/services/dynamodb.html#DynamoDB.Client.update_table.
A custom resource that just piped args to the create_table
and update_table
would make it much easier to evolve dynamodb schema.
I think we should have some standards around parameters and outputs.
ie. sometimes a security group is called SecurityGroup
other times its EmpireMinionSG
.
Maybe there is another way to do this, but this should allow other stacks to give the minion permission to access resources.
The specific thing I'm trying to handle: I have an SNS topic I want to allow a service running within Empire to publish to without having to pass AWS credentials via. environmental variables to the task.
I tried setting timeout: <some val>
in two different regions with ClusterParameters
and both threw an Internal Error
within CF. Have you ever set anything with this?
we should allow passing a value for: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-group.html#cfn-as-group-desiredcapacity
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.