Git Product home page Git Product logo

serverless-fargate-tasks's Introduction

Serverless Fargate Tasks

This Serverless plugin will setup a Fargate cluster and setup services and tasks. With this plugin it's fairly easy to setup a (long running) task which would hook up to kinesis streams or SQS endpoints.

The (minimal) config is as follows:

custom:
  fargate:
    vpc:
      subnets:
        - subnet-1234
        - subnet-5678

    tasks:
      my-task:
        image: 123456789369.dkr.ecr.eu-west-1.amazonaws.com/my-image

Of course, you can customize to your hearts desire, here are all the available options:

custom:
  fargate:
    # you can put global environment variables here, these will be added
    # to all tasks. Optional of course.
    environment:
      foo: bar

    # you can set the execution role that will be used, this will default to the default
    # role for your account
    role: arn:aws:iam::123456789369:role/ecsTaskExecutionRole

    vpc:
      public-ip: DISABLED  # optional, defaults to disabled
      subnets:
        - subnet-1234
        - subnet-5678
      security-groups:  # optional, defaults to vpc default
        - sg-123456678

    tasks:
      my-task:
        name: ${self:service}-${self:provider.stage}-my-task # default name is be the object key (here 'my-task')
        image: 123456789369.dkr.ecr.eu-west-1.amazonaws.com/my-image
        environment:  # optional
          platypus: true
          # local variables will always override global ones
          foo: wut
          # you can also use cloudformation references with eg serverless-pseudo-parameters
          myArn: #{MyResource.Arn}
        cpu: 512  # optional, defaults to 25% -> 256, see cloudformation docs for valid values
        memory: 1GB  # optional, defaults to 0.5GB

Advanced usage

You can override the generated CF resource properties per task with the override properties:

custom:
  fargate:
    tasks:
      my-task:
        image: 123456789369.dkr.ecr.eu-west-1.amazonaws.com/my-image

        # these are all optional
        override:
          task:
            Foo: BAR
          container:
            Foo: Bar
          service:
            Foo: BAR
          vpc:
            Foo: BAR
          role: ARN

serverless-fargate-tasks's People

Contributors

svdgraaf avatar christophebougere avatar wizardishungry avatar tfoster avatar

Stargazers

Sean Kirk avatar Emaad Ali avatar Jinsoo Jeong avatar Italo Moraes avatar Yotam Ishak avatar Chris Phan avatar  avatar Igor Olshevsky avatar Luis Carvajal avatar  avatar Vince Elizaga avatar Zaidin Amiot avatar Cyprien Pannier avatar Alwin Schoemaker avatar Ajinkya Nahar avatar Fayaz avatar Andrejs Agejevs avatar Eoin Shanaghy avatar Diego Esteban avatar Elliott Arnold  avatar Georges Mainville avatar Samer Kanjo avatar Stivenson avatar Github Notification avatar Roman Scher avatar Tobi avatar Pedro Borges avatar wes avatar Paulo Rodriguez avatar Sandro Keil avatar Haydn avatar  avatar  avatar Hermes avatar K Mills avatar  avatar  avatar quizhats avatar  avatar Michal Przytulski avatar  avatar Benjamin Genz avatar Takahiro Horike avatar

Watchers

James Cloos avatar

serverless-fargate-tasks's Issues

Unable to deploy fargate task

Hi,

I'm trying to deploy a fargate task for a long running process but I keep getting this error

TypeError: Cannot read property 'LogGroupName' of undefined

Full trace :
TypeError: Cannot read property 'LogGroupName' of undefined
at /Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/node_modules/@serverless/lib/logsCollection.js:80:69
at Generator.next ()
at asyncGeneratorStep (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/node_modules/@serverless/enterprise-plugin/lib/logsCollection.js:3:103)
at _next (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/node_modules/@serverless/enterprise-plugin/lib/logsCollection.js:5:194)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:97:5)
From previous event:
at PluginManager.invoke (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/classes/PluginManager.js:483:22)
at PluginManager.spawn (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/classes/PluginManager.js:503:17)
at AwsPackage. (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/plugins/aws/package/index.js:70:41)
From previous event:
at Object.package:finalize [as hook] (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/plugins/aws/package/index.js:69:30)
From previous event:
at PluginManager.invoke (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/classes/PluginManager.js:483:22)
at PluginManager.spawn (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/classes/PluginManager.js:503:17)
at /Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:122:50
From previous event:
at Object.before:deploy:deploy [as hook] (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:102:22)
at /Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/classes/PluginManager.js:483:55
From previous event:
at PluginManager.invoke (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/classes/PluginManager.js:483:22)
at /Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/classes/PluginManager.js:518:24
From previous event:
at PluginManager.run (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/classes/PluginManager.js:518:8)
at /Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/Serverless.js:136:33
From previous event:
at Serverless.run (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/lib/Serverless.js:123:74)
at /Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/scripts/serverless.js:54:26
at processImmediate (internal/timers.js:456:21)
at process.topLevelDomainCallback (domain.js:137:15)
From previous event:
at Object. (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/scripts/serverless.js:54:4)
at Module._compile (internal/modules/cjs/loader.js:1151:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
at Module.load (internal/modules/cjs/loader.js:1000:32)
at Function.Module._load (internal/modules/cjs/loader.js:899:14)
at Module.require (internal/modules/cjs/loader.js:1040:19)
at require (internal/modules/cjs/helpers.js:72:18)
at Object. (/Users/david/.nvm/versions/node/v13.9.0/lib/node_modules/serverless/bin/serverless.js:41:1)
at Module._compile (internal/modules/cjs/loader.js:1151:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
at Module.load (internal/modules/cjs/loader.js:1000:32)
at Function.Module._load (internal/modules/cjs/loader.js:899:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47

This is my serverless.yml

plugins:

  • '@jeremigendron/sls-ts-plugin'
  • serverless-dotenv-plugin
  • serverless-plugin-git-variables
  • serverless-offline
  • serverless-certificate-creator
  • serverless-domain-manager
  • serverless-fargate-tasks

custom:
fargate:
vpc:
security-groups:
- ${ssm:/XXX/security-group/id}
subnets:
- ${ssm:/XXX/subnets/private_1a}
- ${ssm:/XXX/subnets/private_1b}
tasks:
sesio-backend-seeder-user:
name: ${self:service}-seeder-user-${self:provider.stage}
image: XXX.dkr.ecr.eu-west-1.amazonaws.com/XXX/seeder-user:latest
cpu: 512
memory: 1GB
environment:
MONGODB_URL: ${ssm:/XXX/database/endpoint}
MONGODB_PORT: 27017
MONGODB_DATABASE: XXX
MONGODB_USERNAME: ${ssm:/XXX/database/usernametrue}
MONGODB_PASSWORD: ${ssm:/XXX/database/password
true}
REDIS_HOST: ${ssm:/XXX/cache/endpoint}
REDIS_PORT: 6379

Your Environment Information ---------------------------
Operating System: darwin
Node Version: 13.9.0
Framework Version: 1.78.0
Plugin Version: 3.7.0
SDK Version: 2.3.1
Components Version: 2.33.4

TypeError: Cannot read property 'Properties' of undefined

I just tried using this plugin from the examples in the README:

  Your Environment Information -----------------------------
     OS:                     darwin
     Node Version:           9.5.0
     Serverless Version:     1.38.0

serverless-fargate-tasks version: 0.1.0

serverless.yml:

service: serverless-test

plugins:
  - serverless-fargate-tasks

custom:
  fargate:
    network:
      public-ip: DISABLED
      subnets:
        - subnet-00
        - subnet-01
    tasks:
      my-task:
        image: alpine:3.8

provider:
  name: aws
  stage: dev
  region: us-east-1

Error:

Fargate Tasks Plugin
Processing my-task

  Type Error ---------------------------------------------

  Cannot read property 'Properties' of undefined

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

TypeError: Cannot read property 'Properties' of undefined
    at Object.keys.forEach.identifier (/Users/bgibbs/secops/git_python/node_modules/serverless-fargate-tasks/lib/index.js:54:54)
    at Array.forEach (<anonymous>)
    at ServerlessFargateTasks.compileTasks (/Users/bgibbs/secops/git_python/node_modules/serverless-fargate-tasks/lib/index.js:37:32)
    at BbPromise.reduce (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:407:55)

I'm fairly new to working with the serverless framework, so I'm hoping that I'm just missing a config option somewhere?

Fargate Spot

Hi,

From what I've seen, we can't launch a task using a spot strategy.
Couldn't this be a configuration on the .yml file to define the Launch Type?

Thank you

Cannot read property 'Properties' of undefined

  TypeError: Cannot read property 'Properties' of undefined
      at /Users/jstillwell/Code/dragonfax/streaming/node_modules/serverless-fargate-tasks/lib/index.js:54:54
      at Array.forEach (<anonymous>)
      at ServerlessFargateTasks.compileTasks (/Users/jstillwell/Code/dragonfax/streaming/node_modules/serverless-fargate-tasks/lib/index.js:37:32)
      at /usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:518:55
      at tryCatcher (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
      at Object.gotValue (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/reduce.js:168:18)
      at Object.gotAccum (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/reduce.js:155:25)
      at Object.tryCatcher (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromise0 (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:729:18)
      at _drainQueueStep (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:93:12)
      at _drainQueue (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:458:21)

Ability to customize task name

Thanks for this plugin @svdgraaf, manually setting up a fargate cluster is quite cumbersome!
Right now the task name is the key of the object, which prevents using variables in it.
In my case, I'd like to have the stage in the task name, so I can deploy my service in different stages on the same AWS account.
Having the ability to override the name in a property would be great, something like this:

custom:
  fargate:
    network:
      subnets:
        - subnet-1234
        - subnet-5678
    tasks:
      my-task:
        name: my-task-${self:provider.stage}
        image: 123456789369.dkr.ecr.eu-west-1.amazonaws.com/my-image

ECS Service stuck on creation

Hi!

Trying to deploy a CloudFormation template via Serverless gets stuck with 'AWS::ECS::Service' stuck on 'CREATE_IN_PROGRESS' with status 'Resource creation Initiated'.

  • Can anyone please tell me if there's a configuration problem with my CloudFormation template?
  • How would i troubleshoot this?

Thank you!

CloudFormation Template

"FargateTasksCluster": {
			"Type": "AWS::ECS::Cluster"
		},
		"FargateTasksLogGroup": {
			"Type": "AWS::Logs::LogGroup"
		},
		"MytaskTask": {
			"Type": "AWS::ECS::TaskDefinition",
			"Properties": {
				"ContainerDefinitions": [{
					"Name": "my-task",
					"Image": "mountainpass/infrastructure:helloworld",
					"Environment": [],
					"LogConfiguration": {
						"LogDriver": "awslogs",
						"Options": {
							"awslogs-region": {
								"Fn::Sub": "${AWS::Region}"
							},
							"awslogs-group": {
								"Fn::Sub": "${FargateTasksLogGroup}"
							},
							"awslogs-stream-prefix": "fargate"
						}
					}
				}],
				"Family": "my-task",
				"NetworkMode": "awsvpc",
				"ExecutionRoleArn": {
					"Fn::Sub": "arn:aws:iam::${AWS::AccountId}:role/ecsTaskExecutionRole"
				},
				"TaskRoleArn": {
					"Fn::Sub": "${IamRoleLambdaExecution}"
				},
				"RequiresCompatibilities": ["FARGATE"],
				"Memory": "0.5GB",
				"Cpu": 256
			}
		},
		"MytaskService": {
			"Type": "AWS::ECS::Service",
			"Properties": {
				"Cluster": {
					"Fn::Sub": "${FargateTasksCluster}"
				},
				"LaunchType": "FARGATE",
				"ServiceName": "my-task",
				"DesiredCount": 1,
				"TaskDefinition": {
					"Fn::Sub": "${MytaskTask}"
				},
				"NetworkConfiguration": {
					"AwsvpcConfiguration": {
						"AssignPublicIp": "DISABLED",
						"SecurityGroups": [],
						"Subnets": ["subnet-2cc37974", "subnet-3c959d5b", "subnet-49d7d200"]
					}
				}
			}
		}

Serverless.yml


frameworkVersion: ">=1.1.0"

plugins:
  - serverless-fargate-tasks
  # - serverless-vpc-plugin
custom:
  bucket: jb-mp-sls
  fargate:
    vpc:
      subnets:
        - subnet-2cc37974
        - subnet-3c959d5b
        - subnet-49d7d200
    tasks:
      my-task:
        image: mountainpass/infrastructure:helloworld


  # vpcConfig:
  #   zones:
  #     - ap-southeast-2a
  #     # - ap-southeast-2b
  #     # - ap-southeast-2c

provider:
  name: aws
  region: ap-southeast-2
  runtime: nodejs12.x
  iamRoleStatements:
    - Effect: Allow
      Action:
        - s3:*
      Resource: "*"

functions:
  postprocess:
    handler: handler.postprocess
    events:
      - s3:
          bucket: ${self:custom.bucket}
          event: s3:ObjectCreated:*
          rules:
            - suffix: .png

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.