rurri / serverless-resources-env Goto Github PK

I have CloudFormation with 100 resources and I get this error when using serverless-resources-env plugin: "Request must be smaller than 5120 bytes for the UpdateFunctionConfiguration operation".

Written .env file has 6100 bytes. I think this is AWS limit.

This is currently using the describeStackResources method which is limited to 100 resources. It should use listStackResources and will have to handle resource pagination.

This looks to be similar to #12 but I believe it's unique, and it may be working as intended, but the way our serverless.yml file is setup isn't working with the plugin:

We have environment-specific configuration values (e.g. AWS account number, region, etc.) in a separate serverless.<stage>.yml where is any of dev, stage, or prod.

In your main serverless.yml file, we're merging these into the config like:

provider:
  name: aws
  stage: ${opt:stage, 'dev'}
  runtime: python3.6
  region: ${file(./serverless.${self:provider.stage}.yml):DEFAULT_REGION}

It doesn't appear that the interpolation that serverless does is being evaluated when pulling the CloudFormation details after the deployment. The plugin exits with the following error:

Serverless: [serverless-resources-env] Looking up resources for CF Named: example-stack-dev

  Unknown Endpoint ---------------------------------------

  Inaccessible host: `cloudformation.'. This service may not be available in the `${file(./serverless.${self:provider.stage}.yml):DEFAULT_REGION}' region.

When serverless-webpack plugin is used getEnvDirectory() function called from different hooks returns different values:

1. afterDeploy hook -> createCFFile() function.
   getEnvDirectory() returns this value:
   /path/to/project/.serverless-resources-env/.us-east-1_dev_functionName
   This path is used to create envs file.

2. beforeLocalInvoke hook.
   getEnvDirectory() returns this value:
   /path/to/project/.webpack/service/.serverless-resources-env/.us-east-1_dev_functionName
   This path is used to read envs file but it doesn't exist because envs were written to the different directory on the previous step.

Would be possible to create environment variables from cloudformation outputs?

Thank you.

Exact error produced in the CLI Log

Serverless: [serverless-resources-env] WARNING: Could not find cloud formation resources for addPartner.Could not find: [List of resources missing]

Problem

If you have a stack that involves more then 100 resources (not hard to do) this plugin will fail to grab some of those resources when it makes the request to Amazon. It is using the describe-stack-resources command that is limited to the first 100 resources in the stack.

Code where the problem exists

index.js

/**
  * Looks up the CF Resources for this stack from AWS
  * @returns {Promise.<String>}
  */
 fetchCFResources() {
   const stackName = this.getStackName();
   this.serverless.cli.log(`[serverless-resources-env] Looking up resources for CF Named: ${stackName}`);
   return Promise.promisify(this.cloudFormation.describeStackResources.bind(this.cloudFormation))({
     StackName: stackName,
   });
 }

Fix

Please update this to instead use the list-stack-resources function provided by AWS. By default it uses pagination. I believe there is an option to not paginate if that is easier.

I would totally update it and make a pull request but javascript is definitely not in my wheelhouse for now. Doesn't mean I'm not going to try, but I don't have all that much time to spend on it.

Appreciation

I really do appreciate that you took the time to make this plugin. It really allows me to create a very dynamic application and deploy it at whim without having to worry about all the resources new id's/keys. I would really like to see this fixed. This is the only fault I've seen so far.

Bribery

Says you're from the Long Beach, CA in your profile. I call the LBC home too. Fix and deploy and I'll buy you a beer! 🍺

I'm getting an error when trying to deploy a function with environment defined.

This is my function definition:

functions:
processor:
handler: functions/processor/handler.handler
events:
- schedule:
rate: rate(5 minutes)
enabled: true
environment:
MAX_NUMBER_OF_MESSAGES: 10
custom:
env-resources:
- processingQueue

The error I'm getting is: Expected params.Environment.Variables['MAX_NUMBER_OF_MESSAGES']
to be a string

I am trying to use this plugin (v0.3.0) with serverless 1.5 for the first time.
For installation and configuration I followed: https://www.npmjs.com/package/serverless-resources-env

When I deploy with serverless deploy I get the following error:

Serverless: [serverless-resources-env] Looking up resources for CF Named: cloudformationStatus-development

  Invalid Client Token Id --------------------------------

     The security token included in the request is invalid.

The detailed stack trace looks like this:

InvalidClientTokenId: The security token included in the request is invalid.
    at Request.extractError (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/protocol/query.js:40:29)
    at Request.callListeners (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:668:14)
    at Request.transition (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:670:12)
    at Request.callListeners (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at Request.emit (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:668:14)
    at Request.transition (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:670:12)
    at Request.callListeners (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at callNextListener (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
    at IncomingMessage.onEnd (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/event_listeners.js:231:11)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7)

I'm getting a warning for the custom property in my function.

Warning: Invalid configuration encountered
  at 'functions.contact': unrecognized property 'custom'

Everything is working fine, but it would be nice if this plugin could somehow tell Serverless that this is expected.

My package info:

        "serverless": "^3.36.0",
        "serverless-finch": "^4.0.3",
        "serverless-resources-env": "^0.3.1",

Hi, I am trying to use your plugin but I have encountered following error for the service called sample with stage dev:

Stack with id sample-dev does not exist

My serverless.yml config is bit longer so I am including only important parts:

service:
  name: ${opt:name, 'sample'}
  publish: false

plugins:
  - serverless-appsync-plugin
  - serverless-webpack
  - serverless-resources-env

provider:
  name: aws
  runtime: nodejs8.10
  region: eu-west-1
  stage: ${opt:stage, 'dev'}
  profile: ${opt:aws-profile, 'default-profile'}

functions:
  someFunction:
    handler: handler.someHandler
    env-resources:
      - CognitoUserPoolUserPool # defined below

resources:
   CognitoUserPoolUserPool:
      Type: AWS::Cognito::UserPool
      Properties:
         - someProperties
         # ...

Do you know where the problem can be? When I remove your plugin from the configuration it works but also I need it so I have Cognito's UserPoolId within my Lambda function. Before trying your plugin I have tried reference it manually via CloudFormation output (https://serverless.com/framework/docs/providers/aws/guide/variables/#reference-cloudformation-outputs) which is probably what the plugin does in the background but with no luck - there was the same error.

Any idea? I am not sure how exactly the CloudFormation stack ID works related to the serverless stages.

Thanks for any help

Hi,

Could you release the latest version of the package, please?

Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading service .zip file to S3 (6.28 MB)...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
....................................
Serverless: Stack update finished...
Serverless: Removing old service versions...
Service Information
service: walkmeApi
stage: dev
region: us-west-2
api keys:
None
endpoints:
POST - https://bdh02rome8.execute-api.us-west-2.amazonaws.com/dev/users
GET - https://bdh02rome8.execute-api.us-west-2.amazonaws.com/dev/users
GET - https://bdh02rome8.execute-api.us-west-2.amazonaws.com/dev/users/{id}
PUT - https://bdh02rome8.execute-api.us-west-2.amazonaws.com/dev/users/{id}
DELETE - https://bdh02rome8.execute-api.us-west-2.amazonaws.com/dev/users/{id}
functions:
walkmeApi-dev-update: arn:aws:lambda:us-west-2:235544613716:function:walkmeApi-dev-update
walkmeApi-dev-triggerEmail: arn:aws:lambda:us-west-2:235544613716:function:walkmeApi-dev-triggerEmail
walkmeApi-dev-triggerAlert: arn:aws:lambda:us-west-2:235544613716:function:walkmeApi-dev-triggerAlert
walkmeApi-dev-delete: arn:aws:lambda:us-west-2:235544613716:function:walkmeApi-dev-delete
walkmeApi-dev-triggerCreate: arn:aws:lambda:us-west-2:235544613716:function:walkmeApi-dev-triggerCreate
walkmeApi-dev-get: arn:aws:lambda:us-west-2:235544613716:function:walkmeApi-dev-get
walkmeApi-dev-list: arn:aws:lambda:us-west-2:235544613716:function:walkmeApi-dev-list
walkmeApi-dev-create: arn:aws:lambda:us-west-2:235544613716:function:walkmeApi-dev-create
Serverless: [serverless-resources-env] Looking up resources for CF Named: walkmeApi-dev

Config Error -------------------------------------------

 Missing region in config

 For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

Stack Trace --------------------------------------------

ConfigError: Missing region in config
at Request.VALIDATE_REGION (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/event_listeners.js:81:45)
at Request.callListeners (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at callNextListener (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
at /usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/event_listeners.js:75:9
at finish (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/config.js:308:7)
at /usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/config.js:324:9
at SharedIniFileCredentials.get (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/credentials.js:126:7)
at getAsyncCredentials (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/config.js:318:24)
at Config.getCredentials (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/config.js:338:9)
at Request.VALIDATE_CREDENTIALS (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/event_listeners.js:70:26)
at Request.callListeners (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:101:18)
at Request.emit (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:668:14)
at Request.transition (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/state_machine.js:14:12)
at Request.runTo (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:394:15)
at Request.send (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/request.js:358:10)
at makeRequest (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/service.js:193:27)
at svc.(anonymous function) [as describeStackResources] (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/service.js:451:23)
at tryCatcher (/Users/robweaver/Sites/workfront/SharedDeployScripts/AWS/serverless/WalkMe/node_modules/bluebird/js/release/util.js:16:23)
at ret (eval at makeNodePromisifiedEval (/Users/robweaver/Sites/workfront/SharedDeployScripts/AWS/serverless/WalkMe/node_modules/bluebird/js/release/promisify.js:184:12), :13:39)
at ServerlessResourcesEnv.fetchCFResources (/Users/robweaver/Sites/workfront/SharedDeployScripts/AWS/serverless/WalkMe/node_modules/serverless-resources-env/index.js:106:99)

Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues

 Please report this error. We think it might be a bug.

Your Environment Information -----------------------------
OS: darwin
Node Version: 6.7.0
Serverless Version: 1.4.0