scaffold-serverless's Introduction

README: Scaffold Serverless

https://circleci.com/gh/v-studios/scaffold-serverless.svg?style=svg&circle-token=ce137f204153b1ed6098121c3d4cb42d287ea772

This is a scaffold for Serverless projects using AWS and the Serverless Framework. It's intended to be a base for creating other projects, with all the tools you usually don't add until the very end -- when it's too late. To that end, this includes:

test runner: py.test
documentation: compiled from RST with Sphinx to HTML and PDF
CI/CD: with CircleCI since that's what we use
pep8, lint, flake8 checks
badges on this page for docs, tests, etc.
(need some serverless code at this point)
multiple stages: local, feature-branch, dev, qa, prod
508 testing
security testing
tox

The application isn't terribly important but ideally should include common use cases, so here are a few features I can think of:

api: allowing GET, POST
uploading media: to S3, perhaps with presigned URL
uploading form data: to DynamoDB
authentication: with Cognito, so we get social media auth
modern web ui: with Angular2+ served from S3

To show the features of Serverless, we probably should include some triggers from (say) S3 uploads to populate the database.

The WebUI should allow login then show some metadata from the DB with links to the media in S3, and allow editing of both.

This will evolve over time and grow features, I hope. But the main point is to get the exostructure (tests, CI/CD, docs) working, since that's the boring stuff everyone avoids.

For details on each feature, head to the docs/ directory.

scaffold-serverless's People

Contributors

Stargazers

Watchers

scaffold-serverless's Issues

Consider using JSON for logs to provide structure

This would make it easier for tools like Elasticsearch and Kibana to parse attributes and produce useful metrics.

APIG has 30 second timeout so max out Lambda to be the same

Add open-source security scanner to CI pipeline

This list has python-specific tools and those that support multiple languages; proprietary ones are marked with (C) symbol:
https://github.com/mre/awesome-static-analysis#multiple-languages

OWASP lists tools including SonarQube as well as open source:
https://www.owasp.org/index.php/Source_Code_Analysis_Tools

Bandit, from OpenStack, looks useful for Python; perhaps 'pyt' as well:
https://github.com/openstack/bandit
https://github.com/python-security/pyt

Use lambda-decorators to simplify logging, return marshalling, etc

Should greatly reduce boilerplate.
https://github.com/dschep/lambda-decorators

Add TravisCI example build/deploy pipeline

TravisCI is free to use for open source code, so could be very helpful for folks who don't use CircleCI

Separate requirements.txt, requirements-dev.txt to reduce uploads

When we start bundling requirements (with https://github.com/UnitedIncome/serverless-python-requirements) , we'll want to keep development stuff out of the bundle. Put things like boto3, pytest and others in a separate requirements-dev.txt file will allow the plugin to skip the bloat

Configure CloudWatch logs to trim after 7 days

If they grow without bound, it can cost significant money.

Document expected uses of log levels

We've used log levels inconsistently; document a standard practice, like:

CRITICAL: wake me up at 3am; example: the lambda can't connect to the database
ERROR: don't wake me up, but definitely tell me about; I probably gotta fix it
WARNING: something the software can recover from, I don't need to be alerted on my phone
INFO: something interesting happened, perhaps a significant milestone, progress in an operation's lifecycle, to answer customer questions like "what happened to that file I uploaded"
DEBUG: noise, probably should be turned on in Dev, QA systems, and off in Prod