Lambda allows you to define various triggers which can be hundreds of different events provided by dozens of different event sources. In our case, we’re going to use the S3 event provider. Events are being fired all of the time in S3 from new files that are uploaded to buckets (incoming bucket),Print data of files and files being copy to processed bucket.
To deploy, Serverless framework required a aws accesskey and secretkey. Which can be set by below command
serverless config credentials --provider aws --key <<KEY>> --secret <<SECRET>>
To deploy this application , need not to create any resource on aws. Deployment will handle all of required resources.To start deployment need to execute below command at command prompt.
sls deploy
Upload one file to "incominglambda" bucket, that has the subscribed event, this should automatically kick off the Lambda function. To confirm this, head over to CloudWatch or click on the Monitoring tab inside of the function itself. This initial view shows a lot of great information about the function’s execution. Testing file "testS3data.txt" is also in this repository.
Log output would looks like :-
START RequestId: 495c1fd6-9c71-4061-9e8f-41759d576dcc Version: $LATEST
START Printing contents of file -> testS3data.txt
Hi
my name is sumit
testing my S3 trigger on lambda -> CloudWatch log
END Printing contents of file -> testS3data.txt
START Transfering file -> testS3data.txt to bucket processedlambda
END Transfering file -> testS3data.txt to bucket processedlambda
To remove deployment just need to execute below command
sls remove
In order to make it proper running , have to do many time clean all S3 bucket. Same would also required when we did many round of test and now time to remove all test files. To cleanup S3 bucket need to to go to aws. Have added one plugin "serverless-s3-remover". Using below command can install it ...
npm install serverless-s3-remover
Cleanup command :- When we're removing our deployemnt , this plugin also doing cleanup...
sls s3remove
Go to S3 service using Service -> search S3. Now need to create two buckets - incominglambda & processedlambda. During creating bucket keep all defaults settings. We'll use incominglambda bucket to upload files and after printing , files will be copied into processedlambda bucket.
When executed, Lambda needs to have permission to access your S3 bucket. Before we get started building our Lambda function, we must first create an IAM role( or assign policy to existing user) which Lambda will use to work with S3.
Right now both bucket "incominglambda" & "processedlambda" are hardcoded into lambda handler. We can make it configurable too. in this example , hardcoded inbucket and out bucket name. We can also dynamically refer inbucket name as
inbucket = j['s3']['bucket']['name']
Same way file, if we want to trigger from SNS event and have to read SNS message we can use like
import ast
sns_message = ast.literal_eval(j['Sns']['Message'])