Note
This mini project's objective is to streamline data insertion into DynamoDB through an S3 bucket using AWS Lambda, enhancing automation efficiency within a serverless architecture.
Keep other settings of the table as default and create the table.
![[Pasted image 20240322172032.png]]
Rest of the settings are default now we can create the lambda function.
In order to land to this page we need to select the lambda function → click on Configuration → click on General configuration → set the time limit.
Policies have been attached to access the S3 bucket and the dynamoDB. In order to land to this page we need to click on the lambda function → select the permissions tab → select the add permissions to the S3 bucket and the dynamodb tables.
{
"CustomerID": "01",
"Product": "Coffee",
"Address": "135, Rose avenue",
"Quantity": "4"
}Create in sample json file in notepad and upload that into the S3 bucket.
In the bottom we can notice that there is Key that had been created. Copy that, which shall be used in the boto3 code.
Look into the request syntax mentioned in the documentation:
https://boto3.amazonaws.com/v1/documentation/api/1.9.42/reference/services/s3.html#S3.Client.get_object
Look into the same link for the response syntax:
This response structure is a dictionary
import json
import boto3
client = boto3.client('s3')
def lambda_handler(event, context):
response = client.get_object(
Bucket='boto3-bucket-01',
Key='Dynamodb_sample.txt',
)
json_data = response['Body']
print(json_data)
print(type(json_data))For the first iteration deploy this code.
Important
Whenever the code is changed we need to deploy the code first and then test the code.
![[Pasted image 20240322195411.png]]
The code is functioning but the output is botocore.response.StreamingBody we need to understand what this means.
this class comes with some response, we need to use the method read() to understand the type of the output it gives. Now edit the code accordingly.
```python
import json
import boto3
client = boto3.client('s3')
def lambda_handler(event, context):
response = client.get_object(
Bucket='boto3-bucket-01',
Key='Dynamodb_sample.txt',
)
# convert from streaming data
json_data = response['Body'].read()
print(json_data)
print(type(json_data))
If we look into the results we can see it shows the output is in bytes. Which means all the contents that logs are displayed are in UTF-8.
Note
”The <class 'bytes'> indicates the data type of the byte string, which is bytes in Python. Bytes are a data type used to represent sequences of bytes (i.e., binary data) in Python. They are immutable and can contain any raw data, including text data encoded in various formats like UTF-8.”
Finally we are trying to convert the byte into a string and code is changed as follows:
import json
import boto3
client = boto3.client('s3')
def lambda_handler(event, context):
response = client.get_object(
Bucket='boto3-bucket-01',
Key='Dynamodb_sample.txt',
)
# convert from streaming data to byte
json_data = response['Body'].read()
# convert data from byte to string
data_string = json_data.decode('UTF-8')
print(data_string)
print(type(data_string))And the result is:
![[Pasted image 20240322195526.png]]
Note
JSON keys can only be strings. The dictionary's keys can be any hashable object. The keys in JSON are ordered sequentially and can be repeated. The keys in the dictionary cannot be repeated and must be distinct.
import json
import boto3
client = boto3.client('s3')
def lambda_handler(event, context):
response = client.get_object(
Bucket='boto3-bucket-01',
Key='Dynamodb_sample.json',
)
# convert from streaming data to byte
json_data = response['Body'].read()
# convert data from byte to string
data_string = json_data.decode('UTF-8')
# print(data_string)
# print(type(data_string))
# convert from json string to dictionary
data_dict = json.loads(data_string)
print(data_dict)
print(type(data_dict))
Note
Start off by adding the service resource dynamodb=boto3.resource('dynamodb')
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html#amazon-dynamodb
import json
import boto3
client = boto3.client('s3')
dynamodb=boto3.resource('dynamodb')
def lambda_handler(event, context):
response = client.get_object(
Bucket='boto3-bucket-01',
Key='Dynamodb_sample.json',
)
# convert from streaming data to byte
json_data = response['Body'].read()
# convert data from byte to string
data_string = json_data.decode('UTF-8')
# print(data_string)
# print(type(data_string))
# convert from json string to dictionary
data_dict = json.loads(data_string)
print(data_dict)
print(type(data_dict))
table = dynamodb.Table('RetailSales770')
table.put_item(
Item=data_dict
)Result:
Now if you modify the same file and upload it there will be new item added to it.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent