Git Product home page Git Product logo

hk3427 / email-spam-detection-aws Goto Github PK

View Code? Open in Web Editor NEW

This project forked from nilanshnet/email-spam-detection-aws

0.0 0.0 0.0 41 KB

A spam detection system built on AWS cloud, that upon receipt of an email message, automatically flags it as spam or not. This is based on the prediction obtained from the machine learning model created using Amazon SageMaker. The definition and provision of the resources on AWS cloud is done through the AWS Cloudformation template

Jupyter Notebook 55.45% Python 44.55%

email-spam-detection-aws's Introduction

Spam Detector

About

A spam detection system built on AWS cloud, that upon receipt of an email message, automatically flags it as spam or not. This is based on the prediction obtained from the machine learning model created using Amazon SageMaker. The definition and provision of the resources on AWS cloud is done through the AWS Cloudformation template. This template can also be used with AWS code pipeline to automate the resource stack formation.

Design

Screenshot 2021-06-22 at 12 44 40 AM

Working

  1. Send an email to the email address that is used by the spam detector.

send

  1. Reply that we get, if the email is a spam:

spam

  1. Reply that we get, if the email is NOT a spam:

notSpam

Procedure

Prerequisites:

  • AWS cloud subscription (AWS Free tier works)
  • A custom domain
  • An email address for that domain ([email protected]). This will require adding up MX and TXT records on the domain provider site to receive an email. Refer the documentation of domain provider to add the records.

Standing up the AWS resources and Machine Learning spam classifier model using AWS Sagemaker:

  1. Use the Cloud Formation template to create the S3 buckets (for storing emails), AWS sagemaker endpoint, notebook instance, Lambda funtion and appropriate roles.
  2. Follow the reference link Build and train a spam filter machine learning model to create, train and deploy the spam classifier model. AWS sagemaker uses XGBoost ML algorithm and the Bank marketing data set to build and train the model. You can use your own custom made machine learning model and data set for this purpose.

Setting up Simple Email Service (SES) on AWS:

  1. Open SES home on AWS console and navigate to Domains. Verify a new domain (prerequisites) that you already own. Copy the 'TXT record' provided on console and add it in your Domain provider's site.
  2. From the SES home, go to the 'SMTP settings' and copy the 'server name'. Add this as a 'MX record' in your Domain provider's site, just like the 'TXT record' added in the last step. This and previous settings will link the Amazon SES with your custom email address.
  3. Now we need to receive the email that is sent on our custom email address, and store it in the S3 bucket. For this - from the SES home, navigate to the Email Receiving > Rule Sets > Create Rule. Use something like the below image to configure the rule set. Use the name of the S3 bucket which was created using the cloud formation template in the previous steps. Screenshot 2021-06-21 at 9 15 07 PM

Setting up Lambda function:

We need to setup a Lambda function that gets triggered whenever an email is received on the custom email address.

  1. Navigate to the lambda function created as part of the cloud formation template. From there, set up the trigger for the Lambda as your S3 bucket. Make sure that S3 has required IAM permissions for the Lambda function resource.
  2. For the code part of the Lambda function, use the Python code. If necessary, add layers in the lambda function for numpy and other libraries. Make sure that Lambda has required IAM permissions for the S3 resource, SES and CloudWatch.
    The lambda function is ready to send the email as shown in the working section, back to the sender using the custom email address linked with Amzon Simple Email Service.

References

email-spam-detection-aws's People

Contributors

nbodepudi avatar nilanshnet avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.