Git Product home page Git Product logo

lambdium's Introduction

lambdium

headless chrome + selenium webdriver in AWS Lambda

This uses the binaries from the serverless-chrome project to prototype running headless chromium with selenium-webdriver in AWS Lambda. I've also bundled the chromedriver binary so the browser can be interacted with using the Webdriver Protocol.

Background

The function interacts with headless Chromium process using chromedriver and a popular webdriver node.js client library.

This is highly experimental and not all chromedriver functions will work. Check issues for known issues.

Since this Lambda function is written using node.js, you can run almost any script written for selenium-webdriver. Example scripts can be found in the examples directory.

Installation

If you don't want to build the archive, you can also download a pre-built AWS Lambda *.zip that you can upload directly to a new node6.10 runtime function using the AWS CLI or Console with at least 256mb of memory. Visit lambdium releases.

Requirements

  • An AWS Account
  • Terraform (optional but highly recommended for function creation and deploy)
  • node.js + npm
  • make

Fetching dependencies

The headless chromium binary is too large for Github, you need to fetch it using a script bundled in this repository. Marco Lüthy has an excellent post on Medium about how he built chromium for for AWS Lambda here.

    $ ./scripts/fetch-dependencies.sh

Building Lambda .zip archive

    $ make

Creating and Deploying Using Terraform

This will create the function using Terraform with all required permissions.

    $ make deploy

The optional DEBUG_ENV environment variable will log additional information to Cloudwatch. The PATH environment variable points to the bin directory in this project—this is required in order to launch the chromedriver process.

Usage

If dependencies are installed and make deploy succeeds you can have Lambda run a script. There's an example of a selenium-webdriver simple script in the examples/ directory that the Lambda function can now run.

Expected JSON input for the function: {"Base64Script": "<Base64 Encoding of Selenium Script>"} (this can also be provided as an environment variable named BASE64_SCRIPT).

To run the example Selenium script, you can use the example with the AWS CLI in the scripts directory. It takes care of base64 encoding the file:

    $ scripts/invoke.sh

To use your own selenium-webdriver script:

    $ scripts/invoke.sh ~/Desktop/my-script.js

Related projects

lambdium's People

Contributors

smithclay avatar andrewmkrug avatar

Stargazers

Brandon avatar Eden avatar

Watchers

James Cloos avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.