Webtask has some unique things regarding their execution envionment. Let's double check that we don't need to do something crazy like mash it into a single file.

Webtask is free to try. I did their hello world and got back... Just by running a one liner. But could not get two files to run.

https://wt-decac903c5c2d6bfe4e6f2927fd436b3-0.run.webtask.io/hello

{"PATH":"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","HOSTNAME":"98ba9fa69a96","USER_ID":"30039","CLUSTER_SIZE":"1","PORT":"/data/io/port.sock","BACKCHANNEL_PORT":"/data/backchannel.sock","OS_LIMITS":"nproc=100:100,nofile=2048:2048","SANDBOX_STATS_HEADER":"1","EDGE_CS_CACHE":"1","EDGE_CS_TEMP_DIR":"/data/io","TRIPWIRE_TIMEOUT_MS":"2000","COOLDOWN_TIMEOUT":"1000","MAX_CODE_SIZE":"100","GET_CODE_TIMEOUT":"10000","MAX_CODE_CACHE_COUNT":"10","VERQUIRE_DIR":"/data/_verquire","NPM_CONFIG_LOGLEVEL":"info","NODE_VERSION":"4.4.5","HOME":"/root","NODE_ENV":"webtask"}

I got the error when trying to load both files.
{
"code": 400,
"message": "Invalid webtask code",
"error": "Supplied code must return or export a function."
}

Each run should also log a date time in epoch as part of the json

See python-lambda-inspector Issue #10

------------------------------------------ Also opened in node because it will need a solution as well.

Turns out lambda functions don't get any access to the internet without the presence of a cost prohibitive NAT gateway. This means that lambda functions running inside of the ThreatResponse AWS account will need to POST their results in a different way than runtimes out in the wild.

Potential options are:

1. Profiler writes a {uuid.hex()}.json.gz file directly to S3 the same way the API does
2. Profiler writes to dynamo and we pick that up somewhere else ( seems unnecessary )
3. Deploy NAT gateway. ( Not cost effective ).
4. Deploy the Lambda in the same VPC as the API box and point it directly at the API instead.

So I'm sure that you've gathered option 1 is preferred. It's just a matter of writing a little logic that only does the S3 upload if you're running from within a lambda function. We'll still need the urllib2.Request method in the python profiler that @jeffbryner wrote. Oddly it doesn't actually cause the function to fail. Simply nothing ever happens...

Option 4. isn't a bad choice either but has implications for if / when we want to go multi-region and puts heavier requirements on the CI/CD pipeline to attach things.

Post collected output as json to:

https://showdown-api.ephemeralsystems.com/

Webtask has a secret store... turns out you can just ask nicely.

https://webtask.io/docs/editor/secrets

• /proc/meminfo to dictionary
• /proc/cpuinfo to dictionary
• contents of /etc/issue
• sandbox variable ( this detects which runtime we're in )
• package_count count of the list of loaded modules
• platform
• warmness fields, is_warm, warm_for

Think it's due to bad JSON.

Currently, posting even very simple JSON structures like: '{"pwd":"/Users/themnem/Code/lambda-env/node-lambda-inspector\\n"}' will work sometimes but not other times.

Last night the POST that completed successfully happened after I commented out everything in the lookups dictionary and only used pwd. Next step could be uncommenting them one by one and seeing what starts returning 400s. Unfortunately, after getting one 400 I was unable to recreate the original 200. Is it possible that the server can get put in a bad state after receiving JSON it doesn't like?

Current work for further testing is in branch upload-to-post

See:
This is the same story but nodejs

ThreatResponse/python-lambda-inspector#22

/Users/akrug/workspace/node-lambda-inspector/profiler.js:219
callback(err, res);

ps to dict like all the things for analysis in ES