architect / functions Goto Github PK

When my default WebSocket Lambda is invoked, it attempts to send the received message to all other connected client:

const arc = require('@architect/functions');

exports.handler = async function handler(payload) {
  const data = await arc.tables();
  const { Items: clients } = await data.connections.scan({});
  const { body: json, requestContext: { connectionId: self } } = payload;
  const body = JSON.parse(json);
  await Promise.all(clients.map(({ connectionId: id }) => {
    if (id === self) {
      return;
    }
    const message = { id, payload: body };
    arc.ws.send(message);
  }));
  return { statusCode: 200 };
}

This works locally in Sandbox, but within AWS it throws an error:

AccessDeniedException: User: arn:aws:sts::<id>:assumed-role/<project>Staging-Role-<id>/<project>Staging-WebsocketDefault-<id> is not authorized to perform: execute-api:Invoke on resource: arn:aws:execute-api:eu-west-1:********9001:<id>/staging/POST/@connections/@connections/Bw91Ye3FjoECFKw%3D

(I've stripped some of the IDs out of that for obvious reasons).

Unsure if this is a functions issue invoking the wrong thing, or deploy/IAM related?

need to look into this! avoids having to upgrade the aws-sdk in the lambdas

https://gist.githubusercontent.com/robzhu/2714d1c5183be9e620beba8b0d567544/raw/ed8b172569a0af95831f1221e0d80fce8dca791d/patch.js

Describe the issue

Whereas

exports.handler = async req => ({ statusCode: 202 })

returns an empty response,

exports.handler = arc.http.async(async req => ({ statusCode: 202 }))

returns '\n' in the body (both in the sandbox and deployed). I think it might be coming from https://github.com/architect/functions/blob/master/src/http/_res-fmt.js#L17

This looks like a bug to me.

Expected behavior

I would still expect an empty response.

Additional context
@architect/[email protected]

I was writing a style-loader here that converts css to js so that the can be imported from js modules. ie. import "styles.css" and am using it like the following.

exports.handler = arc.proxy.public({
  spa: true,
  plugins: {
    scss: ["@architect/proxy-plugin-sass", "@bvkimball/proxy-plugin-style-loader"],
  },

but this means all css and scss files would be converted to js, which is not my intention. I was thinking if the plugin had access to the request.headers.Referrer i would know what file request it and if it was js,jsx,mjs,etc... i would convert or not.

Alternatively i thought maybe loading the file with a query param, but that is not available in the Key value that is passed to the plugin.
eg.

import "styles.css?module"

You would need to pass the request here.
https://github.com/architect/functions/blob/master/src/http/proxy/public.js#L54

remove all calls to aws-sdk and add these to the deps

• arc.http https://www.npmjs.com/package/@aws-sdk/client-s3-node
• arc.events https://www.npmjs.com/package/@aws-sdk/client-sns-node
• arc.queues https://www.npmjs.com/package/@aws-sdk/client-sqs-node
• arc.ws https://www.npmjs.com/package/@aws-sdk/client-apigatewaymanagementapi-node and maybe https://www.npmjs.com/package/@aws-sdk/client-apigatewayv2-node
• arc.tables https://www.npmjs.com/package/@aws-sdk/client-dynamodb-v2-node

Additionally we need to replace calls to SSM with

• https://www.npmjs.com/package/@aws-sdk/client-ssm-node

few use or like this interface so low priority

Describe the issue

Using static in Arc 6.0.0 sandbox fails on one of my machines:

Error
ENOENT: no such file or directory, open 'C:\Users\mikem\Code\temp\appless\src\http\get-index\node_modules\@architect\shared\.arc'0.

Error: ENOENT: no such file or directory, open 'C:\Users\mikem\Code\temp\appless\src\http\get-index\node_modules\@architect\shared\.arc'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:343:35)
    at _static (C:\Users\mikem\Code\temp\appless\node_modules\@architect\functions\src\static\index.js:31:20)
    at C:\Users\mikem\Code\temp\appless\src\http\get-index\node_modules\@architect\shared\layout.js:34:25
    at Array.forEach ()
    at layout (C:\Users\mikem\Code\temp\appless\src\http\get-index\node_modules\@architect\shared\layout.js:33:17)
    at http (C:\Users\mikem\Code\temp\appless\src\http\get-index\index.js:23:11)
    at process._tickCallback (internal/process/next_tick.js:68:7)
    at evalScript (internal/bootstrap/node.js:592:13)
    at startup (internal/bootstrap/node.js:265:9)

Versions:

    "@architect/architect": "^6.0.0",
    "@architect/functions": "^3.3.2",
    "@architect/sandbox": "^1.3.6",

Also Windows 10, using arc.json.

The error occurs in:
https://github.com/architect/functions/blob/master/src/static/index.js#L25

From Slack discussion, arc should no longer expect:

some-lambda/node_modules/@architect/shared/.arc

to exist, but this isn't true according to

https://github.com/architect/functions/blob/master/src/static/index.js#L12

Workarounds

• If I do not use static, the issue goes away.
• If I add the file to some-lambda/node_modules/@architect/shared/.arc, the issue goes away.

Just curious about code style, any particular reason why you've used var and no let or const?

Edit: found some let usage in /index.js ;)

According to the docs, calling res with an Error object lets you return a 404 error if you set the code property of the error object. I'm attempting to do this as follows in a @json route handler:

let arc = require("@architect/functions");

function route(req, res) {
    const error = Error("Record not found");
    error.code = 404;
    res(error);
}

exports.handler = arc.json.get(route);

With npx sandbox, requesting this route crashes the server with an error:

/Users/kyank/code/project/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

Error: Record not found
    at Object.route (/Users/kyank/code/project/src/json/get-record-000recordId/index.js:22:19)
    at _iter (/Users/kyank/code/project/src/json/get-surveys-000surveyId/node_modules/@architect/functions/src/http/_request.js:73:12)
    at _read (/Users/kyank/code/project/src/json/get-surveys-000surveyId/node_modules/@architect/functions/src/http/_request.js:74:9)
    at _find (/Users/kyank/code/project/src/json/get-surveys-000surveyId/node_modules/@architect/functions/src/http/session/read.js:37:5)
    at Response._get (/Users/kyank/code/project/src/json/get-surveys-000surveyId/node_modules/@architect/functions/src/http/session/_find.js:17:9)
    at Request.<anonymous> (/Users/kyank/code/project/node_modules/aws-sdk/lib/request.js:364:18)
    at Request.callListeners (/Users/kyank/code/project/node_modules/aws-sdk/lib/sequential_executor.js:109:20)
    at Request.emit (/Users/kyank/code/project/node_modules/aws-sdk/lib/sequential_executor.js:81:10)
    at Request.emit (/Users/kyank/code/project/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/Users/kyank/code/project/node_modules/aws-sdk/lib/request.js:22:10)

With the project deployed to AWS, requesting this route returns a 500 error, and logs this in CloudWatch:

{
    "errorMessage": "{\"statusCode\":404,\"json\":{\"message\":\"Record not found\",\"code\":404,\"time\":{},\"name\":\"Error\",\"stack\":\"Error: Record not found\\n    at Object.route (/var/task/index.js:22:19)\\n    at _iter (/var/task/node_modules/@architect/functions/src/http/_request.js:73:12)\\n    at _read (/var/task/node_modules/@architect/functions/src/http/_request.js:74:9)\\n    at _find (/var/task/node_modules/@architect/functions/src/http/session/read.js:37:5)\\n    at Response._get (/var/task/node_modules/@architect/functions/src/http/session/_find.js:17:9)\\n    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:364:18)\\n    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)\\n    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)\\n    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)\\n    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)\"}}"
}

Howdy 👋

I ran into a couple of issues getting the arc-example-ws repo working. I was able to get the example app working eventually, but my best guess is that the problems I ran into stem from this project.

• src/ws/index.js requires aws-sdk , but it's declared as a dev dependency in @architect/functions, so it's not available if you just do a normal npm install
  • worked around this by manually installing aws-sdk across src/ws/*
• AccessDeniedException is thrown because the role assigned to the src/ws/* lambda was missing the execute-api:ManageConnections permission (full error message below)
  • worked around this by manually adding that permission through the IAM console
  • I'm assuming that I didn't skip any steps to get here (following the docs), but I guess that could be possible?

Errors

~\code\clones\arc-example-ws $ npx logs src/ws/ws-default
        app ~ test-ws
     region ~ us-west-2
    profile ~ xxx
    version ~ 5.0.7
Feb 03, 05:36:41 PM

ws-default called with { requestContext:

{ routeKey: '$default',

authorizer: '',

messageId: 'UjSOXedLvHcCH1Q=',

integrationLatency: '',

eventType: 'MESSAGE',

error: '',

extendedRequestId: 'UjSOXFftvHcFiow=',

requestTime: '04/Feb/2019:01:36:40 +0000',

messageDirection: 'IN',

stage: 'staging',

connectedAt: 1549244197491,

requestTimeEpoch: 1549244200779,

identity:

{ cognitoIdentityPoolId: null,

accountId: null,

cognitoIdentityId: null,

caller: null,

sourceIp: '73.92.10.73',

accessKey: null,

cognitoAuthenticationType: null,

cognitoAuthenticationProvider: null,

userArn: null,

userAgent: null,

user: null },

requestId: 'UjSOXFftvHcFiow=',

domainName: 'kasodogei2.execute-api.us-west-2.amazonaws.com',

connectionId: 'UjSN2edEvHcCH1Q=',

apiId: 'kasodogei2',

status: '' },

body: '{"text":"hullo"}',

isBase64Encoded: false }
Feb 03, 05:36:41 PM

TypeError: aws.ApiGatewayManagementApi is not a constructor

Object.send (/var/task/node_modules/@architect/functions/src/ws/index.js:54:23)

ws (/var/task/index.js:15:23)

Feb 03, 05:36:41 PM
Billed Duration: 100 ms
Memory Size: 1152 MB
Max Memory Used: 38 MB
Feb 03, 05:55:44 PM

ws-default called with { requestContext:

{ routeKey: '$default',

authorizer: '',

messageId: 'UjVA_ed1PHcCEww=',

integrationLatency: '',

eventType: 'MESSAGE',

error: '',

extendedRequestId: 'UjVA_EL5PHcFeyg=',

requestTime: '04/Feb/2019:01:55:43 +0000',

messageDirection: 'IN',

stage: 'staging',

connectedAt: 1549245340677,

requestTimeEpoch: 1549245343908,

identity:

{ cognitoIdentityPoolId: null,

accountId: null,

cognitoIdentityId: null,

caller: null,

sourceIp: '73.92.10.73',

accessKey: null,

cognitoAuthenticationType: null,

cognitoAuthenticationProvider: null,

userArn: null,

userAgent: null,

user: null },

requestId: 'UjVA_EL5PHcFeyg=',

domainName: 'kasodogei2.execute-api.us-west-2.amazonaws.com',

connectionId: 'UjVAeedwPHcCEww=',

apiId: 'kasodogei2',

status: '' },

body: '{"text":"hullo"}',

isBase64Encoded: false }
Feb 03, 05:55:44 PM

AccessDeniedException: User: arn:aws:sts::226364732856:assumed-role/arc-role/test-ws-staging-ws-default is not authorized to perform: execute-api:ManageConnections on resource: arn:aws:execute-api:us-west-2:********2856:kasod

ogei2/staging/POST/@connections/{connectionId}

Object.extractError (/var/task/node_modules/aws-sdk/lib/protocol/json.js:51:27)

Request.extractError (/var/task/node_modules/aws-sdk/lib/protocol/rest_json.js:52:8)

Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:106:20)

Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:78:10)

Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:683:14)

Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)

AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)

/var/task/node_modules/aws-sdk/lib/state_machine.js:26:10

Request. (/var/task/node_modules/aws-sdk/lib/request.js:38:9)

Request. (/var/task/node_modules/aws-sdk/lib/request.js:685:12)

I did end up getting the example app working, but it took a little bit of debugging to get to hello world.

Thanks! 🤗

Describe the issue
Architect v5 projects using arc.events.publish may unexpectedly fail in the following conditions:

• A function fires a number of publish events in parallel (say, 5)
• The app's SNS has a few hundred events (say, 200-300)

In this block events.publish loops through all topics until it finds what it's looking for, then fires the event; however, by default SNS listTopics can only run 30tps, so enough loops through at the same time and your function will unexpectedly error.

sessions are currently hardcoded to 1 week / we should allow to config

Describe the issue
If I put this in my unit-test:

const arc = require('@architect/functions')

I get this error:

Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.

Steps to reproduce

Make a test like this:

/* global describe, it, expect, beforeAll, afterAll */

// this causes async-hang
const arc = require('@architect/functions')

const sandbox = require('@architect/sandbox')

beforeAll(async () => {
  console.log('Sandbox: start')
  await sandbox.start({ quiet: true })
})
afterAll(async () => {
  console.log('Sandbox: end')
  await sandbox.end()
})

describe('jest', () => {
  it('should have unit-tests', async () => {
    expect(1 + 1).toEqual(2)
  })
})

run jest.

If you comment out the require of @architect/functions, it runs fine and start/stops the sandbox (on 1.5.9-RC.2.)

Expected behavior
I expect no breaking async side-effects on import, so there is no error.

Solutions

I narrowed the problem down to 3 lines in 2 files, in @architect/functions/src/tables/, db.js & doc.js:

both have code like this:

if (!testing) {
  const agent = new https.Agent({
    keepAlive: true,
    maxSockets: 50,
    rejectUnauthorized: true
  })
  aws.config.update({
     httpOptions: { agent }
  })
}

The offending bit is this:

  aws.config.update({
     httpOptions: { agent }
  })

If I remove the whole testing logic-branch (no custom agent) in both files, the issue goes away, but I haven't tested if the client still functions.

{
"devDependencies": {
    "@architect/sandbox": "1.5.9-RC.2",
    "jest": "^25.1.0"
  },
  "dependencies": {
    "@architect/functions": "^3.5.0"
  }
}
node 13.7.0

Describe the bug

Using @architect/functions 2.0.7, arc.http.helpers.static doesn't return the correct dir for Architect 5.5

`<script src="${static('/js/record-events.js')}"></script>`

when using Architect 5.0.x, correctly return:

http://localhost:3333/js/record-events.js

However when using Architect 5.5.x:

static should return http://localhost:3333/_static/js/record-events.js (which is where the file now lives)

But instead it returns

http://localhost:3333/js/record-events.js

Which 404s.

More of question than an issue.

I noticed on arc.codes we always prefix the session methods with await. Looking at the code in this repo, though, neither the old session code nor the new provider-based code leverage async functions.

So my question is: is it necessary? If no, why use it in the docs? Hedging your bet against future changes? What is the impact of using await with a synchronous function?

arc.http.helpers.url is currently hard coded to add /production but we changed the behavior to append /staging so it fails

demo repo
https://github.com/brianleroux/arc-example-binary

Presently we support body parsing (among other things) keying on application/json, but not application/vnd.api+json. We should add that.

/ht @jkarsrud!

Describe the bug
Per Twitter discussion between @brianleroux and I: currently to respond to a websocket, we do:

arc.ws().send()

arc.ws() takes a connectionId and an apiId, and send() takes a connectionId

It would be better to:

• Only require the connectionId once
• Cache apiId if possible

Something like arc.ws.send(connectionId, payload) would be great. This means we could store connectionId in DynamoDB / DocumentDB and use it to respond to websockets from outside the /src/ws - for example we could have an HTTP POST (called from an admin panel via /src/https/post-xxx-xxx-xxx) that sends a particular message to a subset of connected users via websockets.

B seemed to like this, so filing it here!

I might be able to get rid of that param (apiId)
By looking it up once and caching
We do that for events and queues
Hmm, ya we should
xnoɹǝʃ uɐıɹq 🐻
(thx for talking thru this w me)

Describe the issue
Calling arc.events.publish returns 'topic not found' on topics after the first ten returned from SSM.

Steps to reproduce

1. create an arc with 11 topics
2. create a function that calls each topic by name
3. one will fail.

I seems that ssm.getParametersByPath is limited to 10 results, which breaks in

function lookupTopics(callback) {
  let ssm = new aws.SSM
  let Path = `/${process.env.ARC_CLOUDFORMATION}`
  ssm.getParametersByPath({Path, Recursive:true}, function done(err, result) {
    if (err) callback(err)
    else {
      let topic = param=> param.Name.split('/')[2] === 'events'
      let topics = result.Parameters.filter(topic).reduce((a, b)=> {
        a[b.Name.split('/')[3]] = b.Value
        return a
      }, {})
      callback(null, result.Parameters)
    }
  })
}

In use, arc-functions does not look for app.arc, arc.yaml, or arc.json.

Lines: https://github.com/arc-repos/arc-functions/blob/master/src/http/helpers/_static.js#L4

Repro steps:

let arc = require('@architect/functions');

exports.handler = arc.proxy.public({
  spa: true
})

Will be broken without bucket.staging + bucket.production specified in the proxy config.

This is a follow up to this thread: https://architecture-as-text.slack.com/archives/C6BGT0D08/p1566800132296400

I wonder it would be possible to redirect <https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging> automagically to <https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging/>? With the former loading the JS/CSS assets breaks as the paths need to be relative, it tries to load <https://mzix1huilh.execute-api.us-west-2.amazonaws.com/index.js> instead of <https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging/index.js>

Is your feature request related to a problem? Please describe.

When referencing e.g. <script src="index.js"></script in public/index.html, then /index.js is loaded instead of /staging/index.js if the opened URL is https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging.

Describe the solution you'd like

https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging would ideally redirect to https://mzix1huilh.execute-api.us-west-2.amazonaws.com/staging/

Describe alternatives you've considered

Maybe try to redirect using history.pushState() before loading any assets, but not sure if that will work, didn't have a chance to try

Per my in-person conversation with @brianleroux earlier today, we need to set up some deep integration tests on live AWS infra that exercise everything in Functions, with environments for:

• Arc 5
• Arc 6
• sandbox

Describe the issue
A clear and concise description of the bug

Steps to reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error that says '....'

Expected behavior
A clear and concise description of what you expected to happen

Screenshots
If applicable, add screenshots to help explain your problem

Desktop
Please complete the following information (if appropriate):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Mobile
Please complete the following information (if appropriate):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context or notes about the problem here

Is your feature request related to a problem? Please describe.
arc.http (ie callback style) supports forward compatibility to Architect 6, but not the arc.http.middleware (ie async/await style) path. We should add this forwards-compatibility.

Describe the solution you'd like
Likely it'll just be a pure function to remap params right here: https://github.com/architect/functions/blob/master/src/http/middleware/index.js#L30

I'm guessing we'll just key on process.env.ARC_CLOUDFORMATION.

Following on from #33 I've been testing Arc Functions v4.

Using 6c43f8bc70190f8ede03791ebcc3eca40864bfbf from git, await arc.ws.send(connectionID, payload) will break a lambda in the sandbox

No logging, including console.log() before the arc.ws.send(), will be shown if the arc.ws.send() is enabled

Inspecting the arc.ws.send()code it looks like the promise is missing from arc.ws.send(), so wrapping it in a promisify:

let arc = require('@architect/functions'),
  makePayload = require('@architect/shared/make-payload'),
  util = require("util");

const webSocketSend = util.promisify(arc.ws.send);

require('@architect/shared/globals')

exports.handler = async function webSocketConnect(event) {
  let connectionID = event.requestContext.connectionId
  log(`Sending chat default greeting to new client at connection ID: ${connectionID}`)
  var payload = makePayload(`Hey there whats up!`, 'Mike')
  // BUG - if uncommented this will never return
  // await webSocketSend(connectionID, payload)
  log(`ws connect finished`)
  return {
    status: OK
  }
}

(Obviously I've tried it without the promisify too)

How can I make arc.ws.send() work in the sandbox?

This @json route handler returns a 404 error with a JSON response body:

let arc = require("@architect/functions");

function route(req, res) {
    res({
      json: { error: "Record not found" },
      status: 404,
    });
}

exports.handler = arc.json.get(route);

This works fine with npx sandbox, but once deployed to AWS, the response is malformed:

HTTP/1.1 404 Not Found
Content-Type: application/json
Content-Length: 29
Connection: close
Date: Tue, 09 Oct 2018 01:51:46 GMT
x-amzn-RequestId: e0eb42f5-cb65-11e8-8151-61272b94e470
Set-Cookie: 
x-amz-apigw-id: OeZz0GEwPHcF93g=
Location: {"json":{"error":"Record not found"},"status":404,"cookie":"_idx=Bc_8OuTkTXn2rPpp5-YUpJeT.oZB50nUXMa6f1pobTyvDF4lTciizMUZgRyaY8%2BdmrxI; Max-Age=2327449906161; Path=/; Expires=Sat, 03 Oct 2043 01:51:46 GMT; HttpOnly; Secure; SameSite=Lax","statusCode":404}
X-Amzn-Trace-Id: Root=1-5bbc09b2-077d8ab2d93f18d9ff444633;Sampled=0
X-Cache: Error from cloudfront
Via: 1.1 68807936c056006818525c5da31d108e.cloudfront.net (CloudFront)
X-Amz-Cf-Id: YQWk0k2i71QlZZ0OwGGEIA6IspUf4ajJj6WjwNnzUAvq8kIJ8vqGNw==

ord not found"},"status":404

Note the Location header value!

Starter project linters fail on the reserved word static

Describe the issue
In this commit we removed the proxy config in favor of setting an environment variable (ARC_STATIC_BUCKET), which Architect 6 does automatically.

This breaks Architect 5 users, and provides not an automatic means of setting the bucket without manually defining that env var.

Steps to reproduce
Steps to reproduce the behavior:

1. Use arc/fns 3.2.2 or later + Architect 5
2. Run proxy without ARC_STATIC_BUCKET env var set
3. A bucket config error will be returned

Expected behavior
Previously documented and released config params should be respected.

We can also read the Architect project manifest and automatically populate ARC_STATIC_BUCKET, but the user may want to specify a different bucket. (Unlikely, but possible.)

Describe the issue / Steps to reproduce

If the sandbox is launched where NODE_ENV is set to something unexpected (in my case, I used tunnelled, arc.static()` will return non-usable URLs.

This happened when I hapepned to reuse a terminal from another node app.

The issue comes from:

https://github.com/architect/functions/blob/master/src/static/index.js#L22

This defines

let runningLocally = !env || env === 'testing' || process.env.ARC_LOCAL

Which fails in the case where NODE_ENV exists but isn't a known value. Other parts of arc, like arc.http.helpers.url use different parsing logic, looking for explicit strings then falling back.

Options to fix

Ideally, have one function only that is called from anywhere that wants to look up the env, fixing issues like this (also allowing changes to be made more rapidly because one place needs to be modified, rather than many.) This should be like arc.http.helpers.url use different parsing logic, looking for explicit strings then falling back.

Describe the issue

Given logic like this

    const querystring = require('querystring')
    const qs = querystring.stringify({
      ids = [1,2,3,4]
    })
    const res = await fetch(`${rootUrl}records/TEST?${qs}`, { method, headers })

// qs evalues to "ids=1&ids=2&ids=3&ids=4"

you would expect for the underlying lambda to process it the same way

Expected behavior

while using arc.http.asyc

I would expect the value of req.query to be consistent

However.

In sandbox req.query.ids is indeed the array of ids as supplied

but when rootUrl is an actual API Gateway endpoint, req.query.ids is not an array. It holds the value of the last element in the ids array. I ended up needing to use req.multiValueQueryStringParameters.

I think what makes sense is to make it so that req.multiValueQueryStringParameters.ids is the only way to get these types of query strings and make sandbox behave this way too.

The guide at https://arc.codes/guides/background-tasks shows a callback-style @events handler. But as async @http handlers are supported, I assumed the callback example was just legacy. So I use an async @events handler:

export const handler = arc.events.subscribe(async record => {
  console.log(record)
  await doSth(record)
})

This works fine when deployed on AWS. However, today I noticed a problem with the sandbox when automating local system tests. The mocha tests would not terminate properly because a subprocess for src/sandbox/events/_subprocess.js remained. Turns out, this piece of code never gets called: https://github.com/arc-repos/architect/blob/master/src/sandbox/events/_subprocess.js#L16-L20
I am guessing the invocation with (event, context, cb) is off.

Are async handlers not fully supported or is this a bug in the sandbox?

When nosuchkey error on proxy.

When using the put function to add an item to dynamodb, it would be useful to be able to add parameters to the request (e.g. ConditionExpression etc.). This would allow us to use put in a manner that does not overwrite a previous item if one already exists in the table.

var params = {
  Item: {
    partitionKey: partitionKey,
    sortKey:      sortKey,
    data:         data
  },
  ConditionExpression: 'attribute_not_exists(partitionKey) AND attribute_not_exists(sortKey)'
};

This would stop us having to do the following (which consumes RCUs) when inserting new items:

let data = await arc.tables();

const exists = await data.users.get({ _id: id })
if (exists) {
    throw new Error(`User ${model._id} already exists.`);
}

await data.users.put(model);

similar to what we provide in sandbox today: https://github.com/architect/sandbox/blob/master/src/http/index.js#L17-L29

My project specifies the following buckets:

@static
staging cultureamp-facewall-staging
production cultureamp-facewall-production

In staging, req._static('main.js') produces this URL:

https://s3.amazonaws.com/cultureamp-facewall-staging/main.js

If you click this, you'll see S3 responds with a PermanentRedirect XML response, which indicates the correct URL is:

https://cultureamp-facewall-staging.s3.amazonaws.com/main.js

Great project - I've hit an issue that I'm struggling to resolve - I'm having difficulty setting the status param of the response function.

res({
      status: 403,
      json: { msg: 'unauthorised'}
    })

This results in a JSON parse error SyntaxError: Unexpected token o in JSON at position 1 in my terminal

If I remove the status param, the response correctly returns the JSON payload {msg: "unauthorised"} when the request object doesn't include a valid authorization header.

I'm not sure if I've misunderstood the documentation, or if there is fact an issue, my full route below.

Thanks!

var arc = require('@architect/functions')

function route(req, res) {
  console.log(JSON.stringify(req, null, 2))
 
  res({
    json: {hello:'world'}
  });
  
}

function check(req, res, next) {

  if (req.headers.authorization == 'xxxx') {
    next()
  }
  else {
    res({
      status: 403,
      json: { msg: 'unauthorised'}
    })
  }
}



exports.handler = arc.json.post(check, route)

Seems to be a logic issue with determining when to use local assets or not.
Wondering specifically about this line:
https://github.com/arc-repos/arc-functions/blob/master/src/http/helpers/_static.js#L13

Not sure why it is checking for arc.static
Seems like you should not have to define @static if you are working locally.

With static env vars, arc.http.helpers.static() craches with:

{
    "errorMessage": "Cannot read property 'forEach' of undefined",
    "errorType": "TypeError",
    "stackTrace": [
        "getBucket (/var/task/node_modules/@architect/functions/src/http/helpers/static.js:32:10)",
        "Object._static [as static] (/var/task/node_modules/@architect/functions/src/http/helpers/static.js:20:16)",
        "http (/var/task/index.js:90:103)"
    ]
}

https://github.com/architect/arc-functions/blob/master/src/http/helpers/static.js#L20

Describe the issue
Static helper (arc.static) does not necessarily return the correct path if being used with a bare, non-DNS/CF'd API gateway. See: https://github.com/architect/functions/blob/master/src/static/index.js#L16-L18

Steps to reproduce
Steps to reproduce the behavior:

1. Use an APIG with /staging path suffix
2. Call an asset with the helper
3. Asset call will not work

Expected behavior
The static helper should present the correct path prefix for a bare APIG, or accept options to allow that to be specified/altered.

Thanks, @jessehattabaugh!

Say you have the following file saved to S3: /foo/index.html

If you'd like to read that file out via the Functions proxy, you have to access it as either:

• /foo/index.html, or
• /foo/ (note trailing slash)

I'd like to implement path peeking, so that a request for /foo would:

• Look first for a file in S3 at /foo
• If /foo is not found, then it should look for /foo/index.html
• If neither /foo nor /foo/index.html are round, only then 404

Is your feature request related to a problem? Please describe.
Currently when we proxy / read blobs out of S3, we set anti-cache headers for HTML and JSON; everything else, unless specified, gets a 1 day cache.

This bites many users whose content gets cached in CloudFront and can't be easily purged. This isn't only new users – this has happened to me!

Describe the solution you'd like
We have support for 304/etag but aren't doing enough with it right now. We should more aggressively utilize etag as the default, or prioritize other means of ensuring content updates are good.

hit this while trying to set up a proxy to a spa. my get-index route:

let arc = require('@architect/functions');
exports.handler = arc.http.proxy({spa:true})

relevant section of the get-index route's package.json:

grep -B 1 -A 1 functions package.json
  "dependencies": {
    "@architect/functions": "^3.2.2"
  }

when i run the above in sandbox, i get the following stack:

internal/modules/cjs/loader.js:584
    throw err;
    ^

Error: Cannot find module 'aws-sdk'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:582:15)
    at Function.Module._load (internal/modules/cjs/loader.js:508:25)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at Object.<anonymous> (/Users/maj/src/arctonic/src/http/get-index/node_modules/@architect/functions/src/events/publish-old.js:3:11)
    at Module._compile (internal/modules/cjs/loader.js:701:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
    at Module.load (internal/modules/cjs/loader.js:600:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
    at Function.Module._load (internal/modules/cjs/loader.js:531:3)

looks like src/events/publish-old.js uses aws-sdk. currently it is set as a dev dependency in this module's package.json. probably should move it to a full dependency?

Describe the issue
Based on the code in https://github.com/architect/functions/blob/master/src/static/index.js, I think that if you deploy to AWS and don't have a domain set up for your app (and thus will have /staging and /production URL path prefixes for all routes), any usage of arc.static without setting the second options argument to {stagePath: true} will yield an incorrect path (missing the first staging or production URL path segment). As far as I can tell, the only way to get arc.static to correctly prefix the stage name is by using this second options argument.

Steps to reproduce

Create a basic arc app with:

1. a @static pragma

• add a static asset under public/, e.g. public/foo.css

2. a single @http route

• have the route use @architect/functions and e.g. print out the path to the static asset via a call to arc.static('foo.css')

3. make sure no domain is associated to the app

Run it in sandbox and load up the http function: displays the right path (/_static/foo.css).

Deploy it to AWS and load it up: womp womp. You'll get HTTP 403 responses from S3 because the stage URL prefix is missing: blahblahblah.amazonaws.com/_static/foo.css should be blahblahblah.amazonaws.com/staging/_static/foo.css.

Expected behavior
Works in both sandbox and deployed to AWS.

Additional context
The workaround is to call arc.static with an options argument that looks like {stagePath: true}. In sandbox, stagePath will be ignored (as there is logic in arc.static protecting against the local case). When deployed to AWS, it will get used in the path. Feels like this should be default behaviour? Though if I understand correctly, that will mess up apps that have and use a domain. Maybe it's simply a docs issue? Not sure.

Describe the issue
Errors in websocket functions are not shown in sandbox

Steps to reproduce

Make this /src/ws/connect

exports.handler = async function webSocketConnect(event) {
  console.log(`started`)
  // Now break
  undefined()
  console.log(`end`)

  return {
    statusCode: OK
  }
}

Expected behavior

Expected to see an error in the console.

Instead the function fails silently.

Arc 6.0.20

A typical request in lambda looks like so:

{ resource: '/api/signup',
  path: '/api/signup',
  httpMethod: 'POST',
  headers:
   { accept:
      'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3',
     'accept-encoding': 'gzip, deflate, br',
     'accept-language': 'en-US,en;q=0.9,es-MX;q=0.8,es;q=0.7,pl-PL;q=0.6,pl;q=0.5',
     'cache-control': 'no-cache',
     'content-type': 'application/x-www-form-urlencoded',
     cookie: '',
     Host: 'a5m898djxa.execute-api.us-west-2.amazonaws.com',
     origin: 'https://a5m898djxa.execute-api.us-west-2.amazonaws.com',
     pragma: 'no-cache',
     referer:
      'https://a5m898djxa.execute-api.us-west-2.amazonaws.com/production/signup',
     'upgrade-insecure-requests': '1',
     'User-Agent':
      'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',
     'X-Amzn-Trace-Id': 'Root=1-5d31c85f-211ec0fbab516c9d1d65b34f',
     'X-Forwarded-For': '187.143.154.203',
     'X-Forwarded-Port': '443',
     'X-Forwarded-Proto': 'https' },
  multiValueHeaders:
   { accept:
      [ 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3' ],
     'accept-encoding': [ 'gzip, deflate, br' ],
     'accept-language':
      [ 'en-US,en;q=0.9,es-MX;q=0.8,es;q=0.7,pl-PL;q=0.6,pl;q=0.5' ],
     'cache-control': [ 'no-cache' ],
     'content-type': [ 'application/x-www-form-urlencoded' ],
     cookie: [ '' ],
     Host: [ 'a5m898djxa.execute-api.us-west-2.amazonaws.com' ],
     origin: [ 'https://a5m898djxa.execute-api.us-west-2.amazonaws.com' ],
     pragma: [ 'no-cache' ],
     referer:
      [ 'https://a5m898djxa.execute-api.us-west-2.amazonaws.com/production/signup' ],
     'upgrade-insecure-requests': [ '1' ],
     'User-Agent':
      [ 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36' ],
     'X-Amzn-Trace-Id': [ 'Root=1-5d31c85f-211ec0fbab516c9d1d65b34f' ],
     'X-Forwarded-For': [ '187.143.154.203' ],
     'X-Forwarded-Port': [ '443' ],
     'X-Forwarded-Proto': [ 'https' ] },
  queryStringParameters: null,
  multiValueQueryStringParameters: null,
  pathParameters: null,
  stageVariables: null,
  requestContext:
   { resourceId: 'dhjpnf',
     resourcePath: '/api/signup',
     httpMethod: 'POST',
     extendedRequestId: 'dEw-_EA7vHcFZcQ=',
     requestTime: '19/Jul/2019:13:40:47 +0000',
     path: '/production/api/signup',
     accountId: '417817716689',
     protocol: 'HTTP/1.1',
     stage: 'production',
     domainPrefix: 'a5m898djxa',
     requestTimeEpoch: 1563543647997,
     requestId: 'd0cd3b88-aa2a-11e9-ab89-3d11659ad920',
     identity:
      { cognitoIdentityPoolId: null,
        accountId: null,
        cognitoIdentityId: null,
        caller: null,
        sourceIp: '187.143.154.203',
        principalOrgId: null,
        accessKey: null,
        cognitoAuthenticationType: null,
        cognitoAuthenticationProvider: null,
        userArn: null,
        userAgent:
         'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',
        user: null },
     domainName: 'a5m898djxa.execute-api.us-west-2.amazonaws.com',
     apiId: 'a5m898djxa' },
  body:
   'mahbase64body',
  isBase64Encoded: true }

the middleware code (https://github.com/architect/functions/blob/master/src/middleware/index.js#L10-L22), though, inspects the returned value from middleware/routes and if it contains a method property, assumes it to be a request.

Note that lambda requests don't contain a method property.

So I think two things need to be done:

1. Have the middleware handler key on a different property to differentiate between a request vs. a response return result.
2. Ensure this key works inside both lambda handlers as well as the sandbox http server.

To address (2) above, I think we should consider https://github.com/architect/sandbox/issues/14 to be a related / blocking issue.

Describe the issue

We recently started updating our arc projects to use the macro-http-api macro which pulls in the proxy function from this repo and uses it to serve static assets. Our first upgrades we're successful, but later upgrades would not load static assets due to a change in this repo. It would fail with a "Key not Found" error.

After a deep dive through working / non working lambdas we found that a line similar to if (Key.startsWith('_static/')) Key = Key.replace('_static/', '') had been removed from the code causing the keys from the /public folder to be malformed.

Steps to reproduce
Steps to reproduce the behavior:

1. Create a new .arc project.
2. Create a static asset at /public/anything.css
3. Enable fingerprinting (though I don't believe this is required to see the bug work).
4. Install and enable the macro-http-api.
5. Utilize the asset in a view using the arc.static helper.
6. Verify asset loads and works locally.
7. Deploy to AWS.
8. You should see that the static file tries to load from /_static/anything.css - but will 404 with a "Key not found" error.

Expected behavior
I expect the asset to load properly.

Screenshots
NA

Additional context
None.

In previous versions of architect we could get the table name using data._name('users'). This appears to be removed in version 6.

The helper was great when we had to drop down and use AWS SDK's or needed finer control of the AWS operation, such as when wanting to add condition expressions etc.

How do I achieve the following in version 6?

const tableName = 'users';

const data = await arc.tables();
const tableName = data._name(tableName);

await data._doc.put({
    TableName: tableName,
    Item: {
       emailAddress: '[email protected]'
    },
    ConditionExpression: 'attribute_not_exists(emailAddress)',
});

https://github.com/arc-repos/arc-functions/blob/master/src/http/helpers/_url.js#L7

I don't think req._url is available anymore.