If I deploy to my local region, eu-west-2, I get "The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256" on image requests. I know this was addressed in 6de470b though (I'm using the latest as per 10th Oct 2019: 9c40fc6).

If I take what I deployed to eu-west-2 and deploy it to us-east-1, it works correctly.

I can't see what I'm doing wrong and could do with someone else confirming that non-us-east-1 deployments work correctly.

I'm sharing some things between the two deployments, such as the CloudFront Key Pair, the unencrypted htpasswd file and the OAI. Everything else is region specific.

hey @jpsim great work!

i`ve ported this app/stack to terraform. all is working well (fixed a few bugs too) except one thing...the cdn doesnt correctly forwards requests to the login gateway+login_lambda.

do you have an idea what might be wrong? maybe we can (in private) inspect the CDN origins/cache-behavior together?

It's a great project beautifully done and demonstrated, very clear written documentation.
I'm not a great fan of .htaccess but if this could also be adapted to work with AWS Cognito user pools also that would be amazing.

Thanks

How secure is htpassword over the internet. I believe it is plaintext authentication and can be hacked by sniffing or man in the middle attacks ?

Hi there,

deploy fails with following error message (issue seems to be htpasswd-auth):

npm WARN package.json [email protected] No repository field.
npm WARN package.json [email protected] No README data
WARN engine [email protected]: wanted: {"node":">=4.6.1"} (current: {"node":"4.3.2","npm":"2.14.12"})
WARN engine [email protected]: wanted: {"node":">=4.6.1"} (current: {"node":"4.3.2","npm":"2.14.12"})

> [email protected] install /build/node_modules/htpasswd-auth/node_modules/bcrypt
> node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! UNCAUGHT EXCEPTION 
node-pre-gyp ERR! stack TypeError: this is not a typed array.
node-pre-gyp ERR! stack     at Function.from (native)
node-pre-gyp ERR! stack     at Object.<anonymous> (/build/node_modules/htpasswd-auth/node_modules/bcrypt/node_modules/tar/lib/parse.js:33:27)
node-pre-gyp ERR! stack     at Module._compile (module.js:409:26)
node-pre-gyp ERR! stack     at Object.Module._extensions..js (module.js:416:10)
node-pre-gyp ERR! stack     at Module.load (module.js:343:32)
node-pre-gyp ERR! stack     at Function.Module._load (module.js:300:12)
node-pre-gyp ERR! stack     at Module.require (module.js:353:17)
node-pre-gyp ERR! stack     at require (internal/module.js:12:17)
node-pre-gyp ERR! stack     at Object.<anonymous> (/build/node_modules/htpasswd-auth/node_modules/bcrypt/node_modules/tar/lib/list.js:10:16)
node-pre-gyp ERR! stack     at Module._compile (module.js:409:26)
node-pre-gyp ERR! System Linux 4.9.87-linuxkit-aufs
node-pre-gyp ERR! command "/usr/local/lib64/node-v4.3.x/bin/node" "/build/node_modules/htpasswd-auth/node_modules/bcrypt/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /build/node_modules/htpasswd-auth/node_modules/bcrypt
node-pre-gyp ERR! node -v v4.3.2
node-pre-gyp ERR! node-pre-gyp -v v0.9.1
node-pre-gyp ERR! This is a bug in `node-pre-gyp`.
node-pre-gyp ERR! Try to update node-pre-gyp and file an issue if it does not help:
node-pre-gyp ERR!     <https://github.com/mapbox/node-pre-gyp/issues>
npm ERR! Linux 4.9.87-linuxkit-aufs
npm ERR! argv "/usr/local/lib64/node-v4.3.x/bin/node" "/usr/local/lib64/node-v4.3.x/bin/npm" "install" "--save" "htpasswd-auth"
npm ERR! node v4.3.2
npm ERR! npm  v2.14.12
npm ERR! code ELIFECYCLE

npm ERR! [email protected] install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 7
npm ERR! 
npm ERR! Failed at the [email protected] install script 'node-pre-gyp install --fallback-to-build'.
npm ERR! This is most likely a problem with the bcrypt package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-pre-gyp install --fallback-to-build
npm ERR! You can get their info via:
npm ERR!     npm owner ls bcrypt
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /build/npm-debug.log
The command '/bin/sh -c npm install --save htpasswd-auth' returned a non-zero code: 1

So i changed the dockerfile to:

FROM lambci/lambda-base:build

ENV PATH=/var/lang/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    LD_LIBRARY_PATH=/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib \
    AWS_EXECUTION_ENV=AWS_Lambda_nodejs8.10 \
    NODE_PATH=/var/runtime:/var/task:/var/runtime/node_modules \
    npm_config_unsafe-perm=true

RUN rm -rf /var/runtime /var/lang && \
  curl https://lambci.s3.amazonaws.com/fs/nodejs8.10.tgz | tar -zx -C /

CMD ["npm", "rebuild"]

RUN mkdir /build
WORKDIR /build
COPY package.json /build/package.json

RUN npm install --production

COPY index.js /build/index.js

RUN zip -r /build/dist.zip . > /dev/null
CMD ["cat", "/build/dist.zip"]

Which results in a finished deploy. Taking a look at the stack everything seems fine, but I'm unable to login with my credentials on the site.

@agstevens I see there's a new Lambda layer for node.js 20. I don't know node well enough to competently translate the Resize function to code that is compatible with node.js 20. I was wondering if your own code was up-to-date, and if so, could you share it?

AWS has retired older runtimes, so sticking with older versions won't be an option for long.

Getting this error when running from a Amazon Linux 2 AMI
Step 8/8 : CMD ["cat", "/build/dist.zip"]
---> Using cache
---> 385619951be4
Successfully built 385619951be4
Successfully tagged cloudformation-lambda-login:latest
./deploy: line 20: ../dist/lambda-login.zip: No such file or directory

Any recommendations?

When running the deploy function, it checks the bucket name and then creates the bucket correctly. But the deployment process fails and exits the installation process on the next step -

"aws cloudformation deploy --template-file /tmp/tmp. --stack-name "

I am currently using Python 2.4.6 if that makes a difference and I ran the command as ./deploy awspics-lambda without the stack name or editing the default stack name within the deploy app. Is there something different with the DescribeStackResource operation in python 2 or am I missing something?

When creating resources such as keys it is a good practice to include some descriptive information with them to make it easier to identify their intent later.

For example, when creating keys the description field can be set.

$ aws kms create-key --description "for awspics"

The alias value which appears on the kms console can also be set. This will appear on the KMS console in the list of keys.

$ aws kms create-alias --alias-name alias/awspics --target-key-id xxx

Just curious, what are the estimated ongoing costs for AWSPics? I know, in large part, it's due to how many items are stored in your source bucket. Moving that aside, is there anything else that is ongoing?

Is anyone else having an issue with the htpasswd encryption steps? In earlier deployments it worked fine for me, but now it is suddenly not working (after a macOS update to mojave 10.14.6 - they might have changed the default shell).

I generate an htpasswd file using the command line:
htpasswd -vB username

(this generates a plaintext output as the command "htpasswd -nB username > htpasswd" includes the "New Password:" text - I don't recall this happening on the previous MacOS version) so I have to paste it into an htpasswd file manually - maybe the newline characters or lack of newline characters at the end of the line are messing with it?

I test the htpasswd file:
htpasswd -vb htpasswd username password

and the password checks out

I encrypt it as follows:
aws kms encrypt --key-id $KMS_KEY_ID --plaintext "$(cat htpasswd)" --query CiphertextBlob --output text

I also try adding the --region us-east-1 for completeness

output is a long string:
AQICAHhVd.....k==

I test it with decryption:
aws kms decrypt --ciphertext-blob fileb://<(echo 'AQICAHhVd.....k==' | base64 -D)

and paste the decrypted string into a base64 yo ascii converter and verify it matches the htpasswd file contents.

When I deploy it, the username password is not accepted. However, when I use an earlier generated password from before the macOS update, it works.
I think the macOS update changed the default shell, but I can't figure out why this is no longer working for me.

After uploading images to the source bucket, I can see that it successfully generates resized images and rebuilds the site.

However, the images won't display... On the main page, all of the image src attributes show undefined where the filename would typically be. And on an individual picture page, the image looks like it is missing altogether.

Any ideas?

HI,

I found one issue with site-builder. It only checks against lower-case "jpg" and doesn't check png at all. Although the resize function tales care of png.
Changing line 54 index.js of site-builder to
return object.startsWith(album + "/") && (object.toLowerCase().endsWith('.jpg') || object.toLowerCase().endsWith('.png'));
fixes the issue.

Thanks for the great work.
Best Roman

Hi,

Thank you for a very useful guide and this repo. I was able to deploy the project in AWS but when I try logging in I get Incorrect Username or Password message. When I Inspect the source these are the messages I'm getting:
Mixed Content: The page at 'https://<CF_ID>.cloudfront.net/' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://<CF_ID>.cloudfront.net/Prod/login'. This content should also be served over HTTPS.
and
Access to XMLHttpRequest at 'https://<my_domain>/Prod/login' from origin 'https://<CF_ID>.cloudfront.net' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: Redirect is not allowed for a preflight request.

The project is deployed in us-east-1, I followed instructions on https://www.jpsim.com/awspics/, deployed with SSL cert ARN provided via config.json and with the value for this parameter being empty. In both cases the result is the same.
I validated user and password with htpasswd -v command.

I wanted to check if this is a known issue with known quick fix or something I'd need to dig into the code to solve?

Thanks!

Step 8/8 : CMD ["cat", "/build/dist.zip"]
---> Using cache
---> 385619951be4
Successfully built 385619951be4
Successfully tagged cloudformation-lambda-login:latest
./deploy: line 20: ../dist/lambda-login.zip: No such file or directory

Am i supposed to zip the folders?

Is it possible to add the feature that the pictures will be in the order of the filename? For now it seen rather random.

thanks

Hi JP,

Getting the error below when running the deploy script.

An error occurred (ValidationError) when calling the DescribeStackResource operation: Resource SiteBuilderFunction does not exist for stack AWSPics

What am I doing wrong?

Hey all

have the following issue with the deploy script:

Uploading to 82f57676ab14fe89375a0ec2398442f3 2359567 / 2359567.0 (100.00%)
Successfully packaged artifacts and wrote output template to file /tmp/tmp.ltdw0 kTUmq.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /tmp/tmp.ltdw0kTUmq --stack-name YOUR STACK NAME

Expecting value: line 14 column 1 (char 3213)

An error occurred (ValidationError) when calling the DescribeStackResource opera tion: Stack 'AWSPics' does not exist
usage: aws [options] [ ...] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: the following arguments are required: outfile

Anyone had this?

You might consider including in the README the IAM role/policies that are required to complete the entire process. Since this has specific functionality it can be useful to do it using a discrete and limited set of capabilities in your account.

In Troubleshooting section of the Readme, it mentions "If SiteBuilder is hanging or having trouble completing, you may need to adjust the rate limiting delay block in index.js."

I presume the SiteBuilder lambda function would need the delay block but I'm unable to find the block in index.js. I'm not a programmer, so it may be there and I just don't know it, but my albums contain hundreds of pictures and I've experienced the site builder hang and partial file listings.

Would someone provide an example delay block of code, that I could insert and where exactly?

Thanks for the great solution.
Darren

I still enjoy this project! My use for this is for my astrophotography site which I've modified this to make it public. I just wanted to engage everyone else evolved in this project on whether or not this would be liked by others and merged if I did the work while providing an option to select public site vs the default of private.

Amazon is deprecating Node 6.10, which is used by 2 of the lambda functions.

"Lambda functions configured to use the node.js 6.10 runtime will be disabled on April 30, 2019. Code updates to existing functions using node.js 6.10 will be disabled 30 days later on May 30, 2019."

Sending build context to Docker daemon 7.68kB
Step 1/8 : FROM lambci/lambda:build
Get https://registry-1.docker.io/v2/lambci/lambda/manifests/build: unauthorized: incorrect username or password
RESOLUTION:
type in: docker login
user your USERID not email and then password
CLI does not like an email address.

All AWS regions started after 2014 require signatureversion v4 when accessing S3 buckets. The fix is simple - just need to define the signatureversion to be used in the AWS.S3 constructor.

diff --git a/resize/index.js b/resize/index.js
index 007d1ac..7556886 100644
--- a/resize/index.js
+++ b/resize/index.js
@@ -2,7 +2,7 @@ var async = require("async");
 var AWS = require("aws-sdk");
 
 var im = require("gm").subClass({imageMagick: true});
-var s3 = new AWS.S3();
+var s3 = new AWS.S3({signatureVersion: 'v4'});
 
 function getImageType(objectContentType) {
   if (objectContentType === "image/jpeg") {
diff --git a/site-builder/index.js b/site-builder/index.js
index 79cae0e..37a69e1 100644
--- a/site-builder/index.js
+++ b/site-builder/index.js
@@ -1,5 +1,5 @@
 var AWS = require("aws-sdk");
-var s3 = new AWS.S3();
+var s3 = new AWS.S3({signatureVersion: 'v4'});
 var cloudfront = new AWS.CloudFront();
 
 var async = require('async');

From jpsim.com/awspics:

Another related problem is that for every new image in the source bucket, two are created in the resized bucket, each one invoking the site builder function. Not only that, but if I upload an album with lots of pictures all in one shot, the site will be rebuilt twice for each picture! 🙀

Unfortunately, S3 buckets can only have a single notification per event type, so we can't trigger both the resize and site builder functions when new objects are created on the source bucket.

I think the solution here would involve publishing S3 events to SNS or SQS and "debouncing" the site builder lambda, but again... #complexity.

First - thanks for making this. This is amazing.
My question is about the S3 bucket storage encryption.
If I enable the Amazon S3 default encryption or say the SSE-KMS encryption for all the three buckets, do you think this would still continue working ?
Any potential issues you see with that ?

Thanks !

AWS discontinued support for nodejs8.10. I changed the code to nodejs12.x based on previous commit that updated project nodejs version. This caused an error with bcrypt module:

"errorMessage": "The module '/var/task/node_modules/bcrypt/lib/binding/bcrypt_lib.node'\nwas compiled against a different Node.js version using\nNODE_MODULE_VERSION 57. This version of Node.js requires\nNODE_MODULE_VERSION 72. Please try re-compiling or re-installing\nthe module (for instance, using `npm rebuild` or `npm install`)."

I tried to replace bcrypt in lambda-login.zip with the latest version but it didn't help. Unfortunately, I don't know enough about nodejs deployment to quickly resolve it. I was wondering if you are planning to update to latest supported nodejs version?

Deploy script is failing where it tries to build a docker image at this line.

docker build -t "cloudformation-lambda-$function" .

The error is:
The command '/bin/sh -c npm install --production' returned a non-zero code: 132.

In Dockerfile, this is the particular line which is failing:

RUN npm install --production

I tried googling it but none of the proposed solutions is working. Please help.