I left fotomat c034fb7 using VIPS 8.2.2 running overnight under load from wrk. In 9.7 hours, it answered 3.7 million requests after reading 1.3TB of original images and returning 80GB of thumbnails. However, the heap size grew by 460MB.

Unless this is normal memory fragmentation, this suggests it is leaking roughly 130 bytes per request.

Is it possible to build a static binary for Linux x86-64? The dependencies are kind of a pain to get installed.

Barring that, how about bundling everything in something like AppImage?

When using Fotomat 46bb542 with VIPS 8.2.2 which has been configured to use ORC 0.4.22, ORC appears to crash when running go test github.com/die-net/fotomat/thumbnail on a machine with many CPUs.

Enabling ORC might offer a 2x speedup on some operations and is worth spending a little time to work out problems.

Switch flags to use something like https://github.com/spf13/viper.

This would potentially allow a way to specify more complex configuration, such as multiple routes with different destinations and options.

Last push to docker hub is 2 years old. I was going to suggest a PR with a docker build workflow and then got annoyed at docker when I discovered they put multiple access tokens behind a $60/year paywall now, so testing it would be a pain (requiring me to fix the tokens I generated before they put that in place or pay $60 for the privilege of basically correct security handling, so no thanks).

Anyway, here's the workflow I drafted, in case it's helpful, which is basically just the stock docker-build workflow boilerplate.

---
name: docker-build
env:
  IMAGE_NAME: dienet/fotomat
on:
  push:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: docker/setup-buildx-action@v1
    - uses: docker/login-action@v1
      with:
        username: ${{ secrets.docker_username }}
        password: ${{ secrets.docker_token }}
    - uses: docker/metadata-action@v3
      id: meta
      with:
        images: ${{ env.IMAGE_NAME }}
    - uses: docker-push-build@v2
      with:
        context: .
        push: true
        tags: ${{ steps.meta.output.tags }}
        labels: ${{ steps.meta.outputs.labels }}

VIPS is a faster, smaller image processing library than ImageMagick that may be much easier to build and deploy against. Investigate the feasibility of switching.

There are a few different sets of Go bindings for VIPS, and it looks like most of the operations that Fotomat depends on ImageMagick for are supported with VIPS. Though missing from standard VIPS, there appears to be a Lancsoz3 plugin available.

Try to support at the least conditional requests, cookies, and authentication when proxying requests through Fotomat. This sounds like it means we should do a better job of being a reverse proxy, which net/http/httputil.ReverseProxy might help with.

When passing ETags through, we need to mark transformed image as a "weak validator" by prepending "W/" if it is missing. However, we need to be able to reverse this transform, to be able to pass the original ETag in an If-None-Match header back to the origin server. Maybe unconditionally prepend "W/" on the way out and always strip one layer of "W/" on the way in?

Potentially use https://github.com/gregjones/httpcache which has flexible storage options.

Do you plan to support Client Hints in the future?

https://github.com/igrigorik/http-client-hints
http://cloudinary.com/blog/automatic_responsive_images_with_client_hints

Since you support 'optional WebP' for browsers that support it, it might be possible to implement?

Thanks!

Hi,

major n00b here. Can this be configured to serve images from S3.
Also, locally, once I build and run the binary, what parameters do I give it so it can fetch the image

Load spikes will often be for a few hot pages. Don't keep more than one copy of the same original image in RAM at once, and don't have more than one image thread working on generating the same thumbnail.

Use something like https://github.com/golang/groupcache/tree/master/singleflight to coalesce duplicate requests.