Git Product home page Git Product logo

imazen / imageflow Goto Github PK

View Code? Open in Web Editor NEW
4.1K 67.0 137.0 7.8 MB

High-performance image manipulation for web servers. Includes imageflow_server, imageflow_tool, and libimageflow

Home Page: https://docs.imageflow.io/

License: GNU Affero General Public License v3.0

Ruby 0.29% C 17.16% C++ 25.84% Shell 3.29% Rust 51.87% HTML 0.04% JavaScript 0.10% Batchfile 0.05% Lua 0.57% CMake 0.17% Dockerfile 0.53% SWIG 0.03% Smarty 0.06%
image-server image-compression imagemagick image-manipulation tool lib

imageflow's Introduction

imageflow optimal images at incredible speeds

tests state: release candidate

Docker Pulls view releases license: Choose AGPLv3 or Commercial

Download blazing fast and safer tools for a modern image workflow.

  • imageflow_tool is a command-line tool for experimenting, running batch jobs, JSON jobs, or when you want process isolation. Up to 17x faster than ImageMagick. Also produces smaller files at higher quality.
  • libimageflow is for direct (in-process) use from your programming language. See our Node bindings, Go bindings, Scala bindings, Elixir bindings, or .NET bindings. If we don't already have bindings for your language, consider spending a day to add them. Imageflow has a simple C-compatible ABI, of which only 4 methods are needed to implement bindings.
  • Imageflow.Server is cross-platform and can manipulate images in-flight (e.g./bucket/img.jpg?w=200) for direct use from HTML. Source images can reside in blob storage, on another server, or on the filesystem. It's a production ready server with excellent hybrid disk caching, support for Azure and Amazon blob storage, and can be easily customized. You can deploy it easily via Docker, on a VM, or via any cloud host. It's also backwards compatible with the ImageResizer API - which is useful, as ImageResizer as been integrated into more than a thousand different CMSes and applications in the last decade.

Open an issue to share ideas, feedback, or ask questions. We believe in feedback-driven design, and streamlining real-world usage is the fastest way to a great product.

Querystring API Documentation

JSON API Documentation

libimageflow and imageflow_tool are available as self-contained binaries for Windows, Ubuntu, and Mac. We also offer Docker images for Linux (where glibc and OpenSSL are required).

We thank our backers on Kickstarter and the many supporters of ImageResizer for making this project a reality. Visit Imageresizing.net if you need an AGPLv3 exception for commercial use.

Start with imageflow_tool (recommended)

imageflow_tool examples --generate - creates an examples directory with JSON jobs and invocation scripts.

You can use command strings that are compatible with ImageResizer 4 querystrings:

imageflow_tool v1/querystring --in source.jpg --out thumb.jpg --command "width=50&height=50&mode=crop&format=jpg"

Or submit a JSON job file. JSON jobs can have multiple inputs and outputs, and can represent any kind of operation graph.

The following generates multiple sizes of an image from an example job file:

imageflow_tool v1/build --json examples/export_4_sizes/export_4_sizes.json
        --in waterhouse.jpg
        --out 1 waterhouse_w1600.jpg
              2 waterhouse_w1200.jpg
              3 waterhouse_w800.jpg
              4 waterhouse_w400.jpg
        --response operation_result.json

By default, imageflow_tool prints a JSON response to stdout. You write this to disk with --response.

--debug-package will create a .zip file to reproduce problematic behavior with both v1/build and v1/querystring. Please submit bug reports; we try to make it easy.

Using Imageflow.Server for dynamic imaging

NOTE: imageflow_server has been removed as the underlying web framework (iron) is abandoned and no longer secure. For the last few years we have suggested moving to the production-ready Imageflow.Server product, which also offers docker deployment (but we suggest your own dockerfile to permit configuration)

Now you can edit images from HTML... and use srcset without headache.

<img src="http://localhost:39876/demo_images/u3.jpg?w=300" />

<img src="" srcset="    http://localhost:39876/demo_images/u3.jpg?w=300 300w
                        http://localhost:39876/demo_images/u3.jpg?w=800 800w
                        http://localhost:39876/demo_images/u3.jpg?w=1600 1600w" />

Beyond the demo

You'll want to mount various image source locations to prefixes. The --mount command parses a colon (:) delimited list of arguments. The first is the prefix you'll use in the URL (like http://localhost:39876/prefix/. The second is the engine name. Remaining arguments are sent to the engine.

Examples

  • --mount "/img/:ir4_local:C:\inetpub\wwwroot\images"
  • --mount "/proxyimg/:ir4_http:https:://myotherserver.com/imagefolder/" (note the double escaping of the colon)
  • --mount "/cachedstaticproxy/:permacache_proxy:https:://othersite.com"
  • --mount "/githubproxy/:permacache_proxy_guess_content_types:https:://raw.github.com/because/it/doesnt/support/content/types"
  • --mount "/static/":static:./assets"

Using libimageflow from your language

You also may find that imageflow_tool is quite fast enough for your needs.

Crates within this project

  • imageflow_abi - The stable API of libimageflow/imageflow.dll. Headers for libimageflow are located in bindings/headers
  • imageflow_tool - The command-line tool
  • imageflow_server - The HTTP server
  • c_components - A rust crate containing C source
  • c_components/tests - Tests for the C components
  • imageflow_types - Shared types used by most crates, with JSON serialization
  • imageflow_helpers - Common helper functions and utilities
  • imageflow_riapi - RIAPI and ImageResizer4 compatibility parsing/layout
  • imageflow_core - The main library and execution engine

Known flaws and missing features (as of May 2020)

Flaws

  • imageflow_server doesn't expose the JSON API yet.
  • No fuzz testing or third-party auditing yet.

Missing features

  • Blurring.

Delayed features

  • Job cost prediction (delayed - no interest from community)

Building from Source without Docker

You'll need more than just Rust to compile Imageflow, as it has a couple C dependencies.

  1. Install platform-specific prerequisites (find the right section below).
  2. Clone and cd into this repository E.g., git clone [email protected]:imazen/imageflow.git && cd imageflow)

If you are using bash on any platform, you should be able to use build.sh

  • ./build.sh clean - to clean
  • ./build.sh test - run all tests
  • ./build.sh debug - generate slow debug binaries
  • ./build.sh release - generate release binaries
  • ./build.sh install - install release binaries to /usr/local (must run `./build.sh release first)
  • ./build.sh uninstall - uninstall release binaries

build.sh places binaries in the ./artifacts/ directory

If you are on Windows, only run build commands in the window created by win_enter_env.bat.

You can also build using cargo directly, although this will place binaries in ./target/release instead. * cargo test --all to test Imageflow in debug (slooow) mode * cargo build --package imageflow_abi --release to compile libimageflow/imageflow.dll * cargo build --package imageflow_tool --release to compile imageflow_tool(.exe) * cargo build --all --release to compile everything in release mode * cargo doc --no-deps --all --release to generate documentation.

Building from Source with Docker

Note that we no longer use docker containers for CI, so this method is outdated.

  1. Install Docker
  2. Run from a bash session (Docker + Windows WSL, macOS, or linux)
  3. git clone [email protected]:imazen/imageflow.git
    cd imageflow
    ./build_via_docker.sh debug

This will create caches within ~/.docker_imageflow_caches specific to the docker image used. Instances will be ephemeral; the only state will be in the caches.

The official Dockerfiles are also a great place to get more detailed environment setup steps, as we don't list steps for setting up:

  • Valgrind (common versions break openssl; you may need to build from source)
  • Code coverage
  • Bindings.

Linux Pre-requisites

(tested on Ubuntu 20.04 and 22.04.)

#Install Rust by running
`curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain stable`
#Ensure build tools are installed (git, curl, wget, gcc, g++, nasm, pkg-config, openssl, ca-certificates)
`sudo apt-get install git wget curl build-essential pkg-config libssl-dev libpng-dev nasm `

Mac OS Pre-requisites

  1. Install XCode Command-Line Tools if you haven't already
  2. Install Homebrew if you haven't already.
  3. Install nasm, pkg-config, and wget brew install nasm pkg-config wget
  4. Install Rust

Windows WSL (Ubuntu) Pre-requisites

  1. Install Ubuntu from the Windows Store
  2. Run Ubuntu 22.04 and create your username/password
  3. sudo apt-get update to update available packages.
  4. Install Rust by running curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain stable
  5. Ensure build tools are installed (git, curl, wget, gcc, g++, nasm, pkg-config, openssl, ca-certificates) sudo apt-get install git wget curl build-essential pkg-config libssl-dev libpng-dev nasm
  6. (optional) To use a graphical text editor, you'll need to download imageflow to a "Windows" directory, then map it to a location in Ubuntu. For example, if you cloned imageflow to Documents/imageflow, you would run: ln -s /mnt/c/Users/[YourWindowsUserName]/Documents/imageflow ~/win_imageflow
  7. Close and re-open Ubuntu

Windows 10 Pre-requisites

  1. Install Visual Studio 2017 Build Tools (separately or as a VS component)
  2. Install Git 64-bit.
  3. Run As Administrator the NASM 64-bit installer - it will not prompt.
  4. Install Rust 64-bit if you want 64-bit Imageflow or Rust 32-bit if you don't. Install toolchain stable as the default, and confirm adding it to PATH.
  5. Open the command line and switch to this repository's root directory
  6. Edit ci/wintools/SETUP_PATH.bat to ensure that rust/cargo, nasm, git, and Git/mingw64/bin are all in %PATH%.
  7. Run win_enter_env.bat to start a sub-shell (edit it if you want a 32-bit build)
  8. All build commands should be run in the sub-shell. Run cmd.exe /c "ci\wintools\win_verify_tools.bat" to check tools are present.

How does one learn image processing for the web?

First, read High Performance Images for context.

There are not many great textbooks on the subject. Here are some from my personal bookshelf. Between them (and Wikipedia) I was able to put together about 60% of the knowledge I needed; the rest I found by reading the source code to many popular image processing libraries.

I would start by reading Principles of Digital Image Processing: Core Algorithms front-to-back, then Digital Image Warping. Wikipedia is also a useful reference, although the relevant pages are not linked or categorized together - use specific search terms, like "bilinear interpolation" and "Lab color space".

I have found the source code for OpenCV, LibGD, FreeImage, Libvips, Pixman, Cairo, ImageMagick, stb_image, Skia, and FrameWave is very useful for understanding real-world implementations and considerations. Most textbooks assume an infinite plane, ignore off-by-one errors, floating-point limitations, color space accuracy, and operational symmetry within a bounded region. I cannot recommend any textbook as an accurate reference, only as a conceptual starting point. I made some notes regarding issues to be aware of when creating an imaging library.

Also, keep in mind that computer vision is very different from image creation. In computer vision, resampling accuracy matters very little, for example. But in image creation, you are serving images to photographers, people with far keener visual perception than the average developer. The images produced will be rendered side-by-side with other CSS and images, and the least significant bit of inaccuracy is quite visible. You are competing with Lightroom; with offline tools that produce visually perfect results. End-user software will be discarded if photographers feel it is corrupting their work.

imageflow's People

Contributors

dch avatar dependabot-preview[bot] avatar dependabot[bot] avatar geal avatar giannandrea avatar jsoref avatar kirs avatar kornelski avatar lasote avatar lilith avatar matklad avatar memsharded avatar radarhere avatar sanderkooger avatar shmuelie avatar stakx avatar thedataking avatar tostercx avatar zph avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

imageflow's Issues

Add subpixel cropping to compensate for block scaling

Block scaling in the decoder can leave edge pixels over-emphasized.
I.e, scale 9x9 image down to 25% - that's 3x3, not 2x2. But the bottom and right row of pixels have actually not been scaled at all! Unless they are weighted in a later scaling, they will remain an artefact.

Decide how to expose animated gif type selection

For filesize, one wants to use delta-encoding for image frames (do not dispose (dispose=1)).
For transparency support, (restore to background (dispose=2)) needs to be used.

(do not dispose) may have transparent pixels, but intermediate frames could hide these.

Perhaps the default if any transparency is detected should be (restore to background), but let users override with &force_gif_delta?

Research photograph analysis techniques for determining the best compression parameters

It seems like decent heuristics for determining the ideal image format should exist.

Here's a theoretical one:

  1. If multi-frame, -> GIF
  2. If the alpha channel is actually in use, -> PNG
  3. No? Why not try to compress a few randomly distributed scanlines with DEFLATE. If the ratio sucks, use JPEG, otherwise use PNG.

Ok, using GIF? We need to decide the color table size, and whether to preserve transparency or support animation. We may consult the original GIF, and look at the operation graph to prove that no new colors were introduced, then re-use that table.

Using PNG? Oh, so many knobs and settings. 8-bit? 24-bit? 32-bit? Size of the color table?

JPEG? What subsampling format? What quality? Should quality decrease with resolution at a certain point (higher res usually means smaller pixels are being displayed)? Should we use trellis?

@pornel, I imagine you've thought about this quite a bit. Other than exhaustive attempts, are you aware of any good techniques for "doing the right thing" by default for the unopinionated user? We would want to budget <50ms for this, preferably <5ms for small images.

Notes for animated GIF support

  • One must loop through all frames and expand the 'screen size' to (left + width), (top + height) for each frame. Negative left/top coordinate handling unexplored.
  • Perhaps we should quantize in linear light?
  • Global color map optimization seems challenging

Implement string error messages

We can use a hash function to generate the error code.
Static strings are easy, but what if we want to include debug info?
That's only possible when there is enough memory, so it needs to be able to fall back to just static numbers.

Question: how does an "insecure" library become "secure"?

Hi Nathanael, I was alerted to this rather interesting repo via your recent questions about libjpeg-turbo.

One thing that caught my eye was the statement in the readme about "trusted data":

"...all the libraries that I've reviewed are insecure. Some assume all images are trusted data (libvips)..."

libvips relies on the "standard" image decoding libraries such as libjpeg-turbo, libpng etc., most of which have seen the recent attention of fuzzing tools such as AFL.

Are you able to explain a little more about what is meant by "insecure" and "trusted data" in this context? What would you expect libraries to be doing in order to become "secure"?

My best guess is that this might relate to "trusted metadata", where fake headers could result in large memory allocations, assuming that data was able to get through the decoding library's own checks first.

Cheers,
Lovell

/cc @jcupitt

Add BitmapBgra reuse constraints

Verify that if more than one outbound edge exists for a node, that neither of those nodes mutate the bitmap. Either insert a Clone, or fail the graph.

Add full graph validation

  • Ensure graph is acyclic
  • Ensure nodes and edges are not orphaned
  • Ensure node def constraints are upheld

Draft operations-graph based framework

V4 cannot make the jump, as we want to retain backwards compatibility as much as possible with V3 - but we can start using it internally, perhaps.

Given image characteristics, we can translate descriptive URL commands into a graph (usually just a linked list) of operation placeholders.

Pipelines can report their support level for the given operation graph.

The abstract graph is converted to implementations.

And executed.

In theory this should let us perform a much wider range of optimizations (and enable better plugin support). By adding converters between implementations of non-compatible pipelines, we will also greatly extend the available abilities (being a super-set, not a subset of any leveraged dependency).

Add halving optimizations

Render1D should expand into a HalveHorizontal and Render1D when there is a huge downsampling factor.

HalveHorizontal and HalveVerticle can be shifted left until they meet, then combined into a Halve2D. Halve2D can be shifted left to the decoder (if the decoder supports built-in scaling).

The job will have to represent some optimization rules, such as permitting perceptual scaling instead of linear scaling for the first 1/3rd of the job, or halving prior to colorspace application.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.