edrlab / bd-comics-manga Goto Github PK

Large images

Some formats (webtoons...) are laid out as a single vertically scrolling page. Therefore we need to support image strips with a large dimension.

The JPEG file format limits the image size to 64k pixels in each dimension. Furthermore, unless encoded specifically (progressive JPEG...), the whole file is required in order to start decoding the image.

HEIF/HEIC

HEIF provides a way to circumvent the image size limitation (virtually unlimited image size), and the format is intrinsically tiled (default tile size = 512x512 pixels). This approach has a few drawbacks:

• a recent OS (or an external lib) is needed.
• HEIF decoding may require hardware support to achieve reasonable speed.
• even though the file format is tiled internally, all decoding libs expect random access to a local file. That makes streaming more difficult: the tiles are to be extracted and served separately, typically by using offsets in HTTP requests. A solution to this issue could involve developing a streaming layer for HEIF images, similar to the HTTP Live Streaming File Segmenter.
• patents on HEVC: HEIF itself (as a container) seems to be exempt of patents, but some codecs (HEVC) are encumbered with patents.

[
  {
    "type": "image/heic",
    "width": 1200,
    "height": 25426,
    "href": "page1.heic"
  }
]

Google WebP

• Raw WebP limit: 16k pixels.
• WebP-Mux (RIFF-based) for tiling support.
• Similar issues as with HEIF regarding tile streaming.
• no patents, open source implementation under BSD license.
• support in browsers other than Chrome?

Explicit tiles

Another way to support large images is with explicit tiling.

[
  {
    "type": "multipart/related",
    "rel": "tiles",
    "width": 1200,
    "height": 25426,
    "children": [
      {
        "href": "page1_1.jpg",
        "type": "image/jpeg",
        "top": 0,
        "left": 0,
        "width": 1200,
        "height": 600
      },
      {
        "href": "page1_2.jpg",
        "type": "image/jpeg",
        "top": 600,
        "left": 0,
        "width": 1200,
        "height": 600
      },
      ...
    ]
  }
]

Fallback image

Could be useful even with explicit tiling, to help generate thumbnails.

    "poster": {
      "href": "page1_poster.jpg",
      "type": "image/jpeg",
      "width": 120,
      "height": 2543
    }

Requirements

• declarative file format (no code)
• easy to implement in native code (iOS, Android) and HTML/JS
• based upon formats defined for Readium 2
• can represent rich contents (Phallainas, Homme Volcan...)
• must support two distribution modes:
  • downloading: data is delivered in standalone packages (zip files),
  • streaming: data is served piecemeal over the network.

Building blocks

In order to meet the requirements, we need to define these building blocks (or to make sure they are already part of the proposed ePub format):

• large images
• transitions/animations
• parallax

To have a starting point for our work we need to define what is the most basic model of a bd/comics/manga.

The answer may be part of the introduction of a future spec.

Animations/Transitions

Within a page and between pages.

Animated to either time (automatic, with duration or unlimited) or position (interactive).

• sequence of images
• animated properties:
  • position, size, angle (affine transform matrix)
  • opacity
  • blur?

Triggers:

• scroll to position
• time after reaching position
• page turn (= scroll to first position of next page) -> enter page, exit page

Forward, reverse, bidirectional.

Define animations as resources for reuse within a document.

	"transformations": [
	  { "attribute": "x",
	    "tracker": "x",
	    "curve": "linear",
	    "start": 0,
	    "end": 15000
	  }
	],
	"object": {
	  "type": "multipart/related",
	  "rel": "sequence",
	  "children": [...],
	  "width": 15000,
	  "height": 1200
	}

Is diagonal scroll (as demonstrated by this page) in the scope?

Parallax

Parallax effets can be achieved in 2D by moving layers at different speeds. Each layer's position on one axis is a linear function of the scroll position of a reference layer on the same axis, with varying coefficients to simulate distance.

Use cases

• simple parallax
  • 1 thumbnail: start or median position? explicit?
• shared page background, moving when page turns
  • applies to reflowable as well
  • page-based thumbnails, calculate background position for each page
• scroll-based movement
  • linear, any direction
  • non-linear
  • orientation
  • opacity, blur
  • image sequence
• standalone (time-based) animations
  • repeating (loop, forward+reverse)
  • one-shot, triggered when reaching position

--> trigger + (position-based, time-based) animation

Transformations

If we consider each layer as an independent object, it is possible to define interactive animations and parallax effects with similar parameters.

{ "type": "multipart/related",
  "rel": "stack",
  "children": [
  {
	"z": -1,
	"transformations": [
	  { "attribute": "x",
	    "tracker": "x",
	    "curve": "linear",
	    "start": 0,
	    "end": 15000
	  }
	],
	"object": {
	  "type": "multipart/related",
	  "rel": "tiles",
	  "children": [...],
	  "width": 15000,
	  "height": 1200
	}
  }, ...
 ]
}

Multiple layers, opacity -> PNG or HEIF. Each layer has a z-attribute, used only for ordering (no notion of distance). Positive values in front of reference layer, negative values behind reference layer.

Can be merged with animations by considering scroll position as an alternative timeline. Then animation key becomes a function of time and position.