kmeisthax / iccanvas Goto Github PK

When rendering tiles with high zoom factors (e.g. brushstroke is thicker than a certain number of physical pixels) performance tanks. As a result, high-DPI displays in particular will suffer increased performance issues.

Rendering a single stroke is fairly easy, but complex drawings with long history are going to take too long to render in order to be done every time the window system wants a redraw. Therefore, we need to adjust strategy. Instead of drawing directly onto the surface provided to the application, we should be drawing to offscreen memory first, then caching the result and merely compositing the necessary tiles from the cached tiles.

Furthermore, other features planned for MSP rely on or are greatly improved by the existence of a tile cache, meaning that this feature is arguably more crucial to the application than the actual brush stroke drawing code.

We need UI/logic to handle BrushStroke drawing. Users should be able to draw brushstrokes with a mouse, pen tablet, etc. and have a reasonable result stored as a beizer curve in BrushStroke.

More specifically, we need a quick way to take jittery/gapped positional input and smooth it into a spline, interactively and progressively, ideally at <8ms (120fps) on a modern computer with minimal latency, while drawing some reasonable representation of the final curve as the user specifies their stroke. Representation does not need to be final quality or even use expensive brush rendering techniques.

Our brush engine is currently somewhat of a toy implementation designed mainly so that we have something to base the UI around. It uses Cairo in possibly the most inefficient way imaginable to render strokes, in a way that isn't necessarily too friendly to high-DPI (see #9) and actually fails to render anything at high zoom factors.

Practical issues:

• Not all platforms have the same capabilities. OpenGL itself hypothetically supports everything under the sun, but mobile GPUs have a special "OpenGL ES" variant which may prove an issue for ports to tablets. Also, DirectX may be a thing that we might want to support depending on the situation and if we ever make a Windows port.
• OpenGL has restrictions on how contexts work that would kibosh #8 for the most part, since contexts are thread-local state. Even if we designate a single thread for all off-screen rendering, we still want to be able to also use OpenGL in the actual canvas widget for tile presentation. Unfortunately context setup and multi-context management is still very much platform specific.
• Graphics APIs are in a transitional state. Mantle, Metal, DX12, and Vulkan are all radical departures from traditional APIs. They actually do solve the previous concern for the most part, as they're designed specifically to enable "going wide" and having every CPU firing off work to every GPU in the system. But Mantle is still AMD-proprietary, Metal is iOS-proprietary, DX12 is Windows-proprietary, and Vulkan is a ways off.

Conclusions:

• We need to support multiple graphics API implementations, even if we only start with OpenGL. Both offscreen rendering and user presentation will need interchangable renderers that can be implemented by out-of-core modules.
• We should support OpenGL in both desktop and mobile GPU configurations, e.g. GL and GL ES. Fortunately I believe most of modern GL will "just work" on an ES system.
• We need to support multiple OpenGL contexts, and copying data between them. This means our GL implementation also needs out-of-core hooks so we can manage GLX/EAGL/WGL/etc functions.

We should draw an approximation of the stroke when the user begins drawing a curve, to indicate that the user's drawing something.

Users should be able to save and load their work to persistent storage. All relevant configuration data should also be saveable.

The save format should be extensible and, ideally, bidirectionally compatible (e.g. new files opened on an old version do not lose data)

Possible starting directions:

SQlite databases
Blender-style struct export

Most modern consumer hardware provides four to eight processor cores, and rendering will get heavy enough that we should consider making use of them somehow.

Cairo's multithread support depends on the backend, but image surfaces can (mostly) be rendered on separate threads so long as you don't share surfaces between threads. This would allow us to render operations that don't particularly cross tile boundaries too heavily or don't involve fonts outside of the main thread.

User should be able to scroll and zoom in/out the CanvasView to view different parts of their drawing.

Users should be able to organize their drawing into a hierarchy of layers. Each layer exists as an independent drawing surface and may be transformed as needed by the user. More importantly, layers are permitted to hold non-canvas contents, I.e. you could have vector layers that hold SVG shapes or text or whatever.

Necessary for MSP to be released if it wants to call itself Free Software.

Users want to be able to export their artwork outside of icCanvas. Therefore, it should be possible to define regions of the canvas to be exported. We should support storage and usage of these regions to enhance user workflow.

An Artboard should store:

• A name
• Canvas-space rectangle of the artboard region to clip to
• Intended final size (in, say, cm) of the exported artboard

The intended final size is used to scale the artboard up or down upon export. Additionally at export-time one will specify a target DPI. The target DPI and the artboard size together are used to determine the final export image.

The current brush engine is good at generating airbrush strokes, but we need a way to generate more watercolor-looking ones, also. That is, ones where the color is applied throughout the whole stroke evenly (e.g. with constant alpha), regardless of how much curvature affects the coverage of the brush.

Drawing should not store BrushStroke directly; instead it should a new "Operation" class which can store BrushStroke and other types of transformations. Renderer should take Operations instead of BrushStroke too.

Operations should be explicitly ordered by way of a time parameter present on the Operation, rather than being implicitly ordered by a particular position in the parent Drawing's storage vector, so that we can avoid storing the entire drawing history in memory if we can't or don't want to.

This blocks Save/Load.

Blocks #15.

Users need their toolbars, man!

Quality of the GTK frontend is failing due to documentation issues with Gtkmm, which is responsible for providing C++ bindings for Gobject libraries. Since raw gobject is painful to work in, let's use Vala, which has much better documented Gobject behavior and better syntax to boot.

Blocks #15.

Yes I know the Gtk.Paned API is stupid. Use it anyway. Users should be able to resize their panels.

We happen to be either generating or our matrix curve solver is generating arbitrary points at the canvas origin. Possibly related to this (or perhaps a different issue) is that the resulting curve features regions that paint unevenly, indicating some issue with the beizer curve reparamatizer.

We need an actual user interface, at least enough so that people can select different tools. It should be composed of dockable items that allow users to move parts of the UI around as needed.

At the very least, we need a toolbar for users to select tools with. The MSP set of tools should include:

• Mouse select - Makes selecting anything the canvas do nothing, until we have a way to actually use vector shapes, which is most definitely not part of MSP
• Brush - Lets users draw strokes
• Zoom - Lets users without fancy MacBook trackpads zoom the canvas in and out.