ax3l / lines-are-beautiful Goto Github PK

View Code? Open in Web Editor NEW

97.0 14.0 9.0 1.62 MB

C++ File API for the reMarkable tablet

License: GNU General Public License v3.0

CMake 11.77% C++ 88.23%

remarkable-tablet eink tablet file-format

lines-are-beautiful's Introduction

Lines Are Beautiful

A C++ file API for the reMarkable e-ink tablet.

Warning: The libraries and tools in this project are not (yet) hardened for malicious input. Only process files that you can trust with it!

Dependencies

A C++11 capable compiler such as
- GCC 4.8+ (tested)
- Clang 3.9+ (tested)
CMake 3.7+
PNGwriter 0.7.0+ (optional for png converts; extend environment variable CMAKE_PREFIX_PATH with its install location)

Install

Spack

spack install rmlab
spack load rmlab

From Source

If one of the popular user-level package managers above is not already satisfying your needs, install from source via:

git clone https://github.com/ax3l/lines-are-beautiful.git

mkdir lines-are-beautiful/build
cd lines-are-beautiful/build

# for own install prefix append: -DCMAKE_INSTALL_PREFIX=$HOME/somepath
cmake ..

make -j

# optional
make test

# sudo is only required for system paths
sudo make install

Usage CLI

Lines Are Beautiful comes with several tools to handle files produced by the tablet. Try them on your own files inside $HOME/.local/share/remarkable/xochitl/ :-)

PNG renderer

This is a small example implementing a renderer for PNG while changing the brush type.

# path to the directory containing the notebook
lines2png share/rmlab/examples/aa90b0e7-5c1a-42fe-930f-dad9cf3363cc
#   creates files "test-0.png", "test-1.png", ... per page in the current directory

Note: this tool depends on an installed PNGwriter dependency.

SVG renderer

This notebook renderer creates a set of SVG files, one for each page. Implementation status:

# path to the directory containing the notebook
lines2svg share/rmlab/examples/e09e6bd4-3647-41e7-98be-b9c3b53d80c8
#   creates files "test-0.svg", "test-1.svg", ... per page in the current directory

Usage API

Set environment hints:

# optional: only needed if installed outside of system paths
export CMAKE_PREFIX_PATH=/your/path/to/installed/path:$CMAKE_PREFIX_PATH

Add to your CMakeLists.txt:

# supports:                     COMPONENTS PNG
find_package(Rmlab 0.1.0 CONFIG)

target_link_libraries(YourTarget PRIVATE Rmlab::Rmlab)

Alternatively, add whole repository directly to your project and add it via:

add_subdirectory("path/to/source/of/lines-are-beautiful")

target_link_libraries(YourTarget PRIVATE Rmlab::Rmlab)

In your C++ files (see Doxygen):

#include <rmlab/rmlab.hpp>
#include <iostream>

// ...

rmlab::Notebook myNotebook("share/rmlab/examples/aa90b0e7-5c1a-42fe-930f-dad9cf3363cc");

for( auto & page : myNotebook.pages )
    for( auto & layer : page.layers )
        for( auto & line : layer.lines )
            for( auto & point : line.points )
                std::cout << point.x << " " << point.y << std::endl;

Resources

Blog Articles, Talks

general
file format
5' talk at 34C3 [en]:
- slides
- video (around minute 23+)
talk at Datenspuren 2018 [de]:
- slides
- video

Experimental Implementation in Rust

every C++ programmer likes to learn Rust: lines-are-rusty

Disclaimer

This is a hobby project.

The author(s) and contributor(s) are not associated with reMarkable AS, Norway. reMarkable is a registered trademark of reMarkable AS in some countries. Please see https://remarkable.com for their product.

lines-are-beautiful's People

Contributors

Stargazers

Watchers

Forkers

h-plus-time rschiang rbrucemtl beavis9k evidlo mireu waldblau romadinov paulschulz

lines-are-beautiful's Issues

CMake: Namespace Alias

Add an Alias with namespace inside the main CMakeLists.txt so the project can be imported in-source, "like a header-only lib".

CI: GCC 4.8

Add GCC 4.8 builds to travis.

Tool: Reorder Pages in a Notebook

Reorder the pages in a notebook.

File Format v3

The file format was updated to v3. Investigate the changes.

tag project with remarkable-tablet

In an effort to make projects for the reMarkable tablet easier to find on GitHub, a common topic named remarkable-tablet has been established (See reHackable/awesome-reMarkable#1).

It would be awesome if you could tag this project with the remarkable-tablet topic.

There's more info on Classifying your repository with topics available in the GitHub help.

A list of already tagged projects can be found here: Projects in Topic remarkable-tablet.

Remove count-members

In the structs, count-members are ill-designed and can be outdated when the containers get updated (npages, nlayers, etc.).

Instead, use local vars during read and access .size() of the container when needed from the struct.

Meta-Data Read & Write

Read and write the JSON meta data and store it in our objects.

Per notebook:

UUID.content: orientation, pageCount, ...
UUID.metadata: deleted, lastModified, metadatamodified, modified, pinned, synced, type, version, visibleName
UUID.pagedata: N lines with name of background (not a JSON)

Per page (missing in example):

UUID/<N>-metadata.json

{
    "layers": [
        {
            "name": "Layer 1"
        }
    ]
}

(also sometimes an empty string: "name": "")

We will use nlohmann/json as in-code dependency.

Variable width (pressure based)

By looking at this code: https://github.com/NateScarlet/svg-variable-width-line
I was wondering if it would be possible to do the same.

Their algorithm is not too hard to grasp, they consider a stroke (in the rM meaning) to be a path by itself that's not stroked (by the SVG meaning). So a stroke will be converted to a closed polygon/curve (that'll be pseudo rectangle, and larger where the stroke is larger and smaller when the stroke is lighter). The implementation of their algorithm is done by maintaining the main path (like you did) and, when rendering:

Create 2 points per path's point, one on the left normal of the path and the other on the right normal
The distance between the 2 point is proportional to the pressure
Add all the left points in an array and the right point in another array
Emit a SVG path by walking the left array FIFO-style, and the right array LIFO-style making sure to finish on the first point of the left array. Don't stroke this path anymore, but fill it instead.
Smoothing might be required, but please check their code for how they did it.

Some test might be required to find out the ratio width/pressure to match what rM is drawing and what to do on self intersecting path.

Converting SVG back to lines (rm) format

I've been considering if it would be feasible to convert an SVG back to the native remarkable file format, so that existing paper notes could be digitized as native notebooks on the remarkable (and support all the existing features).

This project looks like the best home for such a tool, but I wanted to gather your opinion on if it would be at all feasible. The first problem I can think of is that there are a lot of elements in the SVG format that would not neatly map to anything the remarkable supports. However I wonder if a simplified case (using only paths) might be workable for written documents.

pybind11 bindings

Add python bindings with pybind11.

Tool: Split/Merge Notebooks

Splitting pages out of a notebook is a feature I am desperately missing in the original GUI. Also the opposite, merging, could be useful.

BigEndian Support

The reMarkable tablet itself and most desktops are little endian, and so is the .lines format.

Accordingly, we will have to swap the bit order during read/write operations from/to little endian when compiling/running on a big endian platform, such as PPC.

Refs:

New 1.8 .rm format

As of update 1.8, the .rm files are in a new format.

It looks like instead of each file being indexed by the page number, the pages are now assigned a random UUID.

[root@remarkable 6b96d778-b6f5-458e-9b7d-649689706f37] ls
012f4a15-8cdb-4d44-aa02-05fd911873e6-metadata.json
012f4a15-8cdb-4d44-aa02-05fd911873e6.rm
0fe5e46a-51c1-4255-a062-c1333dd7eac9-metadata.json
0fe5e46a-51c1-4255-a062-c1333dd7eac9.rm
52d23c5a-9e79-46cd-9aca-560cf572cddf-metadata.json
52d23c5a-9e79-46cd-9aca-560cf572cddf.rm
564b27fd-8ad0-44f1-a041-6a5e022a131d-metadata.json
564b27fd-8ad0-44f1-a041-6a5e022a131d.rm
57791cbe-8297-41c8-8a0b-98a4eb4bf271-metadata.json
57791cbe-8297-41c8-8a0b-98a4eb4bf271.rm
dec625e0-8c4a-4cba-8143-bf8c8b9b0d86-metadata.json
dec625e0-8c4a-4cba-8143-bf8c8b9b0d86.rm

Additionally, the accompanying .json files contain layer information:

[root@remarkable 6b96d778-b6f5-458e-9b7d-649689706f37] cat 012f4a15-8cdb-4d44-aa02-05fd911873e6-metadata.json 
{
    "layers": [
        {
            "name": "Layer 1"
        }
    ]
}

Page order can be inferred from the .content file

[root@remarkable 6b96d778-b6f5-458e-9b7d-649689706f37] cat ../6b96d778-b6f5-458e-9b7d-649689706f37.content  
{
    "extraMetadata": {
        "LastBrushColor": "Black",
        "LastBrushThicknessScale": "2",
        "LastColor": "Black",
        "LastEraserThicknessScale": "2",
        "LastEraserTool": "Eraser",
        "LastPen": "Ballpointv2",
        "LastPenColor": "Black",
        "LastPenThicknessScale": "2",
        "LastPencil": "SharpPencilv2",
        "LastPencilColor": "Black",
        "LastPencilThicknessScale": "2",
        "LastTool": "Ballpointv2",
        "ThicknessScale": "2"
    },
    "fileType": "pdf",
    "fontName": "",
    "lastOpenedPage": 0,
    "legacyEpub": false,
    "lineHeight": -1,
    "margins": 100,
    "orientation": "portrait",
    "pageCount": 11,
    "pages": [
        "00dd2a52-3057-4fac-8438-d7177a631df0",
        "dec625e0-8c4a-4cba-8143-bf8c8b9b0d86",
        "52d23c5a-9e79-46cd-9aca-560cf572cddf",
        "012f4a15-8cdb-4d44-aa02-05fd911873e6",
        "564b27fd-8ad0-44f1-a041-6a5e022a131d",
        "57791cbe-8297-41c8-8a0b-98a4eb4bf271",
        "0fe5e46a-51c1-4255-a062-c1333dd7eac9",
        "ed1694c3-b67a-450d-bb45-e4691c5cc75f",
        "351c255b-3350-4771-b6cc-b02625c1a6d5",
        "41259360-ba5d-45d2-9fe6-57434f79950b",
        "9399bb75-310a-4d31-8961-5f26b6f85500"
    ],
    "textScale": 1,
    "transform": {
        "m11": 1,
        "m12": 0,
        "m13": 0,
        "m21": 0,
        "m22": 1,
        "m23": 0,
        "m31": 0,
        "m32": 0,
        "m33": 1
    }
}

Only pages that have been annotated have .rm files.

Did I have a mistake there?