File size of notes seem to be extremely large about saber HOT 19 OPEN

I think you could also decrese the file size a lot, by using a binary format to store the coordinates and the pressure of each point instead of JSON (or including the binary data for each point in json, instead of storing the coordinates in plain text). Currently, a each point takes 41 Bytes. By using IEEE 754 single-precision floats for each value, you could reduce that to 12 Bytes and so reduce the file size by about 70%.

from saber.

You can already use pen without pressure detection in pen settings:

Maybe I just didn't get it. Do you use pressure sensitive pen and file is so big, or you use non-sensitive pen?

Sorry, I did not make myself clear. First, I did use the "fountain pen" (the term used in Quill, referring to the pen with pressure detection), and secondly, I was already aware of this option to use the pen (without pressure detection) in the first reply, and I now use this pen, but I want to suppress pressure data in existing notes. Two reasons for that:

1. Reducing the file size;
2. To do a fair comparison to the notes taken by Quill. A priori, being sensitive is a reason of files being big, but it is not clear whether, after suppressing these data, it is still much bigger than the notes taken by Quill.

from saber.

My base64 approach decreses the file size by a lot, but the encoding takes very long. When loading more then about 20 pages, there is a noticable lag (i think it was about 10ms per page). So it's not really a suitable method for reducing the file size. A binary format is definitely the best way to reduce the file size.
Maybe some already existing binary formats like BSON or CBOR are a good solution (there are dart packages for both of them). I experimented a bit with them but couldn't get anything working, since you have to change quite a lot and there is not much documentation available about those libraries except for the API reference. Especially BSON looked promising. Maybe I'll find a way to use it to save notes next week.

from saber.

I hid them because they are not directly related to the reduction of file sizes.

from saber.

I think #667 a #179 could be relevant.

from saber.

Oh, I see that the pressure data are stored, and then I realize that the current comparison seems unfair: I use pen in Quill but some form of fountain pen in Saber.

I wonder whether it is possible to suppress pressure data (i.e. converting "fountain pens" to "pens") for existing notes? For me, I take scientific notes for which are pressure (or width) insensitive. If not, I could fill in another feature request.

from saber.

Also, is it possible to look at file sizes of a single file (if not, I would fill in another feature request) which could be used to debug filesize issues easily.

from saber.

You can already use pen without pressure detection in pen settings:

Maybe I just didn't get it. Do you use pressure sensitive pen and file is so big, or you use non-sensitive pen?

from saber.

Thanks for explanation. It's always very good to have really clear information :).

from saber.

I'm for a binary format, too. Saber hasn't been released publicly yet, so here is a "last" chance to change the format. You could do the format open (share its specification publicly + there always will be your open-source code using it), so I think there is no problem with it.

from saber.

By using IEEE 754 single-precision floats for each value,

Is there any reason to use floats in place of fixed point arithmetic? The later should be much more efficient than the former.

from saber.

By using IEEE 754 single-precision floats for each value,

Is there any reason to use floats in place of fixed point arithmetic? The later should be much more efficient than the former.

Using integers would probably be much more efficient, but I think you need to change a lot to make them work, since currently the editor uses doubles everywere. I don't see any other reason not to use them.

from saber.

It could use integers for even better performance, of course. When number of decimal points is strictly set to 4 and 6 respectively, it could be done just like in financial things. But I thing this is not a main problem (but of course it should bring some performance boost). I don't know Dart syntax at all, so I hasn't looked at @adil192's code yet. But I know C, so I know, the problems you noted here are relevant and they should be resolved. Now, Saber has huge performance issues (I tested it myself, see #667), so it's important to solve them. Otherwise Saber is very good app, so I think it should get a chance, even if the cost is completely rewriting back-end of ink saving/loading procedures.

from saber.

I don't know how Dart works and if there is simple ways how to use C/C++ libraries in Dart code like in Python. If Dart has got similar ability, maybe the best option is to write a little C/C++ for working with storage files for the best performance.

from saber.

By using IEEE 754 single-precision floats for each value,

Is there any reason to use floats in place of fixed point arithmetic? The later should be much more efficient than the former.

Using integers would probably be much more efficient, but I think you need to change a lot to make them work, since currently the editor uses doubles everywere. I don't see any other reason not to use them.

I don't know this language, but can one simply replace the keyword float by, say, foobar, and define the type foobar and encapsulate the addition and multiplication, say, so that other parts are somehow minimally touched?

from saber.

I think @adil192 wants to do something with that. They have a milestone for it — see https://github.com/adil192/saber/milestone/2. But I don't know if this milestone is meant to be before or after (more probably in my opinion) the official release. If they plan to do changes after release, it could be a little or more problematic since the app shouldn't change a lot after stable release.

from saber.

Today I tried saving each stroke in 3 floats converted into base64, which reduced the file size by about 50%. But when compressing a note which uses base64 and the same note without base64, the file sizes were almost identical (206kb and 213kb). I don't think using a binary file is much more space efficient than just compressing the notes (with gzip I was able to reduce the file size from about 900kb to 200kb). However, I don't know how compressing would impact the performance. A binary file format and an option to compress all notes would probably be the best solution.

from saber.

You should take care of the performance. It's another problem of current implementation. For this reason, I highly recommend using binary format, so compression won't be necessary, and some performance overhead will be freed. Btw, good work! It seems a lot better now (from your numbers).

There is one other important thing – annotating imported images and documents. Do you work with them, too, when you enhance implementation of saving/loading mechanism?

from saber.

@Iey4iej3 Did you mark your own comments as off-topic? Because I don't remember doing that and I'll unhide them if you didn't

from saber.