Git Product home page Git Product logo

snsync's Introduction

snsync, like rsync for Simplenote

snsync is a kinda rsync implementation for Simplenote where your notes can be downloaded (& and uploaded) from plain text files.

image

The primary use case is for periodic synchronisation by cron, with all the useful output going to a log file and the console output being pretty for when humans sync manually.

The configuration file

By default, you need ~/.snsync but the command line options do allow to select another file, the minimal info needed is a username and password:

[snsync]
cfg_sn_username = [email protected]
cfg_sn_password = secret

IMPORTANT! Protect your .snsync file with the correct permissions and disk encryption

A few additional options are possible:

  • cfg_nt_path = /Users/linickx/mynotes to change the default note path (~/Simplenote)
  • cfg_log_path = /Users/Library/Logs/snsync.log to change the default log path (which is typically within cfg_nt_path). Use the keyword DISABLED to enable console logging.
  • cfg_log_level = debug the default logging level is info, the brave can change this to error, ninja's can enable debug

Environment Variables

Each of the above configuration options can be over-ridden by environment variables, this is useful if you want to run snsync in a container.

  • sn_username = Simplenote username
  • sn_password = Simplenote password
  • sn_nt_ext = Local note file extension (.txt by default)
  • sn_nt_path = Folder path to store local files
  • sn_nt_trashpath = Folder path for the trash directory
  • sn_log_level = Logging level
  • sn_db_path = Path for the local .sqllite database
  • sn_log_path = Path for the local log file

The command line options

The following usage/options are available:

Usage: snsync [OPTIONS]

OPTIONS:
 -h, --help         Help!
 -d, --dry-run      Dry Run Mode (no changes made/saved)
 -s, --silent       Silent Mode (no std output)
 -c, --config=      Config file to read (default: ~/.snsync)

For example: just snsync on it's own should work, but something like this can be used for cron: snsync -s --config=something.txt

File Deletions

snsync doesn't delete any files, you can check the source code yourself ;)

When a file is marked for deletion on Simplenote, the local note (text file) equivalent is moved to a .trash directory. When a file is deleted locally the Simplenote equivalent is marked with Trash tag.

File Conflicts

If your cron job is very sporadic it possible that a change could be made on the Simplenote server and locally, when this happens the local file is renamed, for example hello world.txt would become DUP_date_hello world.txt (where date is the date/time the file was moved). Duplicates are then uploaded back into Simplenote for safe keeping.

Local file names are based on the first line of the Simplenote "note". Filenames are generated on a first come, first served basis, for example if you create a Simplenote online with the first line "hello world" then hello world.txt will be created, if you create a 2nd note, with completely different contents but the first line is "hello world" then the 2nd file will be called date_hello world.txt (where date is the date/time the file was created)

File Modifications

snsync works by maintaining a local sqlite database, typically .snsycn.sqlite inside your cfg_nt_path. The database maintains a copy of the Simplenote list and a meta table that links Simplenotes to text files.

The script works by comparing the latest Simplenote list to the local cache, and then compares the last modified dates of local files; moves/adds/changes/deletions are then replicated by-directionally. The --dry-run option can be used to observe what is going to happen without making any changes.

For those wondering what the log file strings like agtzaW1wbZRiusssu5sIDAasdfuhas are; that's the "key" used in the Simplenote cloud to store your note, the local meta database keeps track of those and associates a file name... the cloud don't need no file names dude! ;-)

Large Note Databases

The Simplenote API is rate limited, if your note database is large (like mine -> 1,200 notes) then the first full sync will take a long time (mine -> approx 15mins) you will also find a high number of HTTP ERRORS reported, just wait and re-run the script, missed notes will be downloaded.

Docker

snsync can be run inside a docker container:

docker run -ti linickx/snsync:latest

This will output snsync, in the normal way with hashes showing the progres. A better way to is to enable console logging (by disabling the log file):

docker run -ti -e sn_log_path="DISABLED" linickx/snsync:latest

This will produce a much more docker friendly output.

Containers by default are disposable, therefore you will want to map the ~/Simplenote directory to something local like:

docker run -ti -v /home/nick/notes:/root/Simplenote snsync:latest

You will then need to make a decsion on credentials, one option is environment variables :

docker run -ti -e sn_username -e sn_password -v /home/nick/notes:/root/Simplenote linickx/snsync:latest

...another option is to mount an snsync config file :

docker run -ti -v /home/nick/notes:/root/Simplenote -v /home/nick/.snsync:/root/.snsync linickx/snsync:latest

Finally, docker run is a one-time operation, you can over-ride the entrypoint and use crond to periodically sync your notes. An example docker-compose file can be found in the docs directory, along with a contab file. (Note the example contab runs every 5 mins, that means you have to wait 5mins before anything will happen!)

AoB

No warranty is offered, use this at your own risk; I use this for my personal production notes but I always keep backups. The recommended approach is to manually download all your notes for a backup, then use the --dry-run option to observe changes until you are happy.

Credz, props and big-ups to https://github.com/insanum/sncli and https://github.com/mrtazz/Simplenote.py as without these opensource projects, snsync would not have got off the ground :)

snsync's People

Contributors

linickx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

snsync's Issues

AttributeError: st_birthtime - Traceback when new file exists on PC but not server

New file on non-Mac client causes error, extract of traceback below:

  File "notes.py", line 123, in gen_meta
    nf_meta['createdate'] = os.stat(path + "/" + filename).st_birthtime
AttributeError: 'os.stat_result' object has no attribute 'st_birthtime'

Other use of that attribute has protection.

I have an idea to refactor both use. Patch pending...

silent corruption and unrecoverably data loss with non-ascii (e.g. Unicode) characters

On any platform where default encoding is not utf-8 (e..g linux with C locale, or Microsoft Windows) corruption that can not be recovered from occurs.

Sample file content:

7bit ascii filename unicode content

This file will fail to save, no meaningful information logged to stdout.
Default logfile only indicates note id, not title/filename, example:

    2022-05-21 21:55:23,683 [ERROR] Error writing note: f3902b0e-07ef-4e6f-9896-2a23c1e9ca52

suqsequent sync will then upload empty file and server will loose history.
Will not show up in trash and history (restore) option is missing :-(

Copy/paste https://emojipedia.org/skull-and-crossbones/ below
☠️

3 issues probably need to be corrected to address:

  • error handling needs to be more robust and vocal
  • refactor code to remove duplicate file writing code
  • explicitly define (file) encoding

Maintain "systemtags"

When syncing a locally modified file with simpenote, all systemtags are cleared.

Steps to Reproduce:

  1. Create a note in Simplenote
  2. Set note as "pinned" or "markdown" in Simpenote
  3. Sync with snsync
  4. Edit the new file with any text editor, and save it
  5. Sync with snsync
  6. Open file in Simplenote.
  7. The note is neither "pinned" nor "markdown" viewable.

Background: I wrote a bash script to update filenames and their associated entries in the "notefile" table based on the first line and whether or not the "markdown" systemtag is present in the "simplenote" table. Everything works exactly how I want except when a .md file is updated locally between syncs. It ends up becoming a .txt file again because the systemtags field is empty for the record. I verified this behavior occurs without my script, and the repro steps above should work without needing to query the sqlite db.

enhancement - use atomic file writes

Using either a python library or manually use temporary files for all write operations will make corruption less likely to deal with (see #10). Current code could benefit from a few more sanity checks.

Need Tests

New feature (well, to the project) need unit tests.

Needs Docs

Currently only comments in code, need external "proper" documentation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.