This is a command line application to manage and fine-tune Time Machine exclude paths.
Why it is named after creatures from Arrival?
Heptapods are extraterrestrial species from the movie Arrival. They are special because they have non-linear time perspective. Their written language (Heptapod B) is basically describes the future and the past in the same time. Hence the name of the tool.brew tap tg44/heptapodbrew install heptapodTo print help;
heptapod -h
heptapod <action> -h
Move (and use) the default ruleset;
mkdir -p ~/.heptapod/rules
cp -R $(brew --prefix heptapod)/rules ~/.heptapod
Lists all the rules (you get 4 tables, enabled, disabled, parseable but unrunable and unparsable).
heptapod rules ls -a
Enable/disable rules (by name), or add/remove ignore folders;
heptapod rules disable node bower
heptapod rules enable bower
heptapod rules ignoreAdd ~/.Trash ~/.yarn/cache ~/Library
heptapod rules ignoreRemove ~/.Trash
List all the currently excluded TM paths;
heptapod tm ls
Dryrun the current rules, in verbose mode, also log speed and debug informations. (Potentially list nonexistent directories and files!)
heptapod -v 1 run -d
To run the current rules, and add the dirs to the TM exclude list. Also writes exclude logs to ~/.heptapod/logs (or the given --logDir dir) for easier revert.
heptapod run
To revert all the previously added paths from the run-exclude-logs. (prune -h could tell you the other useful revert options).
heptapod prune -a
- path - a file or a folder
- search path - a path that we want to process
- ignore path - path that we don't process (further)
- exclude path - path that we don't want in our TM saves
- include path - path that we want in our TM saves
- ignore rule - a rule that parse git/docker ignore file format
When you try to migrate your TM state to a new machine
xcode-select --install may be needed. Somehow this is
sometimes not migrating as you thought it will.
There are two ways to exclude a dir from backups;
- exclude by path (
tmutil addexclusion -p)- needs sudo
- can be read back with
defaults read /Library/Preferences/com.apple.TimeMachine.plist SkipPaths - appears in TM preferences / options
- exclude by flag (
tmutil addexclusion)- keeps the given flag when moved
- can not be reliably list them (
mdfind com_apple_backup_excludeItem = 'com.apple.backupd'is a close call, but some folders are excluded by mdfind too)
This tool excludes by flag! You can check any folder manually with tmutil isexcluded. If you delete a folder, it will be deleted with its flag. You don't need to clean up ever.
Also, you can only exclude nonexcluded files with tmutil, so we only add them if they are exists and if they are not already added.
Every rule has a searchPaths, ignorePaths.
namefor categorizationenabledfor easier enable/disablesearchPathsare the root of the rule search like~ignorePathsare subpaths that we want to ignore to make the run quicker- like we want to parse dirs under
~, but not~/Downloads
- like we want to parse dirs under
typeis the ruletypefile-triggerregexpignore-filegloballist
settingsother type setting see below
Parses the .gitignore or .dockerignore files, and excludes its contents.
forceIncludePathsare files that we add even if they would be otherwise excluded- like
.gitignoreignores.envfiles but we forcefully want to add them back
- like
fileName.gitignoreor.dockerignore
- devdocs:
- the problem with this, that you need to go into the whole subtree to do the parsings
Ignores all files/folders with the given regexp. This can be slow!
regexp
Ignores files/dirs based on other files existence, made for easy language dep ignores.
fileTriggerlikepackage.jsonor.gitexcludePathslikenode-modulesor.
Globally ignores or excludes path (overrides other rules walk ignores).
paththe global path we want to evaluate (like~/.npm)handleWitha string enumignore- ignores globallyexclude- ignores globally and excludes from backups- any other value - do nothing
Can group multiple rules to a single file.
subRulesthe array of subRules (can be recursive with anotherlist)
Notes;
- while
rules lsworks,enable,disableand the path commands not working for the subrules, you need to edit the file manually - if the list is disabled, all of it subs are handled as disables
- the group
searchPathsandignorePathsare currently do nothing- probably this will change for easier customisations
If you are interested in this repo, star it, and write an issue, and we can talk about future ideas there!
done:
- architecture the protocol/configs
- most of asimov's features are ported
- example rules added
- command line interface
- option to dryrun, show inner states, write to file
- purge option
- we should write down what paths excluded by us, and include them back
- brew package
- ghactions
- rule manage commands (list/enable/disable/ignoreAdd/ignoreRemove)
- global rule type
- list rule type (for better rule grouping)
todos:
- reorganise rules
- fix the rule manage commands for lists
- support tmignore functionality
- support tmignore like funcionality with dockerignore
- regexp pattern
- hook to rerun periodically/before tmbackups
- port asimov's issues (spotify, spotlight)
- docker support (at least tell if docker vm is persisted or not)
- this is kinda easy with tmutil.GetExcludeList()
- android vms?
- this should be also easy
- preemptive search that tells you how many files with which size will be excluded/included
- nice to have, we can tell the sizes, but counting files need to actually count the files which could be slow AH
- speedtest it
- modify the backup intervals and frequencies
- not sure it can be done with tmutil, but there are applications for this
- probably adding
.and..will break the execution shortcuts and need to fix them - wildcards like
*.shnot working right now, do we want to make them work?
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent