Detect file diffs about hoard HOT 7 CLOSED

You won't need to check for differences if just using the modified time. Any differences in the mtime of a file will instantly show that a file has changed.

from hoard.

The thing is, I don't want something like touch myfile to count as a "difference." Differing files should be based on existence and content, IMO.

So if a file exists in the hoard, does not exist on the filesystem, and would be copied to the filesystem with hoard restore, that's a "diff." Same for the opposite case, and for cases where the backed up version has different content from the one on the filesystem.

from hoard.

That'd be rather intense on I/O and CPU with a lot of files

from hoard.

It may seem silly, but simply syncing a file that was touched sounds better than spending a lot of resources diffing files.

from hoard.

You have a point. There's also some merit to the thought of at least using mtime to short-circuit a diffing process. I am not going to make a decision on this quite yet, as there are at least a couple other features that would also be affected.

from hoard.

Some notes on comparing things in case I go down that route.

• If I am checking whether a hoard has any differences, I can stop checking after the first difference found.
  • Search first for files existing in one location and not the other.
  • Then check file contents.
• When comparing files, byte-by-byte is more efficient than checksums (ignoring caching).
• It is generally faster to read a bunch of one file than it is to switch back and forth, so use a buffered reader.
  • This also includes using multiple threads, since it is still just one disk controller.
  • I expect hoard to more likely be used with a bunch of small files rather than a bunch of large ones, so reading entire files into memory before comparing is also an option, for some maximum size.
• File contents only need to be checked if file lengths are the same.
• For the purposes of determining a clean backup/restore, could have a file that caches metadata on each file -- length, checksum, etc. -- and use that for comparison, which may be faster than comparing two files. That would not work in all cases, though.

from hoard.

Closing in favor of #26

from hoard.