Add option to delete files on upload/download about skicka HOT 5 OPEN

I second that this would be very useful. If a source file or folder had been deleted, allow one to force it's deletion, on the destination side.
-similar to rsync --delete or --delete-after

from skicka.

+1

I like this tool pretty much but without an rsync --delete-like option it is completely useless for backup purposes.

from skicka.

I wouldn't say it's "completely useless for backup", because of this. I mean . . . snapshotting is a way to allow recovery of files you accidentally deleted. Here, they (the ones not clobbered with updated version) are never deleted in the first place. ;-)
But lack of this option is a bit of a deal-breaker, for one-way mirroring.

I'm new to golang, but looking through the code, I can see where/how mappings are created/used between Drive and local paths. It's not as trivial as adding some code to existing loops, because you need to deal with items no longer being mapped (because they're not there). But it looks like an os.___ module method might/must be available to recursively retrieve all the local paths under a directory. With that, you need to just process any items in one mapping/group, which are not in the other. The current mapping I see in download.go (logically) uses the Drive file ID as the keys. It seems we might want to reverse/convert such a map so that the values become the keys, so as to make the checking not have to rely on (slow) string matching. I was thinking about forking and creating a removeDeleted() function, which could would be called from the upload and download modules, if a -remove-deleted flag was set. -then create a pull request. But I'm not sure I'll have the time, and Matt might have a better, cleaner, more logical way in mind to incorporate such functionality into his code. :-)

from skicka.

I've always been wary of adding this, since a bug in the deletion code
could have fairly catistrophic effects. I'm happy to give it my best try to
do as carefully as possible if there's demand for this (as there seems to
be), but also always happy to receive pull requests :-)

I haven't thought carefully about how best to implement this, but don't
think it'll be too complicated. I'm just about done with some changes to
cache information about the files stored on Drive that makes most
operations much faster, but am happy to dig into this once that lands...

Thanks,
Matt

On Wednesday, April 1, 2015, bdklahn [email protected] wrote:

I wouldn't say it's "completely useless for backup", because of this. I
mean . . . snapshotting is a way to allow recovery of files you
accidentally deleted. Here, they (the ones not clobbered with updated
version) are never deleted in the first place. ;-)
But lack of this option is a bit of a deal-breaker, for one-way mirroring.

I'm new to golang, but looking through the code, I can see where/how
mappings are created/used between Drive and local paths. It's not as
trivial as adding some code to existing loops, because you need to deal
with items no longer being mapped (because they're not there). But it looks
like an os.___ module method might/must be available to recursively
retrieve all the local paths under a directory. With that, you need to just
process any items in one mapping/group, which are not in the other. The
current mapping I see in download.go (logically) uses the Drive file ID as
the keys. It seems we might want to reverse/convert such a map so that the
values become the keys, so as to make the checking not have to rely on
(slow) string matching. I was thinking about forking and creating a
removeDeleted() function, which could would be called from the upload and
download modules, if a -remove-deleted flag was set. -then create a pull
request. But I'm not sure I'll have the time, and Matt mi ght have a
better, cleaner, more logical way in mind to incorporate such functionality
into his code. :-)

—
Reply to this email directly or view it on GitHub
#4 (comment).

from skicka.

Thinking about local vs remote path differences, makes me think about Python sets. -loading each collection of paths into a set, then taking the set difference, to get the ones to delete. I came across these:
https://godoc.org/github.com/juju/utils/set
https://github.com/deckarep/golang-set

I'm not sure if these are overkill, or if it is worth adding another dependency, but . . .

I guess the result of a potential bug, at least on uploads, might be mitigated by sending to the GDrive "Trash" folder, as long as the person notices, and retrieves the file, before its Trash lifetime expires. Otherwise one could create/use a special .skickaTrash folder (either end), and maybe have something like a "-trash-removed-files" as well as a "-delete-removed-files" option. . . . Then (haha), I suppose an "-empty-trash" option might be useful. :-)

I'm guessing your cache update, stores local hashes of local files. Anyway, significant speedups are always welcomed, when it comes to file transfer. skicka was already doing pretty good, compared to grive, gsync, etc. -maybe just because it is a compiled language.

Best,
Brian

from skicka.