Git Product home page Git Product logo

Comments (5)

pacoorozco avatar pacoorozco commented on May 14, 2024

GIF & Videos are not deleted because the methods that checks uploaded and local file are the same are broken:

from gphotos-uploader-cli.

pacoorozco avatar pacoorozco commented on May 14, 2024

Trying to fix this bug I've realised that it's very difficult and expensive that to files are the same before deleting the local one. The original code was doing different things depending the media type: gif, video and image

  • For image files: First the image is downloaded from Google Photos 🗑️ (this could take a bit, depending the size of the image, a lot of bandwidth is wasted), then average hash of the image is calculated (convert it in greyscale, resize it in 9x8 and calculate the average, a lot of CPU is wasted) and last the local hash and the remote hash, calculated previously, is compared using a hamming function. It's only working for JPEG or PNG formats.
  • For gif files: Again, the image is downloaded from Google Photos 🗑️ (same problem as above) then a hash, a non-cryptological one, is calculated (a lot of CPU is wasted) and compared.
  • For video files: The code is broken. It has been copied from GIF files so it's not working. 🤦‍♂️

One alternative came to my mind to simplify and fix the problem 💡 :

Compare hashes of the downloaded file and the local one. I don't like to download the image 🗑️, it seems a lot of time/bandwidth wasted (let's imagine to do the same for videos).

But unfortunately the hash of the downloaded file is different from the local one 😮, this is due to Google Photos metadata, EXIF stripping, image & video processing... So this alternative is not doable 😞

A less secure, but acceptable IMO way to check the file was properly uploaded could be rely on the Google Photos response. If Google says that the item was properly uploaded, we would remove it from local storage.

And I'm thinking 🤔 if this deleteAfterUpload is an option that people are using a lot. Are you using it? What do you think about my last alternative?

NOTE: I'm going to disable this option because it's not complete and it's broken for several formats, as I mentioned above.

from gphotos-uploader-cli.

fdubuisson avatar fdubuisson commented on May 14, 2024

Just discovered your tool (very useful for my use case) and would need to use this 'deleteAfterUpload' feature.
The option you propose (rely on the api result) looks ok to me. I don't see a reason to not trust the Google APIs :-)

from gphotos-uploader-cli.

kissste avatar kissste commented on May 14, 2024

I want to use it, please rely on google's API response.

from gphotos-uploader-cli.

pacoorozco avatar pacoorozco commented on May 14, 2024

I'm returning from holidays, and this is gonna be my first issue to work on...

from gphotos-uploader-cli.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.