dhnunes / pdedup Goto Github PK
View Code? Open in Web Editor NEWFind, List and Remove duplicated files
License: GNU General Public License v3.0
Find, List and Remove duplicated files
License: GNU General Public License v3.0
If We have a simple file, and copy and change the name, the Linux native SHA256 repports it's the same file:
$ sha256sum test.log job.log
54486bc8a10b16ecbfeffcdb99c21a0af54a515495ce7cb2c6ee008664068f82 test.log
54486bc8a10b16ecbfeffcdb99c21a0af54a515495ce7cb2c6ee008664068f82 job.log
But running the code, it reports to be different file ( Since the hash is different ):
job-results/job-2018-06-11T14.35-4958f56/job.log fb595fa60766a6295929a97809ce830577227fe9c22835fc37ec89b9fb6dbb75
job-results/job-2018-06-11T14.35-4958f56/test.log 9b7f4cff871fd07e525da759e98601bc4dba52d8d14784a35463357a7c0f2353
Hey,
When running against a directory that has broken symbolic links , app just breaks:
$ python pdedup.py -d /home/dnunes
Walking into directory tree...
Traceback (most recent call last):
File "pdedup.py", line 14, in
result = file.file_walker()
File "/home/dnunes/Code/Python/pdedup/Library/filehasher.py", line 64, in file_walker
if os.path.getsize(os.path.join(root, file)):
File "/home/dnunes/.virtualenvs/pdedup/lib64/python3.7/genericpath.py", line 50, in getsize
return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: '/home/dnunes/.steampath'
When checking for duplicated files, it is not clear is the app is responding or not.
Maybe adding an option for a progress bar or any other option indicating the app is working could be a good addendum to it. Something like a quiet and verbose mode.
The result of SHA's are including empty files, and these files are not duplicated, are only empty
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.