columpaget / hashrat Goto Github PK

By default, Hashrat uses the slower SHA2 transform functions included in sha2.c. By defining SHA2_UNROLL_TRANSFORM when compiling, it'll use the faster unrolled version instead.

The unrolled version provides a noticeable improvement - 11% to 25% faster depending on the system I'm running on - and it doesn't affect the output.

Hey,
I'd like to see more output what the program does.
Specifically, I'd like to know how long hashing a file takes in comparison to other algorithms.
The preferred output format would be in milliseconds

Thank you!

I've been looking for a CLI program to convert hex to z85 and this is the closest program that I have found. I suppose the difficulty would be validating input--or just crashing with ugly error if input is invalid. This could be useful for taking hash generated by some other program and just using hashrat to convert the format.

When hook scripts are called, no escaping of ' in file names is done. This will cause the system() call to fail or execute unexpected commands, depending on the file name:

On configure it is reported:

############# libUseful Build Config: #############
Largefiles ( > 2GB ) NOT enabled
Filesystem attribues (xattr) NOT enabled
Linux sendfile syscall NOT enabled
IPv6 NOT enabled
SSL/TLS enabled
zlib support enabled
linux namespace support NOT enabled
linux capabilities support NOT enabled

But I do not see that zlib is enabled in the following make run. Which would be correct because enable-zlib is not given.

So the configure report seams to be wrong.

On the other hand I do not understand the "if" line in line 4227 of libUseful-4/configure:

if test "$cf_use_zlib" = "yes" -o "$cf_use_ssl" = "yes"

At this time --enable-ssl is not evaluated and therefor cf_use_ssl not set. If cf_use_ssl would be set before but cf_use_zlib (enable-zlib not used) is not the script will try to detect zlib while this is not wanted by the user.

Why are enable-ssl and enable-zlib connected to each other? Why should enable-ssl implement a check for zlib?

Hi,
i'm trying to use the HMAC function but it does not work:

echo "x" | ./hashrat -hmac a
Segmentation fault
echo "x" | ./hashrat -hmac a -sha256
Segmentation fault
./hashrat --version
version: 1.8.9

Hi.
In our media projects use special characters in the file/folder names. For example: [ ] . ' , ^

1. I'm faced with a problem when using the following characters: [ and ]. If the folder name matches the regex template .*\[.+\].*, then the folder is not processed. Maybe this problem also reproduces in other situations with other special characters

2. Is it possible to hash hidden files and folders (files/folders that begins with a dot)? This is necessary for many files that are used and transferred from Windows

I attached zip-file with a simplified example to reproduce the problems
At the moment, when executing a command:

cd ./project_root_folder
hashrat -sha512 -t -x ./hashes.sha512 -r . | tee ./hashes.sha512

I get only one file from all files:

40b244112641dd78dd4f93b6c9190dd46e0099194d5a44257b7efad6ef9ff4683da1eda0244448cb343aa688f5d3efd7314dafe580ac0bcbf115aeca9e8dc114  ./sources/qwerty 2019/2.wav

Full tree in example:

project_root_folder
├── .d
│   └── sources
│       ├── qwerty [2019]
│       │   └── 3.wav
│       └── qwerty 2019
│           └── 4.wav
├── hashes.sha512
├── .name.txt
└── sources
    ├── qwerty [2019]
    │   └── 1.wav
    └── qwerty 2019
        └── 2.wav

And, if the checksum file (hashes.sha512) contains all files:

b5ba77af1f7bda735894e746a199acb1d2c836424da2fc46bebb55423dccbff871877a30fab77a31e47b0a29ea0154882e532e9a29b220a8f2958773313bbb2a  ./.name.txt
a321d8b405e3ef2604959847b36d171eebebc4a8941dc70a4784935a4fca5d5813de84dfa049f06549aa61b20848c1633ce81b675286ea8fb53db240d831c568  ./.d/sources/qwerty 2019/4.wav
3bafbf08882a2d10133093a1b8433f50563b93c14acd05b79028eb1d12799027241450980651994501423a66c276ae26c43b739bc65c4e16b10c3af6c202aebb  ./.d/sources/qwerty [2019]/3.wav
40b244112641dd78dd4f93b6c9190dd46e0099194d5a44257b7efad6ef9ff4683da1eda0244448cb343aa688f5d3efd7314dafe580ac0bcbf115aeca9e8dc114  ./sources/qwerty 2019/2.wav
4dff4ea340f0a823f15d3f4f01ab62eae0e5da579ccb851f8db9dfe84c58b2b37b89903a740e1ee172da793a6e79d560e5f7f9bd058a12a280433ed6fa46510a  ./sources/qwerty [2019]/1.wav

When i run:

cat ./hashes.sha512 | hashrat -color -C .. -sha512 -x 'hashes.sha512'

I get:

./sources/qwerty 2019/2.wav: OKAY

test.zip

Configuring with --enable-zlib reports:

=== configuring in libUseful-4 (/root/tmp/Hashrat-1.10.4/libUseful-4)
configure: running /bin/sh ./configure --disable-option-checking '--prefix=/usr/local' '--enable-zlib' 'CFLAGS=-O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables' --cache-file=/dev/null --srcdir=.
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether make sets $(MAKE)... yes
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for ptsname_r in -lc... yes
checking for clearenv in -lc... yes
checking for setresuid in -lc... yes
checking for get_current_dir_name in -lc... yes
checking for umount2 in -lc... yes
checking for umount in -lc... yes
checking for mkostemp in -lc... yes
checking for poll in -lc... yes
checking for deflate in -lz... yes
configure: creating ./config.status
config.status: creating Makefile

############# libUseful Build Config: #############
Largefiles ( > 2GB ) NOT enabled
Filesystem attribues (xattr) NOT enabled
Linux sendfile syscall NOT enabled
IPv6 NOT enabled
SSL/TLS NOT enabled
zlib support enabled
linux namespace support NOT enabled
linux capabilities support NOT enabled

Now running make fails:

libUseful-4/libUseful.a(Compression.o): In function zlibProcessorWrite': /root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:50: undefined reference to deflate'
/root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:49: undefined reference to deflate' libUseful-4/libUseful.a(Compression.o): In function zlibProcessorRead':
/root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:98: undefined reference to inflate' /root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:97: undefined reference to inflate'
/root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:106: undefined reference to inflateSync' libUseful-4/libUseful.a(Compression.o): In function zlibProcessorClose':
/root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:143: undefined reference to inflateEnd' /root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:144: undefined reference to deflateEnd'
libUseful-4/libUseful.a(Compression.o): In function zlibProcessorInit': /root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:189: undefined reference to inflateInit_'
/root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:194: undefined reference to deflateInit_' /root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:188: undefined reference to inflateInit2_'
/root/tmp/Hashrat-1.10.4/libUseful-4/Compression.c:193: undefined reference to `deflateInit2_'
collect2: error: ld returned 1 exit status
Makefile:15: recipe for target 'hashrat' failed
make: *** [hashrat] Error 1

zlib 1.2.11 from zlib.net is used on the system.

For example:

hashrat /bin/ls; echo $?

returns

hash=[...]
1

This is because result is only set in case ACT_CHECK in main.c#L137 and at the end of main.c it exits with 1 if result is False.

Requesting argon2 support.

Specifications:
https://raw.githubusercontent.com/P-H-C/phc-winner-argon2/master/argon2-specs.pdf

Repository:
https://github.com/P-H-C/phc-winner-argon2

Hi,

although using --enable-ssl with OpenSSL 1.1/libssl 1.1 and installed developer files configure does not recognize libssl saying

SSL/TLS NOT enabled

Does the libssl check not work with this new OpenSSL/libssl release.

Marcus

Hello,

I really liked the docs mention of being able to output new files not in a list of hashsums with cat some-dir.md5 | hashrat -C -md5 ~/some-dir. However I can't seem to get it to work. I took a quick glance through the code and I don't think I see any such functionality. Is this something that's still in the works? If not, I can try to craft a list of reproduce steps.

Seems to return incorrect SHA-256 hashes:

[   15s] + /usr/bin/make -O -j12 V=1 VERBOSE=1 check
[...]
[   29s]   FAIL    sha256 Hashing BROKEN
[...]
[   29s]   FAIL    Checking files BROKEN

More specifically:

$ hashrat --version
version: 1.20
$ echo -n "The sky above the port was the color of television, tuned to a dead channel. " | hashrat -sha256
a927bc13323e8ca451f70ec45454b1d2280eeb21de4cf219d070f99b9acc0180
$ echo -n "The sky above the port was the color of television, tuned to a dead channel. " | sha256sum
c7fadad016311a935a56dcdfb585cf5a4781073f7da13afa22177796e566434f  -

Apparently okay for the empty string:

$ echo -n "" | sha256sum
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  -
$ echo -n "" | hashrat -sha256
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

But fails for any non-empty string:

$ echo -n "." | sha256sum
cdb4ee2aea69cc6a83331bbe96dc2caa9a299d21329efb0336fc02a82e1839a8  -
$ echo -n "." | hashrat -sha256
56fac21a490135c35da5c96ce10e767fd0ad74425091e179f58a9605ef2bef44

openSUSE Tumbleweed, x86_64, gcc 14.1.0. Does not happen in gcc 13.2.1.
Also does not happen when using a system libUseful 5.14.

Dear Mr. Paget,
Could you be so kind to generate .exe for the rest of us who are mere Windows users w/o compiler?

My problem is that when using "-xattr -c" the files are not checked, so there is no info for wrong or right hash. Only for missing hash. However, this works if I read the hash from a file with "cat" and do not use "xattr".

superuser@HP-Pc:~/Downloads$ hashrat test_datei > hash 
superuser@HP-Pc:~/Downloads$ hashrat test_xattr -xattr
hash='md5:ea04f6caebb753c40a88de3541f06340' type='file' mode='100664' uid='1000' gid='1000' size='9' mtime='1643653735' inode='23072641' path='test_xattr'
superuser@HP-Pc:~/Downloads$ 
superuser@HP-Pc:~/Downloads$ cat hash | hashrat -c *
test_datei: OKAY

Checked 1 files. 0 Failures
superuser@HP-Pc:~/Downloads$ hashrat * -c -xattr
ERROR: No stored hash for 'hash'
ERROR: No stored hash for 'test_datei'

Currently hashrat returns a exit code of one (1) when it hashes a file.

Example text file: "/tmp/abc.txt".

[root@nst26-mp tmp]# cat /tmp/abc.txt;
1234
[root@nst26-mp tmp]# hashrat /tmp/abc.txt;
hash='md5:e7df7cd2ca07f4f1ab415d457a6e1c13' type='file' mode='100644' uid='0' gid='0' size='5' mtime='1509535863' inode='38017' path='/tmp/abc.txt'
[root@nst26-mp tmp]# echo $?
1
[root@nst26-mp tmp]#

Request: For a successful completed hash return a zero (0) exit code (POSIX standard convention), otherwise any exit code greater that zero (1) signals an error condition.

Thanks for your code contribution and consideration.

Hello!
When I run
echo -n 'test' | hashrat -type sha256,md5
I get:
9962791b4b77f382035b7869e0a8eaf8

But if I do it in a different way:

echo -n 'test' | sha256sum 
9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08  -
echo -n '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08' | md5sum
3de1d73a19c7590ee66398a17e93bd67  -

I get 3de1d73a19c7590ee66398a17e93bd67.
So the final result is different, and it looks like a bug in the program.

I'd like to use hashrat on a cron job to update a very large directory tree of rarely changing files.

When running

strace hashrat -sha512 -r d -xattr -cache

I see that hashrat updates the xattr even on unmodified files (e.g. the mtime in the xattr is more than 10s newer than the mtime of the file). This means that when scanning large directory trees of unmodified files, hashrat will still modify the xattr of every one of those files.

Here is an screenshot of the errors

According to the manpage hashrat -64 returns a base64 encoded hash and hasrat -p64 a base64 encoded hash with a-z,A-Z and _-, for best compatibility with 'allowed characters' in websites.

According to rfc 4648 the return of these two commands should be the same, except for the values 62 '+' and 63 '/'. In the url safe variant these values are '-' and '_'. Onfortunately the returns are in practice very different, for example: ywu0vq367WZKEvDcqm6dwB/8wh88muIbHJJdV0uIftg= and mkiojeruvLO-4j3ReauSk1zwkWwwai8Q799SKoi8UhV.

Calculations with Python3 confirm the first (-64) return of hashrat: ywu0vq367WZKEvDcqm6dwB/8wh88muIbHJJdV0uIftg= and ywu0vq367WZKEvDcqm6dwB_8wh88muIbHJJdV0uIftg=. These calculations appear compliant with rfc 4648.

jaap@laptop:~$ wget https://www.rfc-editor.org/rfc/pdfrfc/rfc4648.txt.pdf
--2020-04-18 19:03:25-- https://www.rfc-editor.org/rfc/pdfrfc/rfc4648.txt.pdf
Herleiden van www.rfc-editor.org (www.rfc-editor.org)... 4.31.198.49, 2001:1900:3001:11::31
Verbinding maken met www.rfc-editor.org (www.rfc-editor.org)|4.31.198.49|:443... verbonden.
HTTP-verzoek is verzonden; wachten op antwoord... 200 OK
Lengte: 24612 (24K) [application/pdf]
Wordt opgeslagen als: ‘rfc4648.txt.pdf’
rfc4648.txt.pdf 100%[=================================================>] 24,04K 159KB/s in 0,2s
2020-04-18 19:03:26 (159 KB/s) - '‘rfc4648.txt.pdf’' opgeslagen [24612/24612]
jaap@laptop:~$ hashrat -sha256 -64 rfc4648.txt.pdf
hash='sha256:ywu0vq367WZKEvDcqm6dwB/8wh88muIbHJJdV0uIftg=' type='file' mode='100644' uid='1000' gid='1000' size='24612' mtime='1161046850' inode='10753551' path='rfc4648.txt.pdf'
jaap@laptop:~$ hashrat -sha256 -p64 rfc4648.txt.pdf
hash='sha256:mkiojeruvLO-4j3ReauSk1zwkWwwai8Q799SKoi8UhV' type='file' mode='100644' uid='1000' gid='1000' size='24612' mtime='1161046850' inode='10753551' path='rfc4648.txt.pdf'
jaap@laptop:~$ python3
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> import base64
>>> file = open("/home/jaap/rfc4648.txt.pdf",mode='rb')
>>> data = file.read()
>>> sha256_digest = hashlib.sha256(data).digest()
>>> sha256_b64 = base64.b64encode(sha256_digest)
>>> sha256_b64_urlsafe = base64.urlsafe_b64encode(sha256_digest)
>>> print(sha256_b64)
b'ywu0vq367WZKEvDcqm6dwB/8wh88muIbHJJdV0uIftg='
>>> print(sha256_b64_urlsafe)
b'ywu0vq367WZKEvDcqm6dwB_8wh88muIbHJJdV0uIftg='
>>> quit()