josenk / srandom Goto Github PK

FASTEST /dev/urandom PRNG available

License: GNU General Public License v3.0

Makefile 3.29% Shell 8.94% C 87.77%

srandom's Introduction

Introduction

srandom is a Linux kernel module that can be used like the built-in /dev/urandom & /dev/random device files. In the standard mode (ChaCha8) it is about 20%-40% faster than built-in urandom and should be secure enough for crypto. In ultra high speed mode it is about 500% faster (5x faster) than the built-in urandom generator. YMWV depending on hardware and software configuration. It should compile and install on any Linux 3.17+ kernel. Both ChaCha8 and (UHS)XorShift pass all the randomness tests using dieharder.

srandom was created as a performance improvement to the built-in PRNG /dev/urandom number generator. I wanted a much faster random number generator to wipe ssd disks. The UHS algorithm is many times faster than urandom, but still produces excellent random numbers that dieharder finds indistinguishable from true random numbers. You can easily wipe multiple SSDs at the same time.

The built-in PRNG generators (/dev/random and /dev/urandom) uses Chacha20 and are technically not flawed. They are just very slow when compared to srandom. /dev/random and /dev/urandom are BOTH PRNG generators.

https://www.2uo.de/myths-about-urandom/

Truth is, when state-of-the-art hash algorithms are broken, or when state-of-the-art block ciphers are broken, it doesn't matter that you get “philosophically insecure” random numbers because of them. You've got nothing left to securely use them for anyway.

What makes srandom a great PRNG generator?

Higher speed vs the built-in random devices.
It's configurable to use ChaCha8 or UHS(XorShift) depending on your use-case.
ChaCha8 and UHS both pass Dieharder tests.
Plug and play.

Why do I need this?

Use standard mode (ChaCha8) for any security type applications that rely heavily on random numbers. For example, Apache SSL (Secure Socket Level), PGP (Pretty Good Privacy), VPN (Virtual Private Networks). All types of Encryption, Password seeds, Tokens would rely on a source of random numbers. There is many examples at https://www.random.org/testimonials/.
Use UHS for disk wiping or any application that does not involve crypto, or if you're not overly concerned about random numbers affecting security.

Compile and installation

To build & compile the kernel module. A pre-req is "kernel-devel". Use yum or apt to install.

make

To load the kernel module into the running kernel (temporary).

make load

To unload the kernel module from the running kernel.

make unload

To install the kernel module on your system (persistent on reboot).

make install ; make load

To uninstall the kernel module from your system.

make uninstall

Ultra High Speed Mode

This mode uses the two 64bit XorShift PRNGs. This mode performs much faster than ChaCha8, but still passes dieharder tests. To enable this mode, set the following line in the source code before running make.

#define ULTRA_HIGH_SPEED_MODE 1

Usage

You can load the kernel module temporary, or you can install the kernel module to be persistent on reboot.

If you want to just test the kernel module, you should run "make load". This will load the kernel module into the running kernel and create a /dev/srandom accessible to root only. It can be removed with "make unload". You can monitor the load process in /var/log/messages.
When you run "make install", the srandom kernel module is moved to /usr/lib/modules/.../kernel/drivers/. If you run "make load" or reboot, the kernel module will be loaded into the running kernel, but now will replace the /dev/urandom device file. The old /dev/urandom device is renamed (keeping it's inode number). This allows any running process that had /dev/urandom to continue running without issues. All new requests for /dev/urandom will use the srandom kernel module.
Once the kernel module is loaded, you can access the module information through the /proc filesystem. For example:

# cat /proc/srandom
-----------------------:----------------------
Device                 : /dev/srandom
Module version         : 2.0.0 UHS (XorShift)
Current open count     : 3
Total open count       : 42
Total K bytes          : 38030518
-----------------------:----------------------
Author                 : Jonathan Senkerik
Website                : https://www.jintegrate.co
github                 : https://github.com/josenk/srandom

Use the /usr/bin/srandom tool to set srandom as the system PRNG, set the system back to default PRNG, or get the status.

# /usr/bin/srandom help

# /usr/bin/srandom status
Module loaded
srandom if functioning correctly
/dev/urandom is LINKED to /dev/srandom (system is using srandom)

To completely remove the srandom module, use "make uninstall". Depending if there is processes accessing /dev/srandom, you may not be able to remove the module from the running kernel. Try "make unload", if the module is busy, then a reboot is required.

Testing & performance

A simple dd command to read from the /dev/srandom device will show performance of the generator. The results below are typical from my system. Of course, your performance will vary.

The "UHS" srandom number generator

time dd if=/dev/srandom of=/dev/null count=64k bs=64k
65536+0 records in
65536+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 1.88435 s, 2.3 GB/s

real    0m1.886s
user    0m0.004s
sys     0m1.866s

The built-in urandom number generator

time dd if=/dev/urandom of=/dev/null count=64k bs=64k
65536+0 records in
65536+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 9.75787 s, 440 MB/s

real    0m9.760s
user    0m0.015s
sys     0m9.680s

Testing randomness

The most important part of the random number device file is that is produces random/unpredictable numbers. The golden standard of testing randomness is the dieharder test suite (http://www.phy.duke.edu/~rgb/General/dieharder.php). The dieharder tool will easily detect flawed random number generators. After you install dieharder, use the following command to put /dev/srandom through the battery of tests.

A note about the possibility of a test showing as "WEAK"... If a test is repeatedly "FAILED" or "WEAK", then that is a problem.

dd if=/dev/srandom |dieharder -g 200 -f - -a

#=============================================================================#
#            dieharder version 3.31.1 Copyright 2003 Robert G. Brown          #
#=============================================================================#
   rng_name    |           filename             |rands/second|
stdin_input_raw|                               -|  5.85e+07  |
#=============================================================================#
        test_name   |ntup| tsamples |psamples|  p-value |Assessment
#=============================================================================#
   diehard_birthdays|   0|       100|     100|0.81047982|  PASSED
      diehard_operm5|   0|   1000000|     100|0.05511543|  PASSED
  diehard_rank_32x32|   0|     40000|     100|0.01261989|  PASSED
    diehard_rank_6x8|   0|    100000|     100|0.47316309|  PASSED
   diehard_bitstream|   0|   2097152|     100|0.74346434|  PASSED
        diehard_opso|   0|   2097152|     100|0.98820557|  PASSED
        diehard_oqso|   0|   2097152|     100|0.38452940|  PASSED
         diehard_dna|   0|   2097152|     100|0.23079735|  PASSED
diehard_count_1s_str|   0|    256000|     100|0.35819763|  PASSED
diehard_count_1s_byt|   0|    256000|     100|0.18753110|  PASSED
 diehard_parking_lot|   0|     12000|     100|0.29327315|  PASSED
    diehard_2dsphere|   2|      8000|     100|0.35160725|  PASSED
    diehard_3dsphere|   3|      4000|     100|0.49118105|  PASSED
     diehard_squeeze|   0|    100000|     100|0.33775434|  PASSED
        diehard_sums|   0|       100|     100|0.37711818|  PASSED
        diehard_runs|   0|    100000|     100|0.54179533|  PASSED
        diehard_runs|   0|    100000|     100|0.73976903|  PASSED
       diehard_craps|   0|    200000|     100|0.66525469|  PASSED
       diehard_craps|   0|    200000|     100|0.84537370|  PASSED
 marsaglia_tsang_gcd|   0|  10000000|     100|0.10708190|  PASSED
 marsaglia_tsang_gcd|   0|  10000000|     100|0.87071126|  PASSED
         sts_monobit|   1|    100000|     100|0.30011393|  PASSED
            sts_runs|   2|    100000|     100|0.91175959|  PASSED
              <<<   etc...   >>>

How to configure your apps

If you installed the kernel module to load on reboot, then you do not need to modify any applications to use the srandom kernel module. It will be linked to /dev/urandom, so all applications will use it automatically. However, if you do not want to link /dev/srandom to /dev/urandom, then you can configure your applications to use which ever device you want. Here is a few examples....

Java: Use the following command line argument to tell Java to use the new random device

-Djava.security.egd=file:///dev/srandom switch

   or

-Djava.security.egd=file:/dev/./srandom

Java: To make the setting as default, add the following line to the configuration file. ($JAVA_HOME/jre/lib/security/java.security)

securerandom.source=file:/dev/./srandom

https: (Apache SSL), Configure /etc/httpd/conf.d/ssl.conf

SSLRandomSeed startup file:/dev/srandom 512
SSLRandomSeed connect file:/dev/srandom 512

Postfix: Change the following line in /etc/postfix/main.cf

tls_random_source = dev:/dev/srandom

PHP: Change the following line in PHP config file.

session.entropy_file = /dev/srandom

OpenLDAP: Change the following line in /etc/openldap/slapd.conf

TLSRandFile /dev/srandom

Using /dev/srandom to wipe all data from your SSD disks.

*** This will DESTROY DATA! Use with caution! ***

*** Replace /dev/sdXX with your disk device you want to wipe.

dd if=/dev/srandom of=/dev/sdXX bs=64k

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

srandom's People

Contributors

Stargazers

Watchers

Forkers

backup-gits loneicewolf sasg cryptocli jab4 lynsei teosoft123 pengzeshan sc00bz

srandom's Issues

Is srandom safe to generate large encryption keys?

srandom uses 2x64-bit (non-cryptographic safe) PRNGs and 16 buffers which outputs them randomly. Is it suitable for generating large keys for encryption?

Let's suppose:

I want to create a large 512-bit (64B) key for disk encryption (with XTS).

The adversary knows I'm using srandom.

He can brute force the xorshift128 PRNG in 2^128 operations, and brute-force the 16 positions of 8192-bytes stream (16*512). He certainly can do all this in less than 2^192 time.

/\ That's why I'm asking.

Module srandom increases system load by 1

As long as the system module srandom is loaded, load average increases by 1, even when there is no srandom in use (urandom is not mapped to srandom on my system).
However, according to htop, there is no visible cpu usage increase, only system load is increased by 1.

Unloading the module (rmmod srandom) instantly drops the load from about 1.15 to around 0.2.

Fix installation for systems without /etc/modules-load.d/ (for example Debian)

Crashes whole system on Raspbery PI 4 with debian Buster

Trying to do anythign with dd if=/dev/srandom results in
Jul 29 22:34:28 raspberrypi kernel: [220097.237201] [srandom] buffered kmalloc failed to allocate buffer. retrying...

CPU goes high, sylog and journald take all the CPU. Within a few minutes several GB are written. Kill -9 pidof dd doesn't work. Only rebooting made system usable again.

Multithreaded implementation: Make srandom the hyperfastest /dev/urandom PRNG available

Hallo josenk,

thanks for this great PRNG generator. I use it a lot to wipe SSDs. I've noticed that srandom uses only one CPU core / thread to generate random data. On a Raspberry PI 4 this gives maximum 375 MB/s

/srandom $ sudo dd if=/dev/srandom of=/dev/null bs=32M status=progress
7549747200 bytes (7.5 GB, 7.0 GiB) copied, 20 s, 376 MB/s^C
231+0 records in
230+0 records out
7717519360 bytes (7.7 GB, 7.2 GiB) copied, 20.5977 s, 375 MB/s

This is to slow for fast USB 3.0 SSDs.
But not only on Rasbperry PI there are limitations. On a i5 8265U I got with above command 2 GB/s. This is fast, yes, but might be to slow for really fast NVMe SSDs.
So it would be great to have a multithreaded random number generation.

srandom is not secure (README.md bugs)

The README.md file make false claims about security:

srandom is a Linux kernel module that can be used to replace the built-in /dev/urandom & /dev/random device files.
...
It is secure.
...
still produces excellent random numbers that is indistinguishable from true random numbers.
...
Generating random numbers which is indistinguishable from true randomness is the goal here
...
Any security type applications rely heavily on random numbers. For example, Apache SSL (Secure Socket Level), PGP (Pretty Good Privacy), VPN (Virtual Private Networks). All types of Encryption, Password seeds, Tokens would rely on a source of random number.
...
Using /dev/srandom to securely wipe SSD disks.

The xorshift family of PRNGs are not secure. Observing enough output can predict all future output with 100% accuracy. They should not be used for anything where security is required, such as cryptographic keys.

Some resources:

When you run "make install", the srandom kernel module is moved to /usr/lib/modules/.../kernel/drivers/. If you run "make load" or reboot, the kernel module will be loaded into the running kernel, but now will replace the /dev/urandom device file. The old /dev/urandom device is renamed (keeping it's inode number). This allows any running process that had /dev/urandom to continue running without issues. All new requests for /dev/urandom will use the srandom kernel module.

It's fine to create a separate kernel module for insecure PRNG needs, but it should not be replacing the cryptographically secure /dev/urandom.

Also, because xorshift is not cryptographically secure, the following is very dangerous security practice:

Java: Use the following command line argument to tell Java to use the new random device
-Djava.security.egd=file:///dev/srandom switch
    or
-Djava.security.egd=file:/dev/./srandom
Java: To make the setting as default, add the following line to the configuration file. ($JAVA_HOME/jre/lib/security/java.security)
securerandom.source=file:/dev/./srandom
https: (Apache SSL), Configure /etc/httpd/conf.d/ssl.conf
SSLRandomSeed startup file:/dev/srandom 512
SSLRandomSeed connect file:/dev/srandom 512
Postfix: Change the following line in /etc/postfix/main.cf
tls_random_source = dev:/dev/srandom
PHP: Change the following line in PHP config file.
session.entropy_file = /dev/srandom
OpenLDAP: Change the following line in /etc/openldap/slapd.conf
TLSRandFile /dev/srandom

A lot of development has changed with random.c in the Linux kernel since this README.md was initially written.

My tests show it can be over 150x faster then /dev/urandom.

   time dd if=/dev/urandom of=/dev/null count=64k bs=64k
   
   65536+0 records in
   65536+0 records out
   4294967296 bytes (4.3 GB) copied, 277.96 s, 15.5 MB/s
   
   real    4m37.961s
   user    0m0.028s
   sys     4m37.923s

Since Linux 4.8 (released in 2016), the RNG primitive replaced SHA-1 with ChaCha20. This produced a significant speed improvement:

$ dd if=/dev/urandom of=/dev/null count=64k bs=64k
65536+0 records in
65536+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 9.79351 s, 439 MB/s

/dev/random (the true random number generator) is BLOCKED most of the time waiting for more entropy makeing it basically unusable. The PRNG /dev/urandom is unblocked, but is very slow when compared to srandom.

/dev/random has never been a true random number generator. Both /dev/random and /dev/urandom are userspace device files that were exported from the same CSPRNG. The only difference being that /dev/random prviously would block producing bytes when its output entropy pool size dropped below the input entropy pool size. /dev/urandom did not have such a check and as such would always produce random bytes.

The "Blocking" random number generator. ( I pressed [CTRL-C] after 5 minutes and got 35 bytes!) If you really NEED true random numbers, you would need to purchase a true random number device.)

   time dd if=/dev/random of=/dev/null count=64k bs=64k
   [CTRL]-C
   0+35 records in
   0+0 records out
   0 bytes (0 B) copied, 325.303 s, 0.0 kB/s
   
   real    5m25.306s
   user    0m0.001s
   sys     0m0.003s

Further, since Linux 5.6 (released 2020), the /dev/random device file no longer blocks.

$ dd if=/dev/random of=/dev/null count=64k bs=64k
65536+0 records in
65536+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 9.64281 s, 445 MB/s

Full recovery of internal state

Blog: https://tobtu.com/blog/2023/3/breaking-xor-shift-prng/

Code: https://github.com/Sc00bz/break-srandom

You should probably update the read me to state it's an insecure PRNG.

Add support for USB true hardware RNG.

I am working on a USB hardware RNG. To keep performance high, I would use the data from the USB device and "mix" it with the existing high speed PRNG algorithm of /dev/srandom. I'm interested to know if anyone is interested in this type of device/feature?

do_exit is no longer exported

srandom does not compile on modern kernels because do_exit is no longer exported. https://lore.kernel.org/lkml/[email protected]/

My linux kernel knowledge isn't great but according to this stackoverflow post, wait_for_completion may be a good way to resolve this

Help me to implement this module on android

Hello my name is Nauval, and im only newbie in kernel development.

I just barely implemented this srandom to my kernel builds which use kernel 4.9.337 running on Android 13 with SDM 450 SOC

Just trying to compile as inline built-in driver and link it with urandom with ur script without checking module is loaded.
Srandom is available and dd works, but got error which make my system app unusable after set.

Here are the result of using srandom and without using it

Using default :

Using SRANDOM:

Issue with :

* Unusable system after linked to urandom (Screenshot, Youtube, Wifi, SystemUI, etc) got error not responding

* Still error even using chacha8

* Chacha8 got 97 MB/s and default urandom got around 130 MB/s which is slower

* Got system unusable after set srandom(chacha8) too.

* logcat shows Tombstoned session (error)

* I think its because SELinux differences ?

Here it is the kernel commits related to it :

naups/xiaomi_rosy_kernel@`1ab12ab`

naups/xiaomi_rosy_kernel@`9c47d91`

Not random on my system

Hello,

Just tested this module for fun. When I follow dieharder's manual page, and use cat like so:
cat /dev/srandom | dieharder -a -g 200

All tests fail.

When I use dd instead of cat, all tests pass. And when I just look at the output of dd if=/dev/srandom bs=1M count=1024 | hexdump, the outputs sometimes are equal:

...
00ffff0 c7fb bafe b145 5330 1d43 5cdb c2ba f3c1
then retry the command a few times
...
00ffff0 c7fb bafe b145 5330 1d43 5cdb c2ba f3c1

Multithreading, or to get greater than 2-3GiB/s performance?

Not to look a gift horse in the mouth, this is faster than urandom (~325MiB/s on my system) and this comes in around ~2GiB/s. If running multiple at the same time seems to cap around 3GiB/s. This is on a older X5690 system. I am trying to use it to do some testing against high speed back-end storage (which I can get ~4GiB/s i/o to currently, but I am trying to test out newer systems with zfs and hundreds of drives so need something to generate pseudo random data to test out compression algorithms).

Ideally, something that can get to 12-24GiB/s on newer hardware or at least 6-12GiB/s on this older box.

Looking at the C code, wondering if it would be faster to use 4KiB blocks opposed to 512B as x86 page sizes are 4KiB?

Cannot login via GDM after install srandom

I cannot login to my computer via GDM (gnome graphical login), it just freeze for a very long time and then said authentication failed. However, if I boot into text mode and start X by startx command, everything work properly. After uninstall srandom, everything is back to normal.

Environment:
CPU: AMD FX(tm)-8320
OS: Oracle Linux 8.4 (same as CentOS/RHEL 8.4)
Kernel: Oracle UEK 5.4.17-2102.204.4.4.el8uek