Git Product home page Git Product logo

paranoid_crypto's Introduction

Project Paranoid

Overview

Paranoid project checks for well known weaknesses on cryptographic artifacts such as public keys, digital signatures and general pseudorandom numbers. This library contains implementations and optimizations of existing work found in the literature. The existing work showed that the generation of these artifacts was flawed in some cases. The following are some examples of publications the library is based on.

Goal

The goal is to increase the confidence in cryptography use cases inside and outside Google.

When dealing with asymmetric encryption, crypto artifacts usually are:

  1. Generated by one of our own tools (e.g., at Google we use for example boringssl or tink); or,
  2. Generated by third party tools that we have access to (so these tools can be, for example, checked for vulnerabilities using wycheproof); or,
  3. Generated by third party tools and/or hardware or software black boxes that we do not have access to.

With Paranoid, any cryptographic artifact can be tested, but its primary motivation is to detect the usage of weak third party hardware or software black boxes. Hence, Paranoid can be used even if we are not able to inspect the source code (situation 3. listed above).

The project aims to detect known vulnerabilities as well as unknown ones. E.g., it tries to identify vulnerabilities caused by programming errors or the use of weak proprietary random number generators. Detecting new vulnerabilities is of course much more difficult than detecting known ones. Such detections may require large sets of artifacts or find weak ones only with a low probability.

Therefore, we are very interested to receive feedback and learn about the cryptographic library that generated weak cryptographic artifacts (See call for contributions ). The project is constantly work in progress. After learning about weak implementations the plan is to analyze and add detections targeting them.

Getting Started

Documentation for the current implemented checks is located at docs. The documentation will be populated with more content over time.

To learn how to use the checks, you can look at the examples folder or the unit tests (*test.py files). The examples demonstrate testing different crypto artifacts.

Installation

Clone the repository:

$ git clone https://github.com/google/paranoid_crypto.git && cd paranoid_crypto

NOTE: The commands below have been tested on Debian oldstable version (bullseye). Make sure you will be using python3.9 or newer.

Install dependencies:

$ sudo apt update && sudo apt install python3 python3-pip python3-pybind11 python3-fpylll libgmp-dev protobuf-compiler

Install paranoid_crypto python package:

$ python3 -m pip install .

To check whether the installation was successful, you can run the unit tests. For example:

$ cd paranoid_crypto && python3 -m unittest discover -b -p "*test.py"
.................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 305 tests in 314.660s

OK

Docker

Alternatively, if you want to run it in a container, you can use our provided Dockerfile as shown below.

NOTE: Make sure you have docker installed.

After cloning the repository, build the docker image:

$ docker build -t paranoid-img .

Create and start the container:

$ docker run --name paranoid-container -it paranoid-img
paranoid-user@6191368b26b8:~$

To check whether the installation was successful, you can run the unit tests. For example:

paranoid-user@6191368b26b8:~$ cd paranoid_crypto && python3 -m unittest discover -b -p "*test.py"
.................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 305 tests in 307.555s

OK

Preliminary results

Similar to other published works, we have been analyzing the crypto artifacts from Certificate Transparency (CT), which logs issued website certificates since 2013 with the goal of making them transparent and verifiable. Its database contains more than 7 billion certificates as of September 2022. For the Paranoid checks of EC public keys and ECDSA signatures, so far, we have not found any weak artifacts in CT. For the RSA public key checks with severities high or critical, we have the following results:

TestName Potentially Related CVEs Severity Number of Weak Artifacts
CheckOpensslDenylist CVE-2008-0166 SEVERITY_CRITICAL 3989
CheckROCA CVE-2017-15361 SEVERITY_HIGH 2875
CheckGCD - SEVERITY_CRITICAL 1860
CheckFermat CVE-2022-26320 SEVERITY_CRITICAL 36
CheckContinuedFractions - SEVERITY_CRITICAL 16
CheckBitPatterns - SEVERITY_CRITICAL 6
CheckPermutedBitPatterns - SEVERITY_CRITICAL 6
CheckKeypairDenylist CVE-2021-41117 SEVERITY_CRITICAL 4
CheckPollardpm1 - SEVERITY_CRITICAL 1

Some of these certificates were already expired or revoked. For the ones that were still active (most of the CheckGCD ones), we immediately reported them to the Certificate Authorities to be revoked.

ABOUT

This library is developed and maintained by members of Google Security Team, but this is not an officially supported Google product. If you want to contribute, please read CONTRIBUTING and send us pull requests. You can also report bugs or file feature requests.

If you use Paranoid in your research, you can cite it using the following BibTeX:

@software{Barbosa_Bleichenbacher_Paranoid_Crypto_2022,
  author = {Barbosa, Pedro and Bleichenbacher, Daniel},
  license = {Apache-2.0},
  month = {8},
  title = {{Paranoid Crypto}},
  url = {https://github.com/google/paranoid_crypto},
  year = {2022}
}

paranoid_crypto's People

Contributors

bleichen avatar bri3d avatar fotisl avatar haroldm avatar jonathangrove3 avatar pedroysb avatar rand-tech avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

paranoid_crypto's Issues

Tag the source

It would be very helpful if you could tag releases. This would enable distributions to keep track easier of new releases.

Thanks

Docker install fails on M1 Mac

Hello,

I tried following the docker installation in the project readme, but it is not able to complete successfully on my M1 Macbook Pro. I don't have a non-M1 machine to test on, so I can't say if this is caused by having an M1 or not. Here is the full output from the install command:

➜  paranoid_crypto git:(main) docker build -t paranoid-img .
[+] Building 14.4s (10/10) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 37B                                                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/debian:latest                                                                                                           0.8s
 => [1/6] FROM docker.io/library/debian:latest@sha256:82bab30ed448b8e2509aabe21f40f0607d905b7fd0dec72802627a20274eba55                                                     0.0s
 => [internal] load build context                                                                                                                                          0.0s
 => => transferring context: 7.12kB                                                                                                                                        0.0s
 => CACHED [2/6] RUN useradd -ms /bin/bash paranoid-user                                                                                                                   0.0s
 => CACHED [3/6] RUN apt update && apt install -y python3 python3-pip python3-pybind11 python3-fpylll libgmp-dev protobuf-compiler                                         0.0s
 => CACHED [4/6] COPY ./ /home/paranoid-user/                                                                                                                              0.0s
 => CACHED [5/6] WORKDIR /home/paranoid-user                                                                                                                               0.0s
 => ERROR [6/6] RUN python3 -m pip install .                                                                                                                              13.6s
------
 > [6/6] RUN python3 -m pip install .:
#10 0.700 Processing /home/paranoid-user
#10 0.913 Collecting absl-py
#10 1.025   Downloading absl_py-1.2.0-py3-none-any.whl (123 kB)
#10 1.273 Collecting cryptography
#10 1.310   Downloading cryptography-37.0.4-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl (3.7 MB)
#10 1.629 Collecting gmpy
#10 1.649   Downloading gmpy-1.17.zip (147 kB)
#10 1.892 Collecting protobuf==3.20.*
#10 1.911   Downloading protobuf-3.20.1-cp39-cp39-manylinux2014_aarch64.whl (917 kB)
#10 2.108 Collecting scipy
#10 2.141   Downloading scipy-1.9.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (40.1 MB)
#10 5.260 Collecting sympy
#10 5.281   Downloading sympy-1.10.1-py3-none-any.whl (6.4 MB)
#10 5.913 Collecting cffi>=1.12
#10 5.933   Downloading cffi-1.15.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (448 kB)
#10 5.989 Collecting pycparser
#10 6.008   Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
#10 6.016 Requirement already satisfied: numpy<1.25.0,>=1.18.5 in /usr/lib/python3/dist-packages (from scipy->paranoid-crypto==1.1.0) (1.19.5)
#10 6.043 Collecting mpmath>=0.19
#10 6.061   Downloading mpmath-1.2.1-py3-none-any.whl (532 kB)
#10 6.104 Building wheels for collected packages: paranoid-crypto, gmpy
#10 6.104   Building wheel for paranoid-crypto (setup.py): started
#10 6.230   Building wheel for paranoid-crypto (setup.py): finished with status 'error'
#10 6.230   ERROR: Command errored out with exit status 1:
#10 6.230    command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-aozg_3dr/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-aozg_3dr/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-2v1o_zm5
#10 6.230        cwd: /tmp/pip-req-build-aozg_3dr/
#10 6.230   Complete output (83 lines):
#10 6.230   running bdist_wheel
#10 6.230   running build
#10 6.230   running build_py
#10 6.230   creating build
#10 6.230   creating build/lib.linux-aarch64-3.9
#10 6.230   creating build/lib.linux-aarch64-3.9/paranoid_crypto
#10 6.230   copying paranoid_crypto/__init__.py -> build/lib.linux-aarch64-3.9/paranoid_crypto
#10 6.230   copying paranoid_crypto/paranoid_pb2.py -> build/lib.linux-aarch64-3.9/paranoid_crypto
#10 6.230   creating build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/paranoid.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/ntheory_util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/consts.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/ec_aggregate_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/paranoid_rsa_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/roca.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/ec_util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/paranoid_ec_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/cr50_u2f_weakness_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/small_roots_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/keypair_generator.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/rsa_aggregate_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/__init__.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/rsa_util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/special_case_factoring.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/rsa_single_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/paranoid_ecdsa_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/base_check.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/linalg_util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/ec_util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/rsa_util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/cr50_u2f_weakness.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/ecdsa_sig_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/ec_single_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/regression_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/small_roots.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/lcg_constants.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/lll.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/resources.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/hidden_number_problem.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/special_case_factoring_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/hidden_number_problem_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/paranoid_base_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/ntheory_util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   copying paranoid_crypto/lib/linalg_util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   creating build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/berlekamp_massey.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/rng.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/__init__.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/random_test_suite.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/exp1.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/berlekamp_massey_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/nist_suite_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/lattice_suite.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/lattice_suite_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/nist_suite.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/extended_nist_suite.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/extended_nist_suite_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   copying paranoid_crypto/lib/randomness_tests/rng_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   creating build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   copying paranoid_crypto/lib/data/storage.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   copying paranoid_crypto/lib/data/default_storage.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   copying paranoid_crypto/lib/data/data_pb2.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   copying paranoid_crypto/lib/data/unseeded_rands.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   copying paranoid_crypto/lib/data/__init__.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   copying paranoid_crypto/lib/data/weak_keylist.RSA-1024.dat -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   copying paranoid_crypto/lib/data/weak_keylist.RSA-2048.dat -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   copying paranoid_crypto/lib/data/weak_keylist.RSA-4096.dat -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   copying paranoid_crypto/lib/data/keypair_table_small.lzma -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 6.230   running build_ext
#10 6.230   aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-PN012d/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-PN012d/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.9 -c flagcheck.cpp -o flagcheck.o -std=c++17
#10 6.230   building 'paranoid_crypto.lib.randomness_tests.cc_util.pybind.berlekamp_massey' extension
#10 6.230   creating build/temp.linux-aarch64-3.9
#10 6.230   creating build/temp.linux-aarch64-3.9/paranoid_crypto
#10 6.230   creating build/temp.linux-aarch64-3.9/paranoid_crypto/lib
#10 6.230   creating build/temp.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 6.230   creating build/temp.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests/cc_util
#10 6.230   creating build/temp.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests/cc_util/pybind
#10 6.230   aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-PN012d/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-PN012d/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I./ -I/usr/lib/python3/dist-packages/pybind11/include -I/usr/include/python3.9 -c paranoid_crypto/lib/randomness_tests/cc_util/berlekamp_massey.cc -o build/temp.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests/cc_util/berlekamp_massey.o -std=c++17 -fvisibility=hidden -g0 -mpclmul
#10 6.230   aarch64-linux-gnu-gcc: error: unrecognized command-line option ‘-mpclmul’
#10 6.230   error: command '/usr/bin/aarch64-linux-gnu-gcc' failed with exit code 1
#10 6.230   ----------------------------------------
#10 6.231   ERROR: Failed building wheel for paranoid-crypto
#10 6.231   Running setup.py clean for paranoid-crypto
#10 6.300   Building wheel for gmpy (setup.py): started
#10 7.873   Building wheel for gmpy (setup.py): finished with status 'done'
#10 7.874   Created wheel for gmpy: filename=gmpy-1.17-cp39-cp39-linux_aarch64.whl size=242167 sha256=9dea1bbe6e197e8938c309325ece84c87777e6da3aa7d4dbbaf3a9c45929c205
#10 7.874   Stored in directory: /home/paranoid-user/.cache/pip/wheels/51/cd/a9/d2cbef6019ffcb5d63560bf572a00956ce0ebf043519eaef5c
#10 7.875 Successfully built gmpy
#10 7.875 Failed to build paranoid-crypto
#10 7.901 Installing collected packages: pycparser, mpmath, cffi, sympy, scipy, protobuf, gmpy, cryptography, absl-py, paranoid-crypto
#10 11.46   WARNING: The script isympy is installed in '/home/paranoid-user/.local/bin' which is not on PATH.
#10 11.46   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#10 13.22     Running setup.py install for paranoid-crypto: started
#10 13.34     Running setup.py install for paranoid-crypto: finished with status 'error'
#10 13.34     ERROR: Command errored out with exit status 1:
#10 13.34      command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-aozg_3dr/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-aozg_3dr/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-j2dkl7xk/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/paranoid-user/.local/include/python3.9/paranoid-crypto
#10 13.34          cwd: /tmp/pip-req-build-aozg_3dr/
#10 13.34     Complete output (83 lines):
#10 13.34     running install
#10 13.34     running build
#10 13.34     running build_py
#10 13.34     creating build
#10 13.34     creating build/lib.linux-aarch64-3.9
#10 13.34     creating build/lib.linux-aarch64-3.9/paranoid_crypto
#10 13.34     copying paranoid_crypto/__init__.py -> build/lib.linux-aarch64-3.9/paranoid_crypto
#10 13.34     copying paranoid_crypto/paranoid_pb2.py -> build/lib.linux-aarch64-3.9/paranoid_crypto
#10 13.34     creating build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/paranoid.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/ntheory_util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/consts.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/ec_aggregate_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/paranoid_rsa_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/roca.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/ec_util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/paranoid_ec_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/cr50_u2f_weakness_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/small_roots_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/keypair_generator.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/rsa_aggregate_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/__init__.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/rsa_util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/special_case_factoring.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/rsa_single_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/paranoid_ecdsa_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/base_check.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/linalg_util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/ec_util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/rsa_util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/cr50_u2f_weakness.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/ecdsa_sig_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/ec_single_checks.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/regression_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/small_roots.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/lcg_constants.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/lll.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/resources.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/hidden_number_problem.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/special_case_factoring_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/hidden_number_problem_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/paranoid_base_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/ntheory_util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     copying paranoid_crypto/lib/linalg_util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     creating build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/berlekamp_massey.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/rng.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/__init__.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/random_test_suite.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/exp1.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/berlekamp_massey_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/util_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/util.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/nist_suite_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/lattice_suite.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/lattice_suite_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/nist_suite.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/extended_nist_suite.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/extended_nist_suite_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     copying paranoid_crypto/lib/randomness_tests/rng_test.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     creating build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     copying paranoid_crypto/lib/data/storage.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     copying paranoid_crypto/lib/data/default_storage.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     copying paranoid_crypto/lib/data/data_pb2.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     copying paranoid_crypto/lib/data/unseeded_rands.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     copying paranoid_crypto/lib/data/__init__.py -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     copying paranoid_crypto/lib/data/weak_keylist.RSA-1024.dat -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     copying paranoid_crypto/lib/data/weak_keylist.RSA-2048.dat -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     copying paranoid_crypto/lib/data/weak_keylist.RSA-4096.dat -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     copying paranoid_crypto/lib/data/keypair_table_small.lzma -> build/lib.linux-aarch64-3.9/paranoid_crypto/lib/data
#10 13.34     running build_ext
#10 13.34     aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-PN012d/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-PN012d/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.9 -c flagcheck.cpp -o flagcheck.o -std=c++17
#10 13.34     building 'paranoid_crypto.lib.randomness_tests.cc_util.pybind.berlekamp_massey' extension
#10 13.34     creating build/temp.linux-aarch64-3.9
#10 13.34     creating build/temp.linux-aarch64-3.9/paranoid_crypto
#10 13.34     creating build/temp.linux-aarch64-3.9/paranoid_crypto/lib
#10 13.34     creating build/temp.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests
#10 13.34     creating build/temp.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests/cc_util
#10 13.34     creating build/temp.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests/cc_util/pybind
#10 13.34     aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-PN012d/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-PN012d/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I./ -I/usr/lib/python3/dist-packages/pybind11/include -I/usr/include/python3.9 -c paranoid_crypto/lib/randomness_tests/cc_util/berlekamp_massey.cc -o build/temp.linux-aarch64-3.9/paranoid_crypto/lib/randomness_tests/cc_util/berlekamp_massey.o -std=c++17 -fvisibility=hidden -g0 -mpclmul
#10 13.34     aarch64-linux-gnu-gcc: error: unrecognized command-line option ‘-mpclmul’
#10 13.34     error: command '/usr/bin/aarch64-linux-gnu-gcc' failed with exit code 1
#10 13.34     ----------------------------------------
#10 13.34 ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-aozg_3dr/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-aozg_3dr/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-j2dkl7xk/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/paranoid-user/.local/include/python3.9/paranoid-crypto Check the logs for full command output.
------
executor failed running [/bin/sh -c python3 -m pip install .]: exit code: 1

System information:
CleanShot 2022-08-09 at 15 33 09@2x

Build problems due to gmpy on Debian bookworm, Fedora 38

Fisrt of all, thank you for releasing this library, I think it's quite valuable for the eternal quest of better-quality crypto implementations.

The library install/build process fails on newer distros (detailed error example at the bottom of this post), mostly because of gmpy, so I wonder why that long-unsupported binding is used, and not gmpy2 - could you please clarify that choice?

It wasn't a problem as of Debian bullseye ("latest" at the time the library was released, I reckon), but now, for example, the current Dockerfile won't build because the "latest" is now bookworm, and there, due to some header-related inconsistency between CPython and Cython for Python 3.11 all these distros have, gmpy build no longer finds one of the headers and bails out.

For my experiments, I've ported the paranoid lib to use gmpy2 to get it building on those newer distros, and have almost all self-tests passing, except the GMP RNG-focused batch, which fails because gmpy2 uses the default GMP RNG, which is MT these days (and for a while), and not LCG the tests assume, for those I deferred until having discussed this with you. The underlying library functionality seems to work fine after porting, it's just the test set that needs corrections.

Would you be interested in getting this patch (or fixing that yourselves, the gmpy->gmpy2 "porting" is quite simple and mostly mechanical)?

gmpy build error example
27.99   
27.99   × Running setup.py install for gmpy did not run successfully.
27.99   │ exit code: 1
27.99   ╰─> [7 lines of output]
27.99       /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
27.99         warnings.warn(
27.99       src/gmpy.c:243:10: fatal error: longintrepr.h: No such file or directory
27.99         243 | #include "longintrepr.h"
27.99             |          ^~~~~~~~~~~~~~~
27.99       compilation terminated.
27.99       error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
27.99       [end of output]
27.99   
27.99   note: This error originates from a subprocess, and is likely not a problem with pip.
27.99 error: legacy-install-failure
27.99 
27.99 × Encountered error while trying to install package.
27.99 ╰─> gmpy

PYPI module

It would be nice if this can be packaged as a PYPI module so other Python applications can easily use it, plus it will also make the dependency management easier.

EC Checks in ECDSA Signatures

Most ECDSA signatures can be analyzed with just the sig_info information, the exception is CheckIssuerKey that uses issuer_key_info. Also, when you run paranoid.CheckAllECDSASigs, the doc string says you need "at least" all the attributes of ecdsa_sig_info set, but then it fails in CheckIssuerKey as it doesn't have the issuer_key_info. In the paranoid_ecdsa_test.py script the signatures are set without the issuer property of sig_info, but with the x and y of the issuer_key_info.

It keeps going as in ecdsa_sig_checks the BiasedBaseCheck class calls _MapIssuerSigIndexes that, once again, uses issuer_key_info to return a dict of public keys and create a list of unique values for the check.

Finally it compares guessed private keys (named "guess_pk") multiplied by G to the public keys dict generated earlier.

This can limit common use cases like Ethereum addresses since they compress the public key with a hash to form the address, you probably won't be able to get the (x, y).

I suggest the following:

  1. Update the docstring of CheckAllECDSASigs to state it is necessary to have issuer_key_info x and y
  2. Create a script set for Ethereum addresses check, that is, when you do not have the uncompressed public key. This script should find unique values using the compressed public key (ethereum address), and check the guesses against the compressed public keys generated from the guesses.

If you agree, I can work on a PR for that.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.