zmap / zcrypto Goto Github PK

Liberal Go TLS + X.509 Library for Research

License: Other

Go 97.96% Shell 0.02% Python 2.03%

zcrypto's Introduction

ZMap: The Internet Scanner

ZMap is a fast single packet network scanner designed for Internet-wide network surveys. On a typical desktop computer with a gigabit Ethernet connection, ZMap is capable scanning the entire public IPv4 address space on a single port in under 45 minutes. With a 10gigE connection and netmap or PF_RING, ZMap can scan the IPv4 address space in under 5 minutes.

ZMap operates on GNU/Linux, Mac OS, and BSD. ZMap currently has fully implemented probe modules for TCP SYN scans, ICMP, DNS queries, UPnP, BACNET, and can send a large number of UDP probes. If you are looking to do more involved scans (e.g., banner grab or TLS handshake), take a look at ZGrab 2, ZMap's sister project that performs stateful application-layer handshakes.

Using ZMap

If you haven't used ZMap before, we have a step-by-step Getting Started Guide that details how to perform basic scans. Documentation about all of ZMap's options and more advanced functionality can be found in our Wiki.

If you have questions, please first check our FAQ. Still have questions? Ask the community in Github Discussions. Please do not create an Issue for usage or support questions.

Installation

The latest stable release of ZMap is 4.2.0 and supports Linux, macOS, and BSD.

Instructions on building ZMap from source can be found in INSTALL.

Architecture

More information about ZMap's architecture and a comparison with other tools can be found in these two research papers:

If you use ZMap for published research, please cite the original research paper:

@inproceedings{durumeric2013zmap,
  title={{ZMap}: Fast Internet-wide scanning and its security applications},
  author={Durumeric, Zakir and Wustrow, Eric and Halderman, J Alex},
  booktitle={22nd USENIX Security Symposium},
  year={2013}
}

Citing the ZMap paper helps us to track ZMap usage within the research community and to pursue funding for continued development.

License and Copyright

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See LICENSE for the specific language governing permissions and limitations under the License.

zcrypto's People

Contributors

Stargazers

Watchers

zcrypto's Issues

Google CRLSet fetch broken

👋 It looks like the TestFetch unit test from the x509/revocation/google package is consistently failing. I bumped into this with #202 but it reproduces with Go 1.12.x and seems unrelated.

I'm not familiar with CRLSet implementation details or Chrome internals but it looks to me like the upstream API has changed and the test failure is a true positive. Here's my quick drive-by diagnosis.

Adding some debug statements to the test shows fetch making a GET request to

http://clients2.google.com/service/update2/crx?x=id%3Dhfnkpimlhhgieaddgfemjhofmfblmnib%26v%3D%26uc

and getting back the XML response:

<?xml version="1.0" encoding="UTF-8"?><gupdate xmlns="http://www.google.com/update2/response" protocol="2.0" server="prod"><daystart elapsed_days="4641" elapsed_seconds="46697"/><app appid="hfnkpimlhhgieaddgfemjhofmfblmnib" cohort="1:jcl:" cohortname="Auto" status="ok"><updatecheck status="noupdate"/></app></gupdate>

The test is expecting fetch to iterate Apps elements in the response to pull out a URL and Version attribute from child UpdateCheck elements to find a usable crxURL. The response above doesn't include this information in the XML so ultimately fetch returns a errors.New("Failed to parse Omaha response") err and the TestFetch unit tests fails.

It seems like the code in this package is adopted from Agl's crlset-tools and I notice that he recently updated the library to account for a new API version: agl/crlset-tools@eb20879

The approach taken in the crlset-tools package skips signature checking. I'm not familiar enough with this part of zcrypto or its intended use-cases to determine if that's an appropriate trade-off for this package to adopt as well. If it is acceptable then backporting the update from crlset-tools will probably fix this problem.

x509.Cert JSON output removes invalid URL names

Marshaling output to JSON removes names that are not valid URLs - see isValidName function in x509/json.go. This should either be documented or changed, since it does not reflect the actual names contained in a certificate. Sidenote: this logic takes up a significant amount of processing time (discovered while performance profiling)

Certificate validation is incredibly slow with large numbers of intermediates

BenchmarkGolangX509NoIntermediates-4           	 3000000	       578 ns/op
BenchmarkGolangX509WithIntermediates-4         	    2000	    711043 ns/op
BenchmarkGolangX509WithChain-4                 	   10000	    158716 ns/op
BenchmarkRawX509NoIntermediates-4              	10000000	       193 ns/op
BenchmarkRawX509WithIntermediates-4            	    1000	   1032861 ns/op
BenchmarkRawX509WithChain-4                    	   10000	    165573 ns/op
BenchmarkVerifyNSSNoIntermediates-4            	 2000000	       656 ns/op
BenchmarkVerifyNSSWithChain-4                  	   10000	    163489 ns/op
BenchmarkVerifyNSSWithIntermediates-4          	    1000	   1004869 ns/op
BenchmarkVerifyNSSWithIntermediatesAsRoots-4   	    5000	    213618 ns/op

In these benchmarks, WithIntermediates means with the set of 1.5K intermediates trusted by NSS ~May 2015, and WithChain means with the intermediates pool set to the chain served by a website (e.g. the one intermediate that matters).

RawX509 is github.com/zmap/zcrypto/x509, Verify is github.com/zmap/zcrypto/verifier, and GolangX509 is crypto/x509.

This suggests our current validation approach may be too slow for live certificate validation, and is also not appropriate for finding the set of trusted intermediates in Censys given the set of possible intermediates and a root store.

Multiple-Browser Validation

This is a meta-issue for tracking an API to do multi-browser validation. It will be updated as progress is made.

#13 Chrome
#14 Apple
#15 Microsoft

Certificate.CheckSignatureFrom should compare bytes

zcrypto/ct/x509/x509.go

Line 622 in 9791508

if parent.Subject.String() != c.Issuer.String() {

This should use bytes.Equal, see https://github.com/golang/go/blob/master/src/crypto/x509/verify.go#L194-L199 in Golang upstream.

Then again, maybe we don't actually want to do this at all. Windows doesn't!

Need tests for x509.CertificateExtensions JSON Marshalling

x509/extensions.go has very little test coverage on the JSON Marshalling and Unmarshalling methods. This is a superset of the missing tests called for in #83.

Dependency github.com/zmap/rc2 has no license

I'm trying to package github.com/zmap/zcrypto for my distribution, but the dependency github.com/zmap/rc2 has no license and no way to file an issue. Could you please specify under what license is the code distributed?
Thanks.

Add support for OCSP

The verifier should have a way to efficiently check OCSP (optionally). The OCSP parsing code should probably exist in the x509 package, but the usage should be in the verifier package.

subject_dn should not use static order

Right now, the parsed.subject_dn and parsed.issuer_dn fields are output in a static order. They should be output as the fields are ordered in the certificate.

Add Unmarshalers to x509/json

Sub-issue of #78.

There are several types in x509/json, most notably Certificate, which have JSON marshalers but no unmarshalers.

From #78:

We should ensure we have Unmarshalers for anything that has a Marshaler.

Tests are also needed for any new unmarshaler methods added.

Traling data in the IssuerAlternativeNames extension doesn't raise an error.

I was investigating some of the zlint's test certificates, and noticed this one:

This looks strange, as the issuer alternative name looks malformed.
Here is the asn.1 decoder link:

The issuer alt name AttributeTypeAndValue value contains two concatanated ASN.1 structs.
One is an empty sequence (0x30,0x00) followed by empty primitive value (context specific tag) (0x82,0x00)

Here is the openssl asn1parse dump (relevant part):

857:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Issuer Alternative Name
  862:d=5  hl=2 l=   4 prim: OCTET STRING      [HEX DUMP]:30008200

Zcrypto is not raising an error if there is trailing data after the extension value.

Golang x509 doesn't support this extension, so they just leave it in the Extensions slice without parsing, so it is not an upstream bug.

Implement IsURL util func, remove govalidator dependency

The github.com/asaskevich/govalidator dependency could be removed from both zcrypto and zlint if zcrypto provided its own IsURL utility function.

Alternatively, removing the three invocations of govalidator.IsURL in x509/json.go would accomplish similar.

zcrypto/x509/json.go

Lines 458 to 462 in 0afe7b2

 for _, name := range c.URIs { 

 if govalidator.IsURL(name) { 

 jc.Names = append(jc.Names, name) 

 } 

 }

zcrypto/x509/json.go

Lines 464 to 469 in 0afe7b2

 for _, name := range c.IPAddresses { 

 str := name.String() 

 if govalidator.IsURL(str) { 

 jc.Names = append(jc.Names, str) 

 } 

 }

zcrypto/x509/json.go

Lines 557 to 563 in 0afe7b2

 // Check for wildcards and redacts, ignore malformed urls 

 if strings.HasPrefix(name, "?.") || strings.HasPrefix(name, "*.") { 

 ret = isValidName(name[2:]) 

 } else { 

 ret = govalidator.IsURL(name) 

 } 

 return

Heartbeat protocol only partially supported

zcrypto/tls currently only supports some subset of the Heartbeat protocol. It also allows sending the extension in the ClientHello. This could raise problems if a server wants to send arbitrary heartbeat messages to a zcrypto client.

Add support for CAA

We should implement this in two stages:

Standalone CAA package, probably as a subpackage of verifier/. This would take a cert and a parent, and return CAA status.
Add an option to VerificationOptions to do CAA, and expose the results in VerificationResult.

Backport changes from golang 1.8 x509 lib

x509/pkix: Name:ToRDNSequence() doesn't process all fields from the Name struct

For example, EmailAddress and GivenName are not added.
Name:FillFromRDNSequence on the other hand, recognises those fields in the RDN and correctly fills the appropriate fields in the name struct.

Those methods should be complimentary
Both should handle all fields in the Name struct.

`parsed.signature_algorithm.oid` does not get populated for "unknown_algorithm"

{
  "ip": "185.32.226.170",
  "timestamp": "2017-10-11T13:33:26-04:00",
  "data": {
    "tls": {
      "server_hello": {
        "version": {
          "name": "TLSv1.0",
          "value": 769
        },
        "random": "mVWLUC/ux8geHYHDkvBw/vYk6dl2yP7BYZHd4VK3Kfs=",
        "session_id": "fzCpkwjSi90n+WodwutaNoylQZSiL7yqlRP2l4dx/iY=",
        "cipher_suite": {
          "hex": "0x0100",
          "name": "unknown",
          "value": 256
        },
        "compression_method": 0,
        "ocsp_stapling": false,
        "ticket": false,
        "secure_renegotiation": false,
        "heartbeat": false,
        "extended_master_secret": false
      },
      "server_certificates": {
        "certificate": {
          "raw": "MIIEQjCCBACgAwIBAgIMQOTCTcLhpOMAAHA7MA0GCSsGAQQB4nABKwUAMIIBczFPME0GA1UEAx5GBCAEMAQxBD4ERwQ4BDkAIARDBDQEPgRBBEIEPgQyBDUEQARPBE4ESQQ4BDkAIARGBDUEPQRCBEAAIAQtBCEAIAQdBBQEITFPME0GA1UECh5GBBwEOAQ9BDgEQQRCBDUEQARBBEIEMgQ+ACAEPwQ+ACAEPQQwBDsEPgQzBDAEPAAgBDgAIARBBDEEPgRABDAEPAAgBCAEETELMAkGA1UEBhMCQlkxFzAVBgNVBAgeDgQcBDgEPQRBBDoEMARPMRkwFwYDVQQHHhAEMwAuACAEHAQ4BD0EQQQ6MScwJQYDVQQJHh4EQwQ7AC4AIAQhBD4EMgQ1BEIEQQQ6BDAETwAgADkxZTBjBgNVBAseXAQeBEIENAQ1BDsAIAQ4BD0ERAQ+BEAEPAQwBEYEOAQ+BD0EPQQ+BDkAIAQxBDUENwQ+BD8EMARBBD0EPgRBBEIEOAAgBDgAIARBBD8ENQRGBEAEMAQxBD4EQgRLMB4XDTE2MDUyNDA3MTkxMFoXDTE4MDUyMzIwNTk1OVowUDELMAkGA1UEBhMCQlkxDjAMBgNVBAgMBU1pbnNrMQ4wDAYDVQQHDAVNaW5zazEMMAoGA1UECgwDTU5TMRMwEQYDVQQDDAp2YXQuZ292LmJ5MIIBRzAYBgkrBgEEAeJwASWgCwYJKwYBBAHicAcCA4IBKQAwggEkMIGPBgkrBgEEAeJwASMDgYECxT0wsOO1Zmk40kh0uQlEX1JHPMYRHbEGExva0I61w7CZa1az6nzJLTbbd1RYpK9XgYgEgmbN5a4VYVakSoDPDakkBj0dupD3pBV3KyNDt6Bjm7pa/3K6eWFgTu8TRLCW8yVeEy649jOdD83Qh+ix3Aj1vLaRoqeIvnQAOMGoEzAwgY8GCSsGAQQB4nABIAOBgQJeZ1k0gDt+/3tgxy66/qSeJ4HPBmTiTH8F6WzZlacuxRytCoBozB7hiX3kb3HkoyOL2g7L+AHnMMuFa/bywj3X3VqeYSSwUA064AD9sy2nkmCcBcSUkH3NkCrwdXHOp0+dOmqPklqvKeiL7gwatbf3Ncn4e5WEB2vtiapbcPF44KOBpzCBpDAfBgNVHSMEGDAWgBQVM13mg/o62NS7r52iNDbBTV9oNzAJBgNVHRMEAjAAMB0GA1UdDgQWBBSnkZLO1Zw7FEjmrRH6JxIYJWzdgzALBgNVHQ8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwNQYDVR0RBC4wLIIKdmF0Lmdvdi5ieYIOMTg1LjMyLjIyNi4xOTOCDnd3dy52YXQuZ292LmJ5MA0GCSsGAQQB4nABKwUAAy0AHOpQZKZrSSTfRjkz+Q6l+FtOUOvrGBc8kzuQRJGArUZKiDvbg4h4PnJ+OZs=",
          "parsed": {
            "version": 3,
            "serial_number": "20083593288977876645008404539",
            "signature_algorithm": {
              "name": "unknown_algorithm",
              "oid": ""
            },
            "issuer": {
              "country": [
                "BY"
              ]
            },
            "issuer_dn": "C=BY",
            "validity": {
              "start": "2016-05-24T07:19:10Z",
              "end": "2018-05-23T20:59:59Z",
              "length": 63034849
            },
            "subject": {
              "common_name": [
                "vat.gov.by"
              ],
              "country": [
                "BY"
              ],
              "locality": [
                "Minsk"
              ],
              "province": [
                "Minsk"
              ],
              "organization": [
                "MNS"
              ]
            },
            "subject_dn": "C=BY, ST=Minsk, L=Minsk, O=MNS, CN=vat.gov.by",
            "subject_key_info": {
              "key_algorithm": {
                "name": "unknown_algorithm"
              },
              "fingerprint_sha256": "0fb4c67bdc80accea1bd9cb99e09bce1965653675e179ae7c83dddc84ca78557"
            },
            "extensions": {
              "key_usage": {
                "digital_signature": true,
                "key_encipherment": true,
                "value": 5
              },
              "basic_constraints": {
                "is_ca": false
              },
              "subject_alt_name": {
                "dns_names": [
                  "vat.gov.by",
                  "185.32.226.193",
                  "www.vat.gov.by"
                ]
              },
              "authority_key_id": "15335de683fa3ad8d4bbaf9da23436c14d5f6837",
              "subject_key_id": "a79192ced59c3b1448e6ad11fa271218256cdd83",
              "extended_key_usage": {
                "server_auth": true
              }
            },
            "signature": {
              "signature_algorithm": {
                "name": "unknown_algorithm",
                "oid": ""
              },
              "value": "HOpQZKZrSSTfRjkz+Q6l+FtOUOvrGBc8kzuQRJGArUZKiDvbg4h4PnJ+OZs=",
              "valid": false,
              "self_signed": false
            },
            "fingerprint_md5": "9dd655564049ade0b069adfc54a86d52",
            "fingerprint_sha1": "dfda16d720e756fc2859b4f7eda646cfb48bcbc9",
            "fingerprint_sha256": "3a3b99196ea0633eeffc4342b84d34948457e39a92bf417e55da5dc70e478639",
            "tbs_noct_fingerprint": "02b9590a72911ef80545bf0afaa585737eef8f6731a910d9d5b54fe51d715a5a",
            "spki_subject_fingerprint": "b8578fae849c72261337f1726699b8516251a58fbcd1163940a4a596e9374448",
            "tbs_fingerprint": "02b9590a72911ef80545bf0afaa585737eef8f6731a910d9d5b54fe51d715a5a",
            "validation_level": "unknown",
            "names": [
              "vat.gov.by",
              "185.32.226.193",
              "www.vat.gov.by"
            ],
            "redacted": false
          }
        },
        "chain": [
          {
            "raw": "MIIG+zCCBrmgAwIBAgIMQOLBrU4rnJ4AAAABMA0GCSsGAQQB4nABJAUAMIIBZzFDMEEGA1UEAx46BBoEPgRABD0ENQQyBD4EOQAgBEMENAQ+BEEEQgQ+BDIENQRABE8ETgRJBDgEOQAgBEYENQQ9BEIEQDFPME0GA1UECh5GBBwEOAQ9BDgEQQRCBDUEQARBBEIEMgQ+ACAEPwQ+ACAEPQQwBDsEPgQzBDAEPAAgBDgAIARBBDEEPgRABDAEPAAgBCAEETELMAkGA1UEBhMCQlkxFzAVBgNVBAgeDgQcBDgEPQRBBDoEMARPMRkwFwYDVQQHHhAEMwAuACAEHAQ4BD0EQQQ6MScwJQYDVQQJHh4EQwQ7AC4AIAQhBD4EMgQ1BEIEQQQ6BDAETwAgADkxZTBjBgNVBAseXAQeBEIENAQ1BDsAIAQ4BD0ERAQ+BEAEPAQwBEYEOAQ+BD0EPQQ+BDkAIAQxBDUENwQ+BD8EMARBBD0EPgRBBEIEOAAgBDgAIARBBD8ENQRGBEAEMAQxBD4EQgRLMB4XDTA1MDMwMjA3NTczNVoXDTE5MTIyNjIxNTk1OVowggFzMU8wTQYDVQQDHkYEIAQwBDEEPgRHBDgEOQAgBEMENAQ+BEEEQgQ+BDIENQRABE8ETgRJBDgEOQAgBEYENQQ9BEIEQAAgBC0EIQAgBB0EFAQhMU8wTQYDVQQKHkYEHAQ4BD0EOARBBEIENQRABEEEQgQyBD4AIAQ/BD4AIAQ9BDAEOwQ+BDMEMAQ8ACAEOAAgBEEEMQQ+BEAEMAQ8ACAEIAQRMQswCQYDVQQGEwJCWTEXMBUGA1UECB4OBBwEOAQ9BEEEOgQwBE8xGTAXBgNVBAceEAQzAC4AIAQcBDgEPQRBBDoxJzAlBgNVBAkeHgRDBDsALgAgBCEEPgQyBDUEQgRBBDoEMARPACAAOTFlMGMGA1UECx5cBB4EQgQ0BDUEOwAgBDgEPQREBD4EQAQ8BDAERgQ4BD4EPQQ9BD4EOQAgBDEENQQ3BD4EPwQwBEEEPQQ+BEEEQgQ4ACAEOAAgBEEEPwQ1BEYEQAQwBDEEPgRCBEswggFHMBgGCSsGAQQB4nABJaALBgkrBgEEAeJwBwIDggEpADCCASQwgY8GCSsGAQQB4nABIwOBgQIgnKcPDRSTomXSuwQJ7yMjUrOOjtf/yaw9V2yNyYtyA3+AlegUGPu9zZ1d1np3aJp0tRSMv5qecXEVnrOUa2p+rjEVMwjZVvozNx1AXMQa/xd51HVHuVv1KBskQZa06HZdtAiOvV/gD7AHp5wa9w86Uq7xriQyO0K1v4Bgus4yIDCBjwYJKwYBBAHicAEgA4GBAm5uALAHoV/3tYkmGK1rFjVJ3Bq8Jxe8Kdyjp4MKOz8ckQIMv4+yYvfD23+6eOeUYxMAa9xE28g/tiA5DTTmDJkbdJwJ54f5LX2TydaJUN6b5GVcNqY5K9O87SCthJAKGkiA3J256e61PZfnyH5AG3VUrg228zktNjjXZPq0bEeQo4ICRjCCAkIwggGgBgNVHQEEggGXMIIBk4AUdZBaC5y7iZqsbDZKaaBbKFOoGWShggFrMIIBZzFDMEEGA1UEAx46BBoEPgRABD0ENQQyBD4EOQAgBEMENAQ+BEEEQgQ+BDIENQRABE8ETgRJBDgEOQAgBEYENQQ9BEIEQDFPME0GA1UECh5GBBwEOAQ9BDgEQQRCBDUEQARBBEIEMgQ+ACAEPwQ+ACAEPQQwBDsEPgQzBDAEPAAgBDgAIARBBDEEPgRABDAEPAAgBCAEETELMAkGA1UEBhMCQlkxFzAVBgNVBAgeDgQcBDgEPQRBBDoEMARPMRkwFwYDVQQHHhAEMwAuACAEHAQ4BD0EQQQ6MScwJQYDVQQJHh4EQwQ7AC4AIAQhBD4EMgQ1BEIEQQQ6BDAETwAgADkxZTBjBgNVBAseXAQeBEIENAQ1BDsAIAQ4BD0ERAQ+BEAEPAQwBEYEOAQ+BD0EPQQ+BDkAIAQxBDUENwQ+BD8EMARBBD0EPgRBBEIEOAAgBDgAIARBBD8ENQRGBEAEMAQxBD4EQgRLggxA4r1ukxR4jQAAAAEwDAYDVR0TBAUwAwEB/zALBgNVHQ8EBAMCAb4wHQYDVR0OBBYEFBUzXeaD+jrY1LuvnaI0NsFNX2g3MC0GA1UdAgQmMCQwIoAPMjAwNTAzMDIwNzUyMzdagQ8yMDIwMDMwMTIxNTk1OVowMwYJKwYBBAHicAUEBCYwJDAigA8yMDA1MDMwMjA3NTIzN1qBDzIwMjAwMzAyMDc1MjM3WjANBgkrBgEEAeJwASQFAAMtAADCbfkNYQIukDpQqD8rZwz4D7kICFVKDo67dGbvwHDYNT9YsAgv1S3kFfEx",
            "parsed": {
              "version": 3,
              "serial_number": "20081172477449677146523959297",
              "signature_algorithm": {
                "name": "unknown_algorithm",
                "oid": ""
              },
              "issuer": {
                "country": [
                  "BY"
                ]
              },
              "issuer_dn": "C=BY",
              "validity": {
                "start": "2005-03-02T07:57:35Z",
                "end": "2019-12-26T21:59:59Z",
                "length": 467647344
              },
              "subject": {
                "country": [
                  "BY"
                ]
              },
              "subject_dn": "C=BY",
              "subject_key_info": {
                "key_algorithm": {
                  "name": "unknown_algorithm"
                },
                "fingerprint_sha256": "d8de1ea1ae19d1760a0e0157343094c7e0d298e756601a796ec32d168f6a6d5b"
              },
              "extensions": {
                "key_usage": {
                  "digital_signature": true,
                  "key_encipherment": true,
                  "data_encipherment": true,
                  "key_agreement": true,
                  "certificate_sign": true,
                  "crl_sign": true,
                  "value": 125
                },
                "basic_constraints": {
                  "is_ca": true
                },
                "subject_key_id": "15335de683fa3ad8d4bbaf9da23436c14d5f6837"
              },
              "unknown_extensions": [
                {
                  "id": "2.5.29.1",
                  "critical": false,
                  "value": "MIIBk4AUdZBaC5y7iZqsbDZKaaBbKFOoGWShggFrMIIBZzFDMEEGA1UEAx46BBoEPgRABD0ENQQyBD4EOQAgBEMENAQ+BEEEQgQ+BDIENQRABE8ETgRJBDgEOQAgBEYENQQ9BEIEQDFPME0GA1UECh5GBBwEOAQ9BDgEQQRCBDUEQARBBEIEMgQ+ACAEPwQ+ACAEPQQwBDsEPgQzBDAEPAAgBDgAIARBBDEEPgRABDAEPAAgBCAEETELMAkGA1UEBhMCQlkxFzAVBgNVBAgeDgQcBDgEPQRBBDoEMARPMRkwFwYDVQQHHhAEMwAuACAEHAQ4BD0EQQQ6MScwJQYDVQQJHh4EQwQ7AC4AIAQhBD4EMgQ1BEIEQQQ6BDAETwAgADkxZTBjBgNVBAseXAQeBEIENAQ1BDsAIAQ4BD0ERAQ+BEAEPAQwBEYEOAQ+BD0EPQQ+BDkAIAQxBDUENwQ+BD8EMARBBD0EPgRBBEIEOAAgBDgAIARBBD8ENQRGBEAEMAQxBD4EQgRLggxA4r1ukxR4jQAAAAE="
                },
                {
                  "id": "2.5.29.2",
                  "critical": false,
                  "value": "MCQwIoAPMjAwNTAzMDIwNzUyMzdagQ8yMDIwMDMwMTIxNTk1OVo="
                },
                {
                  "id": "1.3.6.1.4.1.12656.5.4",
                  "critical": false,
                  "value": "MCQwIoAPMjAwNTAzMDIwNzUyMzdagQ8yMDIwMDMwMjA3NTIzN1o="
                }
              ],
              "signature": {
                "signature_algorithm": {
                  "name": "unknown_algorithm",
                  "oid": ""
                },
                "value": "AMJt+Q1hAi6QOlCoPytnDPgPuQgIVUoOjrt0Zu/AcNg1P1iwCC/VLeQV8TE=",
                "valid": false,
                "self_signed": false
              },
              "fingerprint_md5": "0c5eec266f5a07159ea462dd4105c70f",
              "fingerprint_sha1": "e059c2c8c2bafe691a8df2b207cbe33f783d5c44",
              "fingerprint_sha256": "69e62e352bf65ba7b7aa8db904b5c0eaa1cf160d9873f6a7d09c465d109accac",
              "tbs_noct_fingerprint": "7cf4d80eafa8d7d7e5e6b72d1ec69df511081224aa0a2ad98600abe28b1f91f0",
              "spki_subject_fingerprint": "4e6479164447cdbde6e35874150de7ba67927305c3c12f9acb3c2e5f493574d5",
              "tbs_fingerprint": "7cf4d80eafa8d7d7e5e6b72d1ec69df511081224aa0a2ad98600abe28b1f91f0",
              "validation_level": "unknown",
              "redacted": false
            }
          },
          {
            "raw": "MIIG7zCCBq2gAwIBAgIMQOK9bpMUeI0AAAABMA0GCSsGAQQB4nABJAUAMIIBZzFDMEEGA1UEAx46BBoEPgRABD0ENQQyBD4EOQAgBEMENAQ+BEEEQgQ+BDIENQRABE8ETgRJBDgEOQAgBEYENQQ9BEIEQDFPME0GA1UECh5GBBwEOAQ9BDgEQQRCBDUEQARBBEIEMgQ+ACAEPwQ+ACAEPQQwBDsEPgQzBDAEPAAgBDgAIARBBDEEPgRABDAEPAAgBCAEETELMAkGA1UEBhMCQlkxFzAVBgNVBAgeDgQcBDgEPQRBBDoEMARPMRkwFwYDVQQHHhAEMwAuACAEHAQ4BD0EQQQ6MScwJQYDVQQJHh4EQwQ7AC4AIAQhBD4EMgQ1BEIEQQQ6BDAETwAgADkxZTBjBgNVBAseXAQeBEIENAQ1BDsAIAQ4BD0ERAQ+BEAEPAQwBEYEOAQ+BD0EPQQ+BDkAIAQxBDUENwQ+BD8EMARBBD0EPgRBBEIEOAAgBDgAIARBBD8ENQRGBEAEMAQxBD4EQgRLMB4XDTA1MDEyNzA3NTUxN1oXDTIwMDEyNjIwNTk1OVowggFnMUMwQQYDVQQDHjoEGgQ+BEAEPQQ1BDIEPgQ5ACAEQwQ0BD4EQQRCBD4EMgQ1BEAETwROBEkEOAQ5ACAERgQ1BD0EQgRAMU8wTQYDVQQKHkYEHAQ4BD0EOARBBEIENQRABEEEQgQyBD4AIAQ/BD4AIAQ9BDAEOwQ+BDMEMAQ8ACAEOAAgBEEEMQQ+BEAEMAQ8ACAEIAQRMQswCQYDVQQGEwJCWTEXMBUGA1UECB4OBBwEOAQ9BEEEOgQwBE8xGTAXBgNVBAceEAQzAC4AIAQcBDgEPQRBBDoxJzAlBgNVBAkeHgRDBDsALgAgBCEEPgQyBDUEQgRBBDoEMARPACAAOTFlMGMGA1UECx5cBB4EQgQ0BDUEOwAgBDgEPQREBD4EQAQ8BDAERgQ4BD4EPQQ9BD4EOQAgBDEENQQ3BD4EPwQwBEEEPQQ+BEEEQgQ4ACAEOAAgBEEEPwQ1BEYEQAQwBDEEPgRCBEswggFHMBgGCSsGAQQB4nABJaALBgkrBgEEAeJwBwIDggEpADCCASQwgY8GCSsGAQQB4nABIwOBgQKVLEsWtjugbtF8yVYa6GjOove4oVFAXJF7K6ofrT3V+Mp3Hm/I0/pFzHEdE1xl9MXlzv4XWVclKi5976sWbU7IxdNw0zmBS0nuS04GnFg+cK5v1Wmw8MYoBj6qjAqo4pk/Kuvfo//NUP6hvculg1l/6kD/xyqyYWZ6ofbW1LwBADCBjwYJKwYBBAHicAEgA4GBApyuUe1T/pMA/IeayGXFHKZ8KGWr3l5/gidYSQdPkhdjm6QHw4J57UHhZivZIrpR4qXXdHHy7SH3PjEl9mKgVph0yQ8g3lEZe9tvUtedW4MsuHaCGNK7c/RJos4xgBuiWshkcQWrrITWwX0ZgwieLTJeq+frgxWqAvOjeuEK6edwo4ICRjCCAkIwggGgBgNVHQEEggGXMIIBk4AUdZBaC5y7iZqsbDZKaaBbKFOoGWShggFrMIIBZzFDMEEGA1UEAx46BBoEPgRABD0ENQQyBD4EOQAgBEMENAQ+BEEEQgQ+BDIENQRABE8ETgRJBDgEOQAgBEYENQQ9BEIEQDFPME0GA1UECh5GBBwEOAQ9BDgEQQRCBDUEQARBBEIEMgQ+ACAEPwQ+ACAEPQQwBDsEPgQzBDAEPAAgBDgAIARBBDEEPgRABDAEPAAgBCAEETELMAkGA1UEBhMCQlkxFzAVBgNVBAgeDgQcBDgEPQRBBDoEMARPMRkwFwYDVQQHHhAEMwAuACAEHAQ4BD0EQQQ6MScwJQYDVQQJHh4EQwQ7AC4AIAQhBD4EMgQ1BEIEQQQ6BDAETwAgADkxZTBjBgNVBAseXAQeBEIENAQ1BDsAIAQ4BD0ERAQ+BEAEPAQwBEYEOAQ+BD0EPQQ+BDkAIAQxBDUENwQ+BD8EMARBBD0EPgRBBEIEOAAgBDgAIARBBD8ENQRGBEAEMAQxBD4EQgRLggxA4r1ukxR4jQAAAAEwHQYDVR0OBBYEFHWQWgucu4marGw2SmmgWyhTqBlkMAsGA1UdDwQEAwIBxjAMBgNVHRMEBTADAQH/MC0GA1UdAgQmMCQwIoAPMjAwNTAxMjcwNzU1MTdagQ8yMDIwMDEyNjIwNTk1OVowMwYJKwYBBAHicAUEBCYwJDAigA8yMDA1MDEyNzA3NTUxN1qBDzIwMjAwMTI3MjA1OTU5WjANBgkrBgEEAeJwASQFAAMtAA6/lp73nfTG3MBW9won75N8vnPWTdi5voizmkFmlXrD9+xP5BD5zduMNMys",
            "parsed": {
              "version": 3,
              "serial_number": "20081152430804329432960991233",
              "signature_algorithm": {
                "name": "unknown_algorithm",
                "oid": ""
              },
              "issuer": {
                "country": [
                  "BY"
                ]
              },
              "issuer_dn": "C=BY",
              "validity": {
                "start": "2005-01-27T07:55:17Z",
                "end": "2020-01-26T20:59:59Z",
                "length": 473259882
              },
              "subject": {
                "country": [
                  "BY"
                ]
              },
              "subject_dn": "C=BY",
              "subject_key_info": {
                "key_algorithm": {
                  "name": "unknown_algorithm"
                },
                "fingerprint_sha256": "c5a9e88628023e1cdc7445ca1e506033fb1e94f99ea89ebaf00032f6ceabf7e2"
              },
              "extensions": {
                "key_usage": {
                  "digital_signature": true,
                  "content_commitment": true,
                  "certificate_sign": true,
                  "crl_sign": true,
                  "value": 99
                },
                "basic_constraints": {
                  "is_ca": true
                },
                "subject_key_id": "75905a0b9cbb899aac6c364a69a05b2853a81964"
              },
              "unknown_extensions": [
                {
                  "id": "2.5.29.1",
                  "critical": false,
                  "value": "MIIBk4AUdZBaC5y7iZqsbDZKaaBbKFOoGWShggFrMIIBZzFDMEEGA1UEAx46BBoEPgRABD0ENQQyBD4EOQAgBEMENAQ+BEEEQgQ+BDIENQRABE8ETgRJBDgEOQAgBEYENQQ9BEIEQDFPME0GA1UECh5GBBwEOAQ9BDgEQQRCBDUEQARBBEIEMgQ+ACAEPwQ+ACAEPQQwBDsEPgQzBDAEPAAgBDgAIARBBDEEPgRABDAEPAAgBCAEETELMAkGA1UEBhMCQlkxFzAVBgNVBAgeDgQcBDgEPQRBBDoEMARPMRkwFwYDVQQHHhAEMwAuACAEHAQ4BD0EQQQ6MScwJQYDVQQJHh4EQwQ7AC4AIAQhBD4EMgQ1BEIEQQQ6BDAETwAgADkxZTBjBgNVBAseXAQeBEIENAQ1BDsAIAQ4BD0ERAQ+BEAEPAQwBEYEOAQ+BD0EPQQ+BDkAIAQxBDUENwQ+BD8EMARBBD0EPgRBBEIEOAAgBDgAIARBBD8ENQRGBEAEMAQxBD4EQgRLggxA4r1ukxR4jQAAAAE="
                },
                {
                  "id": "2.5.29.2",
                  "critical": false,
                  "value": "MCQwIoAPMjAwNTAxMjcwNzU1MTdagQ8yMDIwMDEyNjIwNTk1OVo="
                },
                {
                  "id": "1.3.6.1.4.1.12656.5.4",
                  "critical": false,
                  "value": "MCQwIoAPMjAwNTAxMjcwNzU1MTdagQ8yMDIwMDEyNzIwNTk1OVo="
                }
              ],
              "signature": {
                "signature_algorithm": {
                  "name": "unknown_algorithm",
                  "oid": ""
                },
                "value": "Dr+Wnved9MbcwFb3Cifvk3y+c9ZN2Lm+iLOaQWaVesP37E/kEPnN24w0zKw=",
                "valid": false,
                "self_signed": false
              },
              "fingerprint_md5": "c5c8e4e21bbb499b9208e72a3e5a74ac",
              "fingerprint_sha1": "b1493f619dfeca3cb63cad9ba942eaa8fa7b2d4d",
              "fingerprint_sha256": "2d5983d9a5d167f6716f4502f707c79e0376a3ac5c907fdb91585bd27fd685d9",
              "tbs_noct_fingerprint": "8c12d4a3a96f57840261701d42bb7c09061c1316d9ac676059ab0f4fe9412c05",
              "spki_subject_fingerprint": "bbc08d47edbc5356b2c96921403f3f0387361ab80b0884be8a05ad1e69636648",
              "tbs_fingerprint": "8c12d4a3a96f57840261701d42bb7c09061c1316d9ac676059ab0f4fe9412c05",
              "validation_level": "unknown",
              "redacted": false
            }
          }
        ],
        "validation": {
          "browser_trusted": false,
          "browser_error": "x509: certificate signed by unknown authority (possibly because of \"x509: cannot verify signature: algorithm unimplemented\" while trying to verify candidate authority certificate \"serial:20081172477449677146523959297\")"
        }
      }
    }
  },
  "error": "tls: server's certificate contains an unsupported type of public key: <nil>",
  "error_component": "tls"
}

Add support for CRL's

ZCrypto should be able to take in certificate revocation lists (CRLs) to check revocation status of certificates. At minimum, it should support some way of inputting the Chrome CRLSet and the Firefox OneCRL, and then read from it during verification.

Add 1.3.6.1.5.5.8.2.2

We should detect when 1.3.6.1.5.5.8.2.2 extended key usage is present.

Here's an example certificate:

https://crt.sh/?id=209727518&opt=cablint

CT package should use zcrypto/x509

The CT package currently contains it's own fork of crypto/x509 and crypto/asn1. It's not entirely clear why these are needed. At the very least, we should make sure whatever changes in the ct/x509 module are in zcrypto/x509, and try to remove that dependency. However, this will create a circular dependency between zcrypto/x509 and zcrypto/ct. So we'll then have to figure how what the ct module needs any form of x509 anyway, and possible extract that to a separate package.

Another possible solution might be to make it ct subpackage of zcrypto/x509.

Populate a map of OID -> pkix.Extension in Certificate objects.

tl;dr - I think zcrypto should offer a map[string]pkix.Extension field on x509.Certificate to avoid lots of linear searching in zlint's many lints.

In zlint a common pattern in the CheckApplies(c *x509.Certificate) bool functions of individual lints is to check a zcrypto.X509.Certificate for a specific x509 extension. e.g. consider the lint_ext_san_empty_name lint's CheckApplies function:

return util.IsExtInCert(c, util.SubjectAlternateNameOID)

The util.IsExtInCert function in turn relies on a util.GetExtFromCert function that performs a linear search over the zcrypto.X509.Certificate's Extensions field:

	for i := range cert.Extensions {
		if oid.Equal(cert.Extensions[i].Id) {
			return &(cert.Extensions[i])
		}
	}

Looking across the whole zlint codebase with a quick ack I see there are 200 references to util.GetExtFromCert or util.IsExtInCert:

$ ack -c -h "GetExtFromCert|IsExtInCert" $GOPATH/src/github.com/zmap/zlint
200

Benchmarking confirms that github.com/zmap/zlint/util.GetExtFromCert and encoding/asn1.ObjectIdentifier.Equal are present int the top 10 nodes accounting for cpu time in a pprof trace:

$ cd $GOPATH/src/github.com/zmap/zlint
$ go test --run=XXX -bench=. -cpuprofile all.profile
<snipped>
$ go tool pprof all.profile
File: zlint.test
Type: cpu
Time: Jul 26, 2019 at 10:58am (EDT)
Duration: 4.13mins, Total samples = 4.79mins (115.90%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top10
Showing nodes accounting for 146.71s, 51.07% of 287.25s total
Dropped 522 nodes (cum <= 1.44s)
Showing top 10 nodes out of 221
      flat  flat%   sum%        cum   cum%
    45.89s 15.98% 15.98%    142.36s 49.56%  runtime.mallocgc
    31.59s 11.00% 26.97%     39.26s 13.67%  runtime.heapBitsSetType
    18.20s  6.34% 33.31%     18.20s  6.34%  runtime.nextFreeFast
    11.27s  3.92% 37.23%     20.05s  6.98%  runtime.scanobject
     9.90s  3.45% 40.68%      9.90s  3.45%  runtime.memclrNoHeapPointers
     8.31s  2.89% 43.57%      8.31s  2.89%  encoding/asn1.ObjectIdentifier.Equal
     6.92s  2.41% 45.98%     13.52s  4.71%  github.com/zmap/zlint/util.GetExtFromCert
     5.63s  1.96% 47.94%         7s  2.44%  runtime.heapBitsForAddr
     4.60s  1.60% 49.54%    130.51s 45.43%  runtime.newobject
     4.40s  1.53% 51.07%    245.67s 85.52%  github.com/zmap/zlint.BenchmarkZlint.func4

The CheckApplies function is called for all configured lints, for all input certificates because the function is used to gate the further application of the lint logic.

It seems like much of this processing could be avoided if zcrypto were to provide a new field on x509.Certificate that holds map[string]pkix.Extension where the string key is the asn1.ObjectIdentifier.String() of the pkix.Extension.Id.

This map could be populated once at the time the certificate was parsed. Subsequently all of the lints that need to check if a given certificate have an extension they are interested in can do it with an O(1) membership check on the map field.

What do folks think? Am I missing an obvious downside (other than slightly more memory per Certificate)? If you're game I will take a crack at implementing this and then adjusting the downstream lints afterwards.

cc @dadrian @zakird

Microsoft Validation

Implement validation mimicking the behavior of Microsoft SChannel. This includes:

Accepting the Microsoft Root Store
Defining the set of certificates blacklisted by Microsoft. This may require inspecting a binary to determine.
Defining the set of Microsoft whitelisted certificates, e.g. "old enough" Startcom certificates. This may also require inspecting a binary to determine.
Any other extraneous Microsoft validation behavior.

Define interface for logging from scanner.go

scanner.go depends on the go-logging library, and this requirement is exposed in zcrypto's public interface (scanner.New).

Per a chat with @dadrian, logrus is now preferred for our development. Rather than exposing a dependency on a specific logger library, zcrypto should define an interface it uses for logging our of scanner.go, and clients of that API can pass in their own logger.

Support X25519 key exchange

I believe that upstream golang already supports this.

verifier/ should test with certs that have invalid signatures.

We're at about 87% test coverage in the verifier package, the remaining coverage is almost exclusively limited to testing lines that check signatures of certificates with matching subject/issuers. We need to write some tests that make valid-looking certificates, but with bits flipped in the signature. Given that the signature checking function is tested in x509/, we have low risk of being incorrect here, but I'd like to have this covered. Having these tests will prevent us from accidentally reversing our error handling, since that's easy to do in Golang due to it's lack of option types.

Self-signed certificates always have signature.valid = false

Self-signed certificates should only have signature.valid = false when they're self-signed (i.e. issuer matches subject), but the signature is actually not from the key in the certificate.

Chrome cipher list is out of date

See https://github.com/zmap/zgrab/issues/309

I'm not sure how far out of sync we are---I know Chrome's using all the fancy DJB crypto (X25519, ChaCha, etc) and I'm not sure what we support. We'll need to grab a cipherlist from Chrome and see what they send, then figure out which of those ciphers we can even speak.

Refactor Key Marshaling in x509/json

Sub-issue of #78.

From #78:

Key marshaling is very inconsistent. We have some help from zgrab/ztools/keys with this. We should probably instead define these Marshalers somewhere in ZCrypto, and use them within ZCrypto and ZGrab as needed.

zcrypto can output invalid base64 for certs' subject_key_id

The following certificates have parsed.subject_key_id values that the Censys frontend fails at parsing as base64 and hex-encoding:

@zakird asked me to "file a bug in ZCrypto and figure out why it's outputting invalid b64." for these certs.

pkix.Name should explicitly parse email addresses

Right now, the only email address that get parsed in the X.509 library at those included in SANs, not in the DN itself.

Example certificate: https://crt.sh/?caid=699

Remove Private Member Structs from x509/json

Sub-issue of #78.

From #78:

Composed types should mostly have their own MarshalJSON functions implemented, instead of doing the whole thing top down with private structs.

x509.Certificate JSON Marshalling does not use best practices

The implementation of MarshalJSON for x509.Certificate seems to have been written before we knew about omitempty. We need to go through and clean up the implementation in several ways:

Composed types should mostly have their own MarshalJSON functions implemented, instead of doing the whole thing top down with private structs.
Key marshaling is very inconsistent. We have some help from zgrab/ztools/keys with this. We should probably instead define these Marshalers somewhere in ZCrypto, and use them within ZCrypto and ZGrab as needed.
We should not use a map[string]interface{} anywhere.
We should ensure we have Unmarshalers for anything that has a Marshaler.
We should write tests for the Marshalers and Unmarshalers.

We may want to consider breaking this issue down into several subissues, but I think this provides a decent overview of the problem. Effectively, x509/json.go needs a large rewrite/refactor.

Simplify / document process of adding new tests for TLS

We use the tests for TLS from Golang in 2014. We should probably be pulling in the TLS tests from Golang 1.8. Additionally, we don't have good tests for our new ciphers that aren't in Golang stdlib (e.g. export ciphers).

The test runner compares a "deterministic" TLS handshake to a hex dump of a handshake with OpenSSL. We should figure out how this works, document/script it, and make it easier to add new tests of this form for when we add features.

Zcrypto refuses to comile with go 1.10rc1 on macos

Hi, I have been testing the [email protected] and discovered that this package doesn't compile:
I am getting:
vendor/github.com/zmap/zcrypto/ct/x509/root_darwin.go:73: cannot use nil as type _Ctype_CFDataRef in assignment

After a quick glance I have discovered that zcrypto/ct/x509 is quite outdated compared to the https://github.com/google/certificate-transparency-go and the baseline x509 package. The same thing for zcrypto/ct/asn1 and possibly zcrypto/x509.

Updating the offending functions should be easy enough, and I could do that, but I tried to merge github.com/google/certificate-transparency-go into zcrypto and it turned out bit tricky.
There are small changes sprinkled here and there, and without knowing what and where have been modified it is very easy to miss something. And not all is wrapped in // Begin CT changes, // End CT changes.

I can open a PR to just update the offending parts, so there is no nasty surprise when other users of this repo jump to go 1.10, but I'd suggest an general update to the x509/asn1 packages.

ct/x509: Package fails to build on Darwin with Go 1.10.1

zgrab fails to build on OS X running Go 1.10.1:

$ go get -u github.com/zmap/zgrab
# github.com/zmap/zgrab/vendor/github.com/zmap/zcrypto/ct/x509
vendor/github.com/zmap/zcrypto/ct/x509/root_darwin.go:73: cannot use nil as type _Ctype_CFDataRef in assignment

Apple Validation

Implement validation mimicking the behavior of Apple SecureTransport. This includes:

Accepting the Apple Root Store
Defining the set of blacklisted certificates in Apple. This may require inspecting a binary to determine.
Defining the set of whitelisted certificates in Apple, e.g. "old enough" Startcom certificates. This may also require inspecting a binary to determine.
Any other extraneous Apple validation behavior.

Remove map[string]interface{} usage in x509/json

Sub-issue of #78, we should remove all usage of map[string]interface{} when marshaling keys in x509/json.

tls: Expose supported algorithms

There should be some way for users to determine if a given e.g. cipher suite or signature algorithm is actually supported by zcrypto (without waiting for a server to try using it).

pkix.Name MarshalJSON() should not use map[string]interface{}

It should instead use a private struct with omitempty set correctly. It looks like this exists in code, but is not used.

Permit the decoding of non-minimally-encoded ASN.1 INTEGERs

ZLint throws a fatal error for this certificate:
https://crt.sh/?id=628153824&opt=zlint

The problem is that the public exponent is DER-encoded as 0x020400010001 instead of 0x0203010001. Whilst I'd expect the Go standard library's encoding/asn1 decoder to complain about this, ISTM that zcrypto/ct/asn1 should be more permissive here.

Need tests for x509.Certificate JSON Marshalling

There is currently no test coverage for x509/json.go. Before starting on the MarshalJSON refactor required in #78 , it would be good to have complete test coverage on the current marshaling code. With testing code in place we can ensure that the refactor doesn't accidentally change the structure of the json output (which must remain identical).

Set up Godoc

There's some way to autogenerate documentation from comments. The original Google code already has the comments, so at the very least that would autogenerate. We should generate it, and modify it according to our changes.

Chrome Validation

Implement validation mimicking the behavior of Chrome. This includes:

Accepting the "Chrome Root Store", which is actually the OS stores. It's unclear how to handle this.
Defining the set of blacklisted certificates in Chrome.
Defining the set of whitelisted certificates in Chrome, e.g. "old enough" Startcom certificates.
Any other extraneous Chrome validation behavior.

Performance impact of publicsuffix parsing

The weppos publicsuffix package has a significant impact on the performance of zcrypto/x509 and zmap/zlint, https://github.com/globalsign/publicsuffix might help to improve performance significantly.

See the pprof snippet below where the publicsuffix.ParseFromListWithOptions call takes about 35% of the whole runtime.

GeneralName, NameConstraints, and IAN tests for JSON marshaling need to be replaced

Previously these tests were not actually testing what they said they were. See comments in code

https://github.com/zmap/zcrypto/pull/82/files#diff-c9d4962cba8f74fa4d4d7c0ed169ae95R1082
https://github.com/zmap/zcrypto/pull/82/files#diff-89a3839efd1dc91065329d41db6a63b2R19

The tests should be implemented as table-driven tests, see pkix/json_test.go for an example.

Parse TorServiceDescriptorHash x509 extension

Based on feedback on a zlint PR (zmap/zlint#263) zcrypto is the correct place to parse the TorServiceDescriptorHash x509 extension (oid 2.23.140.1.31) described initially in CABF Ballot 144.

I'm planning to submit a PR for this in the next week or so to complete zmap/zlint#14

I need an issue to reference in my TODO list so I don't forget about it :-)

tls: Make signatureAndHash list configurable

Currently the signature and hashes used in the Client Hello message are read from tlsConfig.SignatureAndHashes if present, but its type is []signatureAndHash -- so the type is not exported, nor are its related consts / helper functions.

Adding certificates to a x509.CertPool gets increasingly slow

We should investigate if there's a clear way to speed this up. This becomes visible as you add more than 10-20K certificates to a single pool.

Add tests for x509/cert_pool.go

CertPool currently has no tests. We've added some functionality to it, so we should definitely test that. Current versions of the Golang stdlib might have added tests for cert_pool that we could use as a basis.

The new functions as of May 15, 2017 are:

CertPool.Certificates()
CertPool.Contains()
CertPool.Sum()

Test chains from x509/verify_test.go should also be used by verifier/verifier.go

The x509 package has a bunch of nice tests to ensure chains are built correctly. The certificates used are stored as PEM's as Golang strings in the the test file. We should create a shared testdata package that can be accessed by both the tests for x509 and verifier.

It may also make sense to convert the test cases themselves to some sort of shared format that each package can hook in to.

	for _, name := range c.URIs {
	if govalidator.IsURL(name) {
	jc.Names = append(jc.Names, name)
	}
	}

	for _, name := range c.IPAddresses {
	str := name.String()
	if govalidator.IsURL(str) {
	jc.Names = append(jc.Names, str)
	}
	}

	// Check for wildcards and redacts, ignore malformed urls
	if strings.HasPrefix(name, "?.") \|\| strings.HasPrefix(name, "*.") {
	ret = isValidName(name[2:])
	} else {
	ret = govalidator.IsURL(name)
	}
	return

zmap / zcrypto Goto Github PK

zcrypto's Introduction

ZMap: The Internet Scanner

Using ZMap

Installation

Architecture

License and Copyright

zcrypto's People

Contributors

Stargazers

Watchers

Forkers

zcrypto's Issues

Recommend Projects

Recommend Topics

Recommend Org