paulsec / crt.sh Goto Github PK

Can I run a crt.sh search using a CSV of domain names?

Hi - is there any way to include additional certificate fields in the response JSON object?

[...{"crtsh_id": "114422", "pem_url": "https://crt.sh/?d=114422", "logged_at": "2013-03-26", "not_before": "2011-07-08", "domain": "2014-07-07", "issuer": "*.uber.com"}...]

Seems its not just returning a date in the domain field, is this API side?

A lot more functionality can be included by using a connection to the crt.sh database with PostgreSQL. Only some filters are exposed through the web interface.

For example, this query:
https://crt.sh/?q=%.github.com&showSQL=Y&exclude=expired
Can be performed with this PostgreSQL command:
psql -h crt.sh -p 5432 -U guest certwatch

certwatch=> SELECT ci.ISSUER_CA_ID,
        ca.NAME ISSUER_NAME,
        ci.NAME_VALUE NAME_VALUE,
        min(c.ID) MIN_CERT_ID,
        min(ctle.ENTRY_TIMESTAMP) MIN_ENTRY_TIMESTAMP,
        x509_notBefore(c.CERTIFICATE) NOT_BEFORE,
        x509_notAfter(c.CERTIFICATE) NOT_AFTER
    FROM ca,
        ct_log_entry ctle,
        certificate_identity ci,
        certificate c
    WHERE ci.ISSUER_CA_ID = ca.ID
        AND c.ID = ctle.CERTIFICATE_ID
        AND reverse(lower(ci.NAME_VALUE)) LIKE reverse(lower('%.github.com'))
        AND ci.CERTIFICATE_ID = c.ID
        AND x509_notAfter(c.CERTIFICATE) > statement_timestamp()
    GROUP BY c.ID, ci.ISSUER_CA_ID, ISSUER_NAME, NAME_VALUE
    ORDER BY MIN_ENTRY_TIMESTAMP DESC, NAME_VALUE, ISSUER_NAME;

I have not yet had success executing this command with psycopg2.

psycopg2.DatabaseError: long transactions not allowed
SSL connection has been closed unexpectedly

The project could benefit of a new release tag and PyPI package.

The crtsh.crtshAPI.search function can currently take the expired argument which is not in the latest 0.2.0 version.

Adding a new version and package release would allow packages which depend on this project to download it directly from PyPI and use these new features by including it directly in a package's setup.py and requirements.txt.

You should consider hosting this on PIP so it's easier for users to fetch.

PS: crt.sh also expose intentionally their postgreSQL DB if you give the address a quick port scan. Don't know if it's easier than scraping the JSON.