tantalor93 / dnspyre Goto Github PK

View Code? Open in Web Editor NEW

78.0 2.0 10.0 38.57 MB

CLI tool for a high QPS DNS benchmark

Home Page: https://tantalor93.github.io/dnspyre/

License: MIT License

Go 99.76% Shell 0.24%

dns benchmark cli dns-over-tls concurrency go plotting doh dns-over-https dns-over-tcp

dnspyre's Introduction

dnspyre

dnspyre is a command-line DNS benchmark tool built to stress test and measure the performance of DNS servers. You can easily run benchmark from MacOS, Linux or Windows systems.

This tool is based and originally forked from dnstrace, but was largely rewritten and enhanced with additional functionality.

This tool supports wide variety of options to customize DNS benchmark and benchmark output. For example, you can:

benchmark DNS servers using DNS queries over UDP or TCP
benchmark DNS servers with all kinds of query types like A, AAAA, CNAME, HTTPS, ... (--type option)
benchmark DNS servers with a lot of parallel queries and connections (--number, --concurrency options)
benchmark DNS servers for a specified duration (--duration option)
benchmark DNS servers with DoT (DNS over TLS)
benchmark DNS servers using DoH (DNS over HTTPS)
benchmark DNS servers using DoQ (DNS over QUIC)
benchmark DNS servers with uneven random load from provided high volume resources (see --probability option)
plot benchmark results via CLI histogram or plot the benchmark results as boxplot, histogram, line graphs and export them via all kind of image formats like png, svg and pdf. (see --plot and --plotf options)

Documentation

For installation guide, examples and more, see the documentation page

dnspyre's People

Stargazers

Watchers

Forkers

admpub mpavlikwandera tosunkaya rgs1 cleonyc raysajuuk m0zgen fubs

dnspyre's Issues

Support benchmarking with DNSSEC

Describe the feature
As part of benchmark I want to be able to specify that I want to the server to do the DNSSEC validation (set DO=1)

Why do you need this feature
i want to be able to observe performance of the tested server, when it is doing DNSSEC validation

Cannot check limit of number of files

Cannot check limit of number of files. Skipping check. Please make sure it is sufficient manually. RlimitStack is not implemented on Windows

What does this message mean, and could you provide a way to suppress this warning?

Keepalive on/off ?

Describe the feature
When running a TCP-based test (TCP/TLS/DoH/etc), it's hard to tell if connections are being reused or not. Could you add a flag to enable/disable connection reuse (aka keepalive)?

Why do you need this feature
I'm trying to get our DoH server to handle 200,000 QPS with <100ms response time for separate users. If the dnspyre clients are using keepalive, then the results may differ from what we'll see in production.

Can't connect to a url to get list

Reproduction:

docker run  tantalor93/dnspyre:v2.12.1 --server 8.8.8.8 https://raw.githubusercontent.com/Tantalor93/dnspyre/master/data/1000-domains

There was an error while starting benchmark: failed to download file 'https://raw.githubusercontent.com/Tantalor93/dnspyre/master/data/1000-domains' with error 'Get "https://raw.githubusercontent.com/Tantalor93/dnspyre/master/data/1000-domains": tls: failed to verify certificate: x509: certificate signed by unknown authority'

Using multiple source ips?

Describe the feature
If you want to do a tcp-based test, you can only do at most 64k simultaneous connections before you start reusing port numbers, and since a connection is defined by source+destination ip+port, there would be no way to tell the connections apart.

Many programs allow the user to specify a source ip address chosen from the ips of configured interfaces. Adding just that feature would let you start multiple copies of dnspyre, one per source ip, but that's going to split the statistics into per-interface buckets.

What would really be useful would be to specify a list of source ips to use. In fact, the loopback interface allows you to use any ip address in the 127.0.0.0/8 range, so in that special case it would be reasonable to specify a subnet, not just a list of ips.

Why do you need this feature
64k ports is not enough for really big tests.

TSIG support?

Describe the feature
We'd like to benchmark using TSIG on requests to see if it's faster that DoH or any of the other encrypted protocols.

Why do you need this feature
We've got a client who's worried about replay attacks, and requests that we support 200,000 requests per second with 100ms latency. We've achieved that with plain UDP, now we just have to get some sort of replay protection set up. DoH, DoT, DoQ and DNSSEC all provide that, but we're having problems reaching the necessary speed, and are hoping that TSIG will do the trick.

Since "Hope is Not a Plan", we're going to have to benchmark it.

The equivalent dig command is:

$ pdnsutil generate-tsig-key foobar hmac-sha512
Create new TSIG key foobar hmac-sha512 asdfasdfYpM2nUHdJtC3mDJnbBIXzUGgDU2/dwz8cDAfEQQ1O9OEON3+5aqTjQlHzXTyZwU

$ pdnsutil activate-tsig-key example.com foobar primary

$ cat > /tmp/foobar.tsig <<< 'EOF'
key "foobar" {
	algorithm hmac-sha512;
	secret "asdfasdfYpM2nUHdJtC3mDJnbBIXzUGgDU2/dwz8cDAfEQQ1O9OEON3+5aqTjQlHzXTyZwU";
};
EOF

$ dig +short -k /tmp/foobar.tsig txt testing.example.com @127.0.0.4
"this is a test"
"this is another test"

$ dig +short -k /tmp/snafu.tsig txt testing.example.com @127.0.0.4
;; Couldn't verify signature: expected a TSIG or SIG(0)

Adding ECS support, like dig's `+subnet=x.x.x.x/yy` option

Describe the feature
I'm glad to see that dnspyre has an --ednsopt option that lets you add arbitrary EDNS options, like Client Subnet (08).
Could you also add an option like --subnet=x.x.x.x/yy or --ecs=x.x.x.x/yy to make it a little easier?
Also, is there a way to specify a list of subnets to send on different queries?
Aaaand can it do IPv6 too?

Why do you need this feature
We're performance testing a version of PowerDNS authoritative server that's ECS-aware, and typing --ednsopt="08:00011600cc0fdc" instead of --subnet=204.15.220.0/22 is a bit tedious. :-)

Changed Server Certificate Validation for DoT in dnspyre v2.19.0+?

What happened

After upgrading to dnspyre v2.19.0 or v2.21.0, I encountered an issue specific to NextDNS during DoT testing:

tls: failed to verify certificate: x509: cannot validate certificate for 45.90.30.0 because it doesn't contain any IP SANs	1 (100.00)%

Unlike other DNS services like Quad9 and 1.1.1.1, which include extensive IP SANs in their server certificates, NextDNS uses a wildcard SAN. This seems to be causing the validation failure in dnspyre from v2.19.0 onwards. This issue did not occur in earlier versions, such as v2.18.2 or v2.17.0, so it's triggered only under certain circumstances.

What you expected to happen

I expected dnspyre to validate NextDNS's certificate without issues, as it did in versions prior to v2.19.0, considering that other DNS services with detailed IP SANs in their certificates are not affected.

dnspyre command

v2.18:

$ dnspyre --version
2.18.2-linux-amd64

$ dnspyre --dot --server="test.dns.nextdns.io" google.com
Using 1 hostnames
Benchmarking test.dns.nextdns.io:853 via tls with 1 concurrent requests 

Total requests:		1
DNS success codes:	1

DNS response codes:
	NOERROR:	1

DNS question types:
	A:	1

Time taken for tests:	 172.07ms
Questions per second:	 5.8
DNS timings, 1 datapoints
	 min:		 167.77ms
	 mean:		 171.97ms
	 [+/-sd]:	 0s
	 max:		 176.16ms
	 p99:		 176.16ms
	 p95:		 176.16ms
	 p90:		 176.16ms
	 p75:		 176.16ms
	 p50:		 176.16ms

v2.19+:

$ dnspyre --version
2.19.0-linux-amd64

$ dnspyre --dot --server="test.dns.nextdns.io" google.com
Using 1 hostnames
Benchmarking 45.90.30.0:853 via tls with 1 concurrent requests 


Total requests:		1
Read/Write errors:	1

Time taken for tests:	 76.7ms
Questions per second:	 13.0
Total Errors: 1
Top errors:
tls: failed to verify certificate: x509: cannot validate certificate for 45.90.30.0 because it doesn't contain any IP SANs	1 (100.00)%

$ dnspyre --dot --server="anycast.dns.nextdns.io" google.com
Using 1 hostnames
Benchmarking 45.90.28.0:853 via tls with 1 concurrent requests 


Total requests:		1
Read/Write errors:	1

Time taken for tests:	 111.21ms
Questions per second:	 9.0
Total Errors: 1
Top errors:
tls: failed to verify certificate: x509: cannot validate certificate for 45.90.28.0 because it doesn't contain any IP SANs	1 (100.00)%

$ dnspyre --dot --server="firefox.dns.nextdns.io" google.com 
Using 1 hostnames
Benchmarking 217.146.31.87:853 via tls with 1 concurrent requests 


Total requests:		1
Read/Write errors:	1

Time taken for tests:	 646.89ms
Questions per second:	 1.5
Total Errors: 1
Top errors:
tls: failed to verify certificate: x509: cannot validate certificate for 217.146.31.87 because it doesn't contain any IP SANs	1 (100.00)%

cert digging command

$ openssl s_client -connect dns.quad9.net:853 -showcerts | openssl x509 -noout -text | grep -A1 'Subject Alternative Name:'
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert TLS Hybrid ECC SHA384 2020 CA1
verify return:1
depth=0 C = US, ST = California, L = Berkeley, O = Quad9, CN = *.quad9.net
verify return:1
            X509v3 Subject Alternative Name: 
                DNS:*.quad9.net, DNS:quad9.net, IP Address:9.9.9.9, IP Address:9.9.9.10, IP Address:9.9.9.11, IP Address:9.9.9.12, IP Address:9.9.9.13, IP Address:9.9.9.14, IP Address:9.9.9.15, IP Address:149.112.112.9, IP Address:149.112.112.10, IP Address:149.112.112.11, IP Address:149.112.112.12, IP Address:149.112.112.13, IP Address:149.112.112.14, IP Address:149.112.112.15, IP Address:149.112.112.112, IP Address:2620:FE:0:0:0:0:0:9, IP Address:2620:FE:0:0:0:0:0:10, IP Address:2620:FE:0:0:0:0:0:11, IP Address:2620:FE:0:0:0:0:0:12, IP Address:2620:FE:0:0:0:0:0:13, IP Address:2620:FE:0:0:0:0:0:14, IP Address:2620:FE:0:0:0:0:0:15, IP Address:2620:FE:0:0:0:0:0:FE, IP Address:2620:FE:0:0:0:0:FE:9, IP Address:2620:FE:0:0:0:0:FE:10, IP Address:2620:FE:0:0:0:0:FE:11, IP Address:2620:FE:0:0:0:0:FE:12, IP Address:2620:FE:0:0:0:0:FE:13, IP Address:2620:FE:0:0:0:0:FE:14, IP Address:2620:FE:0:0:0:0:FE:15

$ openssl s_client -connect 1.1.1.1:853 -showcerts | openssl x509 -noout -text | grep -A1 'Subject Alternative Name:'
Can't use SSL_get_servername
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert TLS Hybrid ECC SHA384 2020 CA1
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = cloudflare-dns.com
verify return:1
            X509v3 Subject Alternative Name: 
                DNS:cloudflare-dns.com, DNS:*.cloudflare-dns.com, DNS:one.one.one.one, IP Address:1.0.0.1, IP Address:1.1.1.1, IP Address:162.159.36.1, IP Address:162.159.46.1, IP Address:2606:4700:4700:0:0:0:0:1001, IP Address:2606:4700:4700:0:0:0:0:1111, IP Address:2606:4700:4700:0:0:0:0:64, IP Address:2606:4700:4700:0:0:0:0:6400

$ openssl s_client -connect test.dns.nextdns.io:853 -showcerts | openssl x509 -noout -text | grep -A1 'Subject Alternative Name:'
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust ECC Certification Authority
verify return:1
depth=1 C = AT, O = ZeroSSL, CN = ZeroSSL ECC Domain Secure Site CA
verify return:1
depth=0 CN = dns.nextdns.io
verify return:1
            X509v3 Subject Alternative Name: 
                DNS:dns.nextdns.io, DNS:*.dns.nextdns.io

Insecure self signed certificates

Would it possible to add an insecure flag to ignore self signed certificates?
Load testing on an internal environment with ephemeral DNS clusters would benefit greatly from being able to accept an untrusted certificate.

Thanks

DoH support

Add support for querying DNS servers over DoH

Configurable exit code

Describe the feature
I'd like to have dnspyre return a nonzero exit code if any requests failed. Different people might have different definitions of failure at different times (e.g. connect timeout, nxdomain, duplicate answers, wrong answers, bad certificate, etc), so the parameter might take an argument: --fail="timeout,nxdomain"

Why do you need this feature
I have a program that does binary searches. It'll run a specified program and fill in a specified parameter with a number, and based on the return code of the program, will search until it finds the number where the program just starts to fail:

bsearch -l low -h high -- command args args {} args args
  This will find the largest number that can be used in place of the {},
  in the range of low to high.
  
  Options:
    -l low : the low number of the range to test
    -h high : the high number of the range to test

Without this feature, I have to write a wrapper script that runs dnspyre and then greps for a success or failure string.

For an example, see the --fail argument for curl.

Thanks!

PS: This looks like the best DNS benchmarking program out there! DNSperf / DNSRes are hard to use, Flamethrower is hard in a different way, and DNS shotgun is... weird? Dnspyre looks polished, and it has all the protocols I'm looking for.

JSON output flag for using programatically

Having used dnspyre for a while it's been great, but I kept ended up either grepping the stdout output or using --csv flag going to a third or fourth pipe to pass it downstream to do something with it. However the csv flag doesn't give me the rest of the output summary, like successful vs failed request counts and the p99 etc overviews.

Would it be possible to make a mode where the stdout output is json only, and non json output goes into stderr?

I was considering running dnspyre in a loosely isolated distributive mode on kubernetes but need some way to take the response from a worker pod back to a controlling operator, and json probably makes the most sense - can capture the stdout and curl it back to the operator http server to collect results per worker.