dmachard / go-dnscollector Goto Github PK

Ingesting, pipelining, and enhancing your DNS logs with usage indicators, security analysis, and additional metadata.

License: MIT License

Go 95.66% Python 3.50% Dockerfile 0.08% Lua 0.02% Shell 0.06% C 0.39% Makefile 0.29%

dns dnstap sniffer collector statistics openmetrics golang grafana pcap fluentd

go-dnscollector's Introduction

DNS-collector acts as a passive high speed ingestor with pipelining support for your DNS logs, written in Golang. It allows enhancing your DNS logs by adding metadata, extracting usage patterns, and facilitating security analysis. The DNS traffic can be collected and aggregated from simultaneously sources like DNStap streams, network interface or log files and relays it to multiple other listeners with some transformations on it (traffic filtering, user privacy, ...).

Additionally, DNS-collector also support

Extended DNStap with TLS encryption, compression, and more metadata capabilities

DNS protocol conversions (to plain text, JSON, and more... )

DNS parser with Extension Mechanisms for DNS (EDNS) support

IPv4/v6 defragmentation and TCP reassembly

Nanoseconds in timestamps

NOTE: The code before version 1.x is considered beta quality and is subject to breaking changes.

Features

Pipelining
Collectors
- Listen for logging traffic with streaming network protocols
  - DNStap with tls|tcp|unix transports support and proxifier
  - PowerDNS streams with full support
  - DNSMessage for internal DNS data structure
  - TZSP protocol support
- Live capture on a network interface
  - AF_PACKET socket with BPF filter
  - eBPF XDP ingress traffic
- Read text or binary files as input
  - Read and tail on Plain text files
  - Ingest PCAP or DNSTap files by watching a directory
Loggers
- Local storage of your DNS logs in text or binary formats
  - Stdout console in text or binary output
  - File with automatic rotation and compression
- Provide metrics and API
  - Prometheus exporter
  - Statsd support
  - REST API with swagger to search DNS domains
- Send to remote host with generic transport protocol
  - Raw TCP client
  - Syslog with TLS support
  - DNSTap protobuf client
- Send to various sinks
  - Fluentd
  - InfluxDB
  - Loki client
  - ElasticSearch
  - Scalyr
  - Redis publisher
  - Kafka producer
  - ClickHouse client (not yet production ready)
- Send to security tools
  - Falco
Transformers
- Custom Relabeling for JSON structure
- Add additionnal Tags
- Traffic Filtering and Reducer
- Latency Computing
- Apply User Privacy
- Normalize DNS messages
- Add Geographical metadata
- Various data Extractor
- Suspicious traffic Detector and Prediction

Get Started

Download the latest release binary and start the DNS-collector with the provided configuration file. The default configuration listens on tcp/6000 for a DNSTap stream and DNS logs are printed on standard output.

./go-dnscollector -config config.yml

If you prefer run it from docker, follow this guide.

Configuration

The configuration of DNS-collector is done through a file named config.yml. When the DNS-collector starts, it will look for the config.yml from the current working directory.

See the full configuration guide for more details.

Run the DNS-collector in dry mode to verify the configuration.

./go-dnscollector -config config.yml -test-config
INFO: 2023/12/24 14:43:29.043730 main - config OK!

Usage examples

The _examples folder from documentation contains a number of various configurations to get you started with the DNS-collector in different ways.

The _integration folder contains DNS-collector configuration files and docker compose examples for popular tools:

Performance

Tuning may be necessary to deal with a large traffic loads. Please refer to the performance tuning guide if needed.

Contributing

See the development guide for more information on how to build it yourself.

go-dnscollector's People

Contributors

Stargazers

Watchers

go-dnscollector's Issues

syslog format: json

Please add format:json to syslog output. dnstap-receiver can do it.

Thx
csszep

return suspicious client ip list

return suspicious client ip list from rest api
a client ip is tagged suspicious if malformed dns packet is received

[dnstap logger] Atoi error with dnstap logger

Got error with dnstap logger enabled:

ERROR: 2022/09/04 09:08:13.989261 logger dnstap sender - error to encode dnstap response port strconv.Atoi: parsing "-": invalid syntax

Fault on initial install on MacOS Big Sur (others?)

Attempting to install from scratch on new system after a fresh "git clone":

zot12-2:dns-collector jtodd$ go run .

github.com/dmachard/go-dnscollector/collectors

collectors/dnssniffer.go:86:11: undefined: unix.SockFprog
collectors/dnssniffer.go:91:9: undefined: unix.SetsockoptSockFprog
collectors/dnssniffer.go:91:58: undefined: syscall.SO_ATTACH_FILTER
collectors/dnssniffer.go:95:55: undefined: syscall.SO_DETACH_FILTER
collectors/dnssniffer.go:167:28: undefined: syscall.AF_PACKET
collectors/dnssniffer.go:167:71: undefined: syscall.ETH_P_ALL
collectors/dnssniffer.go:179:9: undefined: syscall.SockaddrLinklayer
collectors/dnssniffer.go:189:54: undefined: syscall.SO_TIMESTAMPNS
collectors/dnssniffer.go:246:26: undefined: syscall.SCM_TIMESTAMPNS
zot12-2:dns-collector jtodd$ go version
go version go1.17.6 darwin/amd64
zot12-2:dns-collector jtodd$

rdata for format: text

Please add rdata to text-format fields.

Thx
csszep

ioutil.ReadDir/ReadFile is deprecated since go 1.16

"io/ioutil" has been deprecated since Go 1.16: As of Go 1.16, the same functionality is now provided by package io or package os, and those implementations should be preferred in new code. See the specific function documentation for details.

Loggers prometheus, logfile and pcapfile impacted

files, err := ioutil.ReadDir(o.filedir)

caCert, err = ioutil.ReadFile(s.config.Loggers.Prometheus.CertFile)

// Create a CA certificate pool and add cert.pem to it
var caCert []byte
caCert, err = ioutil.ReadFile(s.config.Loggers.Prometheus.CertFile)
if err != nil {
s.logger.Fatal(err)
}

More info https://pkg.go.dev/io/ioutil#ReadDir

Possibility to configure output fields format.

Hi there,

Thank you for this new wonderful application. I found for my self that it will be really great to have option to define output format for logs in configuration settings.

Like as we have for nginx or apache server.

Is it possible to implement it in new dnscollector?

TCP encoding for PCAP logger - truncated response

Regarding #157 (comment):
Having a PCAP logger and enforcing the query with TCP (dig +tcp ...) or with a response, which must be truncated, then we only see the TCP query and no response.

For example a truncated response (from PCAP logger-output, shown with tshark)

Querying the server (dig @server www.example.ch AAAA)

Frame 3: 97 bytes on wire (776 bits), 97 bytes captured (776 bits)
Ethernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)
Internet Protocol Version 4, Src: 10.100.2.62, Dst: 10.100.102.21
User Datagram Protocol, Src Port: 40772, Dst Port: 53
Domain Name System (query)
    Transaction ID: 0xf7b2
    Flags: 0x0120 Standard query
        0... .... .... .... = Response: Message is a query
        .000 0... .... .... = Opcode: Standard query (0)
        .... ..0. .... .... = Truncated: Message is not truncated
        .... ...1 .... .... = Recursion desired: Do query recursively
        .... .... .0.. .... = Z: reserved (0)
        .... .... ..1. .... = AD bit: Set
        .... .... ...0 .... = Non-authenticated data: Unacceptable
    Questions: 1
    Answer RRs: 0
    Authority RRs: 0
    Additional RRs: 1
    Queries
        www.example.ch: type AAAA, class IN
            Name: www.example.ch
            [Name Length: 14]
            [Label Count: 3]
            Type: AAAA (IPv6 Address) (28)
            Class: IN (0x0001)
    Additional records
        <Root>: type OPT
            Name: <Root>
            Type: OPT (41)
            UDP payload size: 1232
            Higher bits in extended RCODE: 0x00
            EDNS0 version: 0
            Z: 0x0000
                0... .... .... .... = DO bit: Cannot handle DNSSEC security RRs
                .000 0000 0000 0000 = Reserved: 0x0000
            Data length: 12
            Option: COOKIE
                Option Code: COOKIE (10)
                Option Length: 8
                Option Data: e8852c36809558c8
                Client Cookie: e8852c36809558c8
                Server Cookie: <MISSING>

UDP-Response with Truncated-Flag set:

Frame 4: 113 bytes on wire (904 bits), 113 bytes captured (904 bits)
Ethernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)
Internet Protocol Version 4, Src: 10.100.102.21, Dst: 10.100.2.62
User Datagram Protocol, Src Port: 53, Dst Port: 40772
Domain Name System (response)
    Transaction ID: 0xf7b2
    Flags: 0x8780 Standard query response, No error
        1... .... .... .... = Response: Message is a response
        .000 0... .... .... = Opcode: Standard query (0)
        .... .1.. .... .... = Authoritative: Server is an authority for domain
        .... ..1. .... .... = Truncated: Message is truncated
        .... ...1 .... .... = Recursion desired: Do query recursively
        .... .... 1... .... = Recursion available: Server can do recursive queries
        .... .... .0.. .... = Z: reserved (0)
        .... .... ..0. .... = Answer authenticated: Answer/authority portion was not authenticated by the server
        .... .... ...0 .... = Non-authenticated data: Unacceptable
        .... .... .... 0000 = Reply code: No error (0)
    Questions: 1
    Answer RRs: 0
    Authority RRs: 0
    Additional RRs: 1
    Queries
        www.example.ch: type AAAA, class IN
            Name: www.example.ch
            [Name Length: 14]
            [Label Count: 3]
            Type: AAAA (IPv6 Address) (28)
            Class: IN (0x0001)
    Additional records
        <Root>: type OPT
            Name: <Root>
            Type: OPT (41)
            UDP payload size: 1232
            Higher bits in extended RCODE: 0x00
            EDNS0 version: 0
            Z: 0x0000
                0... .... .... .... = DO bit: Cannot handle DNSSEC security RRs
                .000 0000 0000 0000 = Reserved: 0x0000
            Data length: 28
            Option: COOKIE
                Option Code: COOKIE (10)
                Option Length: 24
                Option Data: e8852c36809558c80100000063735e1a4c3e81b1c4692d53
                Client Cookie: e8852c36809558c8
                Server Cookie: 0100000063735e1a4c3e81b1c4692d53
    [Request In: 3]
    [Time: 0.000306000 seconds]

PCAP-Logging output shows only the TCP query

Frame 5: 127 bytes on wire (1016 bits), 127 bytes captured (1016 bits)
Ethernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)
Internet Protocol Version 4, Src: 10.100.2.62, Dst: 10.100.102.21
Transmission Control Protocol, Src Port: 36295, Dst Port: 53, Seq: 1, Len: 73
Domain Name System (query)
    Length: 71
    Transaction ID: 0x3119
    Flags: 0x0120 Standard query
        0... .... .... .... = Response: Message is a query
        .000 0... .... .... = Opcode: Standard query (0)
        .... ..0. .... .... = Truncated: Message is not truncated
        .... ...1 .... .... = Recursion desired: Do query recursively
        .... .... .0.. .... = Z: reserved (0)
        .... .... ..1. .... = AD bit: Set
        .... .... ...0 .... = Non-authenticated data: Unacceptable
    Questions: 1
    Answer RRs: 0
    Authority RRs: 0
    Additional RRs: 1
    Queries
        www.example.ch: type AAAA, class IN
            Name: www.example.ch
            [Name Length: 14]
            [Label Count: 3]
            Type: AAAA (IPv6 Address) (28)
            Class: IN (0x0001)
    Additional records
        <Root>: type OPT
            Name: <Root>
            Type: OPT (41)
            UDP payload size: 1232
            Higher bits in extended RCODE: 0x00
            EDNS0 version: 0
            Z: 0x0000
                0... .... .... .... = DO bit: Cannot handle DNSSEC security RRs
                .000 0000 0000 0000 = Reserved: 0x0000
            Data length: 28
            Option: COOKIE
                Option Code: COOKIE (10)
                Option Length: 24
                Option Data: e8852c36809558c80100000063735e1a4c3e81b1c4692d53
                Client Cookie: e8852c36809558c8
                Server Cookie: 0100000063735e1a4c3e81b1c4692d53

The complete dig response looks like this:

$ dig @test www.example.ch AAAA
;; Truncated, retrying in TCP mode.

; <<>> DiG 9.18.7 <<>> @test www.example.ch AAAA
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12569
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 222, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: e8852c36809558c80100000063735e1a4c3e81b1c4692d53 (good)
;; QUESTION SECTION:
;www.example.ch.			IN	AAAA

;; ANSWER SECTION:
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:dd00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:f500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ce00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:f600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:eb00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:cd00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:fe00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:f200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ed00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:fa00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8b00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:f800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ae00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ab00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3b00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:fd00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:bb00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:be00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6b00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4b00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9b00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1b00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:da00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ac00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:aa00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:1600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:cc00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ba00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:a200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:7c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9a00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:f300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:f900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:9300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ca00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:f100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:bd00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:bc00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5b00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:cb00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:dc00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5200::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:2b00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8d00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:4c00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:c000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:db00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:8500::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:b300::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:e600::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ea00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:f000::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:5100::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ec00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:6800::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ee00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:ad00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:f400::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:fc00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:de00::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:3900::affe
www.example.ch.		1800	IN	AAAA	2001:db8:cafe:d600::affe

;; Query time: 0 msec
;; SERVER: 10.100.102.21#53(test) (TCP)
;; WHEN: Tue Nov 15 10:38:34 CET 2022
;; MSG SIZE  rcvd: 6287

Include subnet information for: dnscollector_requesters_top_total

It's also will be useful to have some additional statistics to analyze number queries from similar subnets.
Example:

dnscollector_requesters_top_total{stream="beta-vm01",ip="172.253.206.36"} 38 
dnscollector_requesters_top_total{stream="beta-vm01",ip="178.20.157.236"} 35
dnscollector_requesters_top_total{stream="beta-vm01",ip="172.253.255.35"} 34
dnscollector_requesters_top_total{stream="beta-vm01",ip="178.20.156.245"} 33
dnscollector_requesters_top_total{stream="beta-vm01",ip="148.66.138.87"} 33
dnscollector_requesters_top_total{stream="beta-vm01",ip="114.119.176.141"} 31
dnscollector_requesters_top_total{stream="beta-vm01",ip="165.227.94.23"} 31
dnscollector_requesters_top_total{stream="beta-vm01",ip="172.253.1.133"} 30
dnscollector_requesters_top_total{stream="beta-vm01",ip="172.253.255.37"} 30
dnscollector_requesters_top_total{stream="beta-vm01",ip="172.253.1.132"} 30
dnscollector_requesters_top_total{stream="beta-vm01",ip="172.253.255.33"} 26
dnscollector_requesters_top_total{stream="beta-vm01",ip="172.253.206.33"} 26
dnscollector_requesters_top_total{stream="beta-vm01",ip="172.253.206.37"} 25

I am pretty sure:
172.253.206.36, 172.253.206.33, 172.253.206.37 it's same subnet range

If geoip module for AS information enabled, is it possible to add additional label, like as: subnet_range
after that, I will be able over grafana calculate number of queries from same subnet range.

probably, same feature can be useful for another metrics.

what is the goal?
I just would like to have some system, which will give possibility to recognize attack to our DNS infrastructure.

Partition Loki streams by peer's identity

I think that partitioning the streams by the peer's—or DNS server's—identity would likely benefit many users.
This feature would speed up drilling down by DNS server without Loki needing to process the other server's logs.

dnscollector crashed

Sorry, it's difficult to find any good output, just got from collector logs:

`INFO: 2021/12/04 22:00:47.482078 processor dnstap parser - running... waiting incoming dns message
INFO: 2021/12/04 22:00:47.483445 collector dnstap - receiver framestream initialized
INFO: 2021/12/04 22:00:47.582560 collector dnstap - 10.5.2.5:51020 - connection closed
INFO: 2021/12/04 22:00:47.974034 collector dnstap - 10.5.2.5:51024 - new connection
INFO: 2021/12/04 22:00:47.974083 dnstap processor - initialization...
INFO: 2021/12/04 22:00:47.974499 processor dnstap parser - geoip is enabled
INFO: 2021/12/04 22:00:47.974519 processor dnstap parser - running... waiting incoming dns message
INFO: 2021/12/04 22:00:47.976095 collector dnstap - receiver framestream initialized
panic: runtime error: slice bounds out of range [:26] with capacity 24

goroutine 33916470 [running]:
github.com/dmachard/go-dnscollector/subprocessors.DecodeAnswer(0x8767, 0xc0000315f0, {0xc005b2a5d0, 0x16, 0x18})
/build/subprocessors/dnsparser.go:436 +0x451
github.com/dmachard/go-dnscollector/subprocessors.(*DnstapProcessor).Run(0xc0003a7de0, {0xc0001fe6f0, 0x2, 0x2})
/build/subprocessors/dnstapparser.go:212 +0xed9
created by github.com/dmachard/go-dnscollector/collectors.(*Dnstap).HandleConn
/build/collectors/dnstap.go:69 +0x265
INFO: 2021/12/05 17:50:08.043053 main - config loaded...
INFO: 2021/12/05 17:50:08.043162 main - starting dnslogger...
INFO: 2021/12/05 17:50:08.043179 webserver - enabled
INFO: 2021/12/05 17:50:08.043261 logger logfile - enabled
INFO: 2021/12/05 17:50:08.043465 collector dnstap - enabled
INFO: 2021/12/05 17:50:08.044879 main - running all collectors and loggers...
INFO: 2021/12/05 17:50:08.044924 collector dnstap - starting collector...
INFO: 2021/12/05 17:50:08.045052 webserver - running in background...`

DoT/DoH-Queries and responses are not logged in pcap-logging-mode

Using the latest go-dns-collector (0.24.0), DoH and DoT queries AND responses are not logged in PCAP logging mode. In the stdout and logfile-mode, these queries are logged correctly.

Output from stdout and the logfile...

#DOH
2022-11-10T07:45:11.365021544Z pc CLIENT_QUERY NOERROR 10.100.2.47 35675 INET DOH 58b www.microsoft.com A 0.000000
2022-11-10T07:45:11.365342574Z pc CLIENT_RESPONSE NOERROR 10.100.2.47 35675 INET DOH 247b www.microsoft.com A 0.000000

#DOT
2022-11-10T07:49:14.405353046Z pc CLIENT_QUERY NOERROR 10.100.2.47 44411 INET DOT 58b www.microsoft.com A 0.000000
2022-11-10T07:49:14.410022956Z pc CLIENT_RESPONSE NOERROR 10.100.2.47 44411 INET DOT 247b www.microsoft.com A 0.000000

..., but the PCAP is empty for DoH and DoT-queries.

My current config:

multiplexer:
  collectors:
    - name: dnstap_remote
      dnstap:
        sock-path: /run/dnsdist/unix_remote.sock
        tls-support: false
      transforms:
        filtering:
          log-queries: true
          log-replies: true

  loggers:
    - name: logfile
      logfile:
       enable: true
       file-path: /var/log/dns/dns.log
       mode: text

    - name: console
      stdout:
        mode: text

    - name: pcap
      pcapfile:
        file-path: /var/log/dns/dns.cap
        max-size: 100
        max-files: 10
        compress: false

  routes:
    - from: [ dnstap_remote ]
      to: [ console, logfile, pcap ]

Target is no longer accessible from Prometheus with the "expected label name, got "INVALID" error

The Prometheus polling falls off in case of a badly formed metric.
The error 'expected label name, got "INVALID' appears in the target section of Prometheus.

powerdns collector: not working and not sure why

Hi,

I'm using the following configuration as a proof of concept to see if it will log to a file before I attempt to log to InfluxDB.

global:
  trace:
    verbose: true
    log-malformed: true

multiplexer:
  collectors:
    - name: protobuf
      powerdns:
        listen-ip: 10.0.0.20
        listen-port: 6000
        quiet-text: false
        tls-support: true
        cert-file: "/etc/dnscollector/server.domain.com.crt"
        key-file: "/etc/dnscollector/server.domain.com.key"

  loggers:
    - name: file
      logfile:
        enable: true
        file-path: "/var/run/dnscollector/powerdns.log"
        max-size: 100
        max-files: 10
        mode: text
        flush-interval: 1
        compress: false
        compress-interval: 5
        compress-command: null
        text-format: ""
        postrotate-command: null
        postrotate-delete-success: false

  routes:
    - from: [ protobuf ]
      to: [ file ]

In the logs I see:

Oct 07 01:32:50 server.domain.com systemd[1]: Started Go DnsCollector.
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.330411 main - version 0.23.0
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.330459 main - starting dns-collector...
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.330464 main - loading loggers...
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.330634 [file] logger logfile - enabled
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.330677 main - loading collectors...
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.330755 [protobuf] pdns collector - enabled
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.330759 main - routing: collector[protobuf] send to logger[file]
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.330944 main - running all collectors and loggers...
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.330972 [protobuf] pdns collector - starting collector...
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.331000 [protobuf] pdns collector - running in background...
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.331002 [protobuf] pdns collector - tls support enabled
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.331173 [file] logger to file - running in background...
Oct 07 01:32:50 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:32:50.331479 [protobuf] pdns collector - is listening on 10.0.0.20:6000
Oct 07 01:33:16 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:33:16.769043 [protobuf] pdns collector - 10.0.0.29:55206 - new connection
Oct 07 01:33:16 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:33:16.769060 [protobuf] powerdns processor - initialization...
Oct 07 01:33:16 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:33:16.769066 [protobuf] processor powerdns parser - config
Oct 07 01:33:16 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:33:16.769362 [protobuf] processor powerdns parser - running... waiting incoming dns message
Oct 07 01:33:21 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:33:21.769656 [protobuf] pdns collector - 10.0.0.29:54000 - new connection
Oct 07 01:33:21 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:33:21.769673 [protobuf] powerdns processor - initialization...
Oct 07 01:33:21 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:33:21.769700 [protobuf] processor powerdns parser - config
Oct 07 01:33:21 server.domain.com go-dnscollector[1141940]: INFO: 2022/10/07 01:33:21.769955 [protobuf] processor powerdns parser - running... waiting incoming dns message

But nothing is logged to file, despite reporting the inbound connections.

The recursor.lua config on the DNS server:

protobufServer("10.0.0.20:6000", {
  taggedOnly=false,
  logQueries=true,
  logResponses=false,
  exportTypes={'A', 'AAAA', 'CNAME', 'MX', 'NS', 'PTR', 'SPF', 'SRV', 'TXT'}
})

rest api: dump full list of clients ip and domains

Optimization number of generated metrics

After a lot of my requests, now I see a big list of metrics, some of them can be combined with each other, example:

dnscollector_as_numbers_top_total{stream="alpha-vm01",number="15169"} 4259
dnscollector_as_owners_top_total{stream="alpha-vm01",owner="GOOGLE"} 4259

dnscollector_as_numbers_top_total{stream="alpha-vm01",number="14618"} 1222
dnscollector_as_owners_top_total{stream="alpha-vm01",owner="AMAZON-AES"} 1222

can be merged by labels

likes as:
dnscollector_as_stats{stream="alpha-vm01",as_number="14618",as_owner="GOOGLE"} 4259
dnscollector_as_stats{stream="alpha-vm01",as_number="14618",as_owner="AMAZON-AES"} 1222

Replace identiy/identity in code

Probably there are 2 mistakes in code and need replace identiy to identity

as example here:

go-dnscollector/dnsutils/message.go

Line 18 in e503fc1

Identity string `json:"identiy" msgpack:"identiy"`

prom2json failed

Hi, I tried integrate zabbix + go-dnscollector, but faced with parsing error. I used prom2json to check what can be wrong.

Steps to reproduce:

curl  -q -u admin:changeme http://10.5.0.107:9990/metrics > dns_metrics.txt

docker run -it -v /root/metrics:/metrics.txt golang:latest
GO111MODULE=on go install github.com/prometheus/prom2json/cmd/prom2json@latest

prom2json /metrics.txt
FATA[0000] error reading metrics:reading text format failed: text format parsing error in line 383: second HELP line for metric name "dnscollector_requesters_total"  source="main.go:73"

Could you please check.

Issue regarding logging of EDNS client subnet sent from Powerdns recursor

Issue regarding capturing of DNS request sent from Powerdns recursor via protobufServer .
I can confirm that EDNS data sent from Powerdns recursor via protobufserver is working as the data has been observed via wireshark.

TCP remote logger issue

I run the collector in another machine that the LB.

But when collector receive a frame the error message is logged

INFO: 2021/11/08 18:06:55.225441 collector dnstap - ip:34116 - new connection
INFO: 2021/11/08 18:06:55.225563 dnstap processor - initialization...
INFO: 2021/11/08 18:06:55.225727 processor dnstap parser - running... waiting incoming dns message
ERROR: 2021/11/08 18:06:56.225753 collector dnstap - error stream receiver initialization: frame too large error

new metrics to count malformed packet

cf #16

Resolve ip to ASN

Hi @dmachard

First of all, I would like to say: thank you for this wonderful new collector, it works really well and collector has really good performance.
I would like to ask you about new feature, which probably is not related exactly to collector directly, but really useful for us to develop new feature for block dns attach from some specific ASN.

It will be really useful to have some additional information about top source ip requesters which will include also subnet range from whois and asn information as additional labels:

https://github.com/ammario/ipisp

also, I guess it will raise a performance issue and probably one of the solution will be integrate go-dnscollector with redis db for some cache options.

Loggers, text mode doesn't include geoip information

Text mode for loggers in stdout doesn't include any geoip configuration

# list of loggers
loggers:

  stdout:
    # to enable, set the enable to true
    enable: true
    # output format: text|json
    mode: text

2021-09-04T19:11:56.814486033Z beta-vm03 CLIENT_RESPONSE NXDOMAIN 188.2.193.164 55966 INET UDP 120b smtp.aloup.com.ua A 0.002530
2021-09-04T19:11:56.815125065Z beta-vm03 CLIENT_QUERY NOERROR 8.0.1.19 16146 INET UDP 50b xn--h1aidq.xn--p1ai A 0.000000
2021-09-04T19:11:56.816296045Z beta-vm03 CLIENT_RESPONSE NOERROR 8.0.1.19 16146 INET UDP 66b xn--h1aidq.xn--p1ai A 0.001171

But in the same time, when I defined mode: json, json include geoip field.

Feature request: Support for resolver label in prometheus logger

Hey,

I've started using the DNS sniffer collector which is working really well. I was wondering if there could be an option to add a label for what resolver is used in the metrics exposed by prometheus. I think the data is captured already (ResponseIP) but this would increase cardinality which is why I'm not convinced it should be enabled by default.

Get the version of the go-dnscollector binary

Hello Denis,

Can you add a solution to get the version of go-dnscollector please?
Either by a command or a Prometheus metric?

Anthony

Add option to increase SO_RCVBUF on dnstap listen socket

hey,

Unbound is super aggressive (for good reasons) in closing the blocking TCP sockets should there be even a tiny receive queue on collector side (due to IOPS peaks, for example).

Perhaps add config option to manually set setReadBuffer on the dnstap collector listening socket so kernel could at least buffer more data in such scenarios?

Add Tags for PowerDNS Protobuf json ouput

Hi,

I have been trying to get Tags from Protobuf but it seems it is not added to the json output. Is it possible to add tags?

I have tested with this dnsdist config:

newServer({address="8.8.8.8", pool="forward"})
cache = newPacketCache(10000)
getPool("forward"):setCache(cache)

addAction(AllRule(), SetTagAction("view", "view-1"))
addResponseAction(AllRule(), SetTagResponseAction("view", "view-1"))

remoteLogger = newRemoteLogger("192.168.0.10:6001", 1, 100, 1)

addAction(AllRule(), RemoteLogAction(remoteLogger))
addResponseAction(AllRule(), RemoteLogResponseAction(remoteLogger))

addAction(AllRule(), PoolAction("forward"))

and this collector config:

# If turned on, log some applications messages
trace:
  # debug informations
  verbose: true
  # log malformed packet
  log-malformed: false
  # filename is the file to write logs to.
  filename: ""
  # maximum size in megabytes of the log file it gets rotated
  max-size: 10
  # maximum number of old log files to retain
  max-backups: 10

collectors:
  powerdns:
    enable: true
    listen-ip: 0.0.0.0
    listen-port: 6001

loggers:
  stdout:
    enable: true
    mode: json
    text-format: ""

and the collector output is:

{
  "network": {
    "family": "INET",
    "protocol": "UDP",
    "query-ip": "192.168.64.9",
    "query-port": "41722",
    "response-ip": "192.168.64.9",
    "response-port": "53",
    "as-number": "-",
    "as-owner": "-"
  },
  "dns": {
    "length": 54,
    "opcode": 0,
    "rcode": "-",
    "qname": "www.google.fr",
    "qname-public-suffix": "fr",
    "qname-effective-tld-plus-one": "google.fr",
    "qtype": "A",
    "flags": {
      "qr": false,
      "tc": false,
      "aa": false,
      "ra": false,
      "ad": false
    },
    "resource-records": {
      "an": [],
      "ns": [],
      "ar": []
    },
    "malformed-packet": 0
  },
  "edns": {
    "udp-size": 0,
    "rcode": 0,
    "version": 0,
    "dnssec-ok": 0,
    "options": []
  },
  "dnstap": {
    "operation": "DNSQueryType",
    "identity": "",
    "timestamp-rfc3339ns": "2022-04-08T14:04:26.000517095Z",
    "latency": "-"
  },
  "geo": {
    "city": "-",
    "continent": "-",
    "country-isocode": "-"
  }
}

{
  "network": {
    "family": "INET",
    "protocol": "UDP",
    "query-ip": "192.168.64.9",
    "query-port": "41722",
    "response-ip": "192.168.64.9",
    "response-port": "53",
    "as-number": "-",
    "as-owner": "-"
  },
  "dns": {
    "length": 58,
    "opcode": 0,
    "rcode": "NOERROR",
    "qname": "www.google.fr",
    "qname-public-suffix": "fr",
    "qname-effective-tld-plus-one": "google.fr",
    "qtype": "A",
    "flags": {
      "qr": false,
      "tc": false,
      "aa": false,
      "ra": false,
      "ad": false
    },
    "resource-records": {
      "an": [
        {
          "name": "www.google.fr.",
          "rdatatype": "A",
          "ttl": 283,
          "rdata": "142.250.179.99"
        }
      ],
      "ns": [],
      "ar": []
    },
    "malformed-packet": 0
  },
  "edns": {
    "udp-size": 0,
    "rcode": 0,
    "version": 0,
    "dnssec-ok": 0,
    "options": []
  },
  "dnstap": {
    "operation": "DNSResponseType",
    "identity": "",
    "timestamp-rfc3339ns": "2022-04-08T14:04:26.000541422Z",
    "latency": "0.026901"
  },
  "geo": {
    "city": "-",
    "continent": "-",
    "country-isocode": "-"
  }
}

json log duplicates

using v0.10.0 or v0.11.0 go-collector binary with this config:

trace:
verbose: false

collectors:
dnstap:
enable: true
listen-ip: 0.0.0.0
listen-port: 6000
tls-support: false

loggers:
logfile:
enable: true
file-path: "/var/log/dnstap.log"
max-size: 100
max-files: 20
mode: json

I am getting massive amounts of duplicates. I also tried:

syslog:
enable: true
severity: INFO
facility: DAEMON
transport: local
# Remote address host:port
# remote-address: ""
# output text format, please refer to the default text format to see all available directives
# use this parameter if you want a specific format
# text-format: ""
# output format: text|json
mode: json

and curiously I am getting duplicates as well, however they are in a strange staircase format where the duplicate entry gets appended at the end of the previous one for 2-5 times.

any tips on how to troubleshoot this?

my source is dnsdist v1.5.1

using this as config:
--from https://dmachard.github.io/posts/0001-dnstap-testing/#tcp-stream
fsul = newFrameStreamTcpLogger("xxx.xxx.xxx.xxx:6000")
addAction(AllRule(), DnstapLogAction("dnsdist", fsul))
addResponseAction(AllRule(), DnstapLogResponseAction("dnsdist", fsul))
-- Cache Hits
addCacheHitResponseAction(AllRule(), DnstapLogResponseAction("dnsdist", fsul))

Need to output answered/resolved ip to syslog

At the moment it seems the answer is not accessible to the loggers. All I am seeing is the ip address of the forwarder DNS server when I want access to the resolved IP for the query. I can see Bind is sending this via dnstap but can't seem to get the syslog logger to output it. What am I missing?

Request: Add RPZ policy name

Apologies, raised an issue when I should have been adding a discussion instead...

See #130

pcap file collector source?

Would it be possible to implement collecting DNS traffic from a pcap file in addition to the live sniffing of traffic?

Appreciate there's not a lot to go on here.

Convert all domain names to lower case

I found that this wonderful magic tool was extended by new metrics, which possible to retrieve over curl, but also I found duplicate for some domains

dnscollector_domains_slow_top{stream="global",domain="INVErITA.COM"} 2
dnscollector_domains_slow_top{stream="global",domain="INVERItA.COM"} 2
dnscollector_domains_slow_top{stream="global",domain="iNvErITA.COM"} 2

everything of this is the same domain, it will be good to have an option in configuration file, which convert all domain to lowercase or upper case.

GeoIP database seems not working with Unbound DNSTAP input

Hi everyone.

I'm using the latest release version 0.25.0-beta1 (from GitHub releases) along with Unbound 1.13.1 (from Ubuntu APT) act as a forwarder and cacher for our team. I'm using fluentd as the output pipe and do some sort of field hacks (that's another story), and eventually put all DNS logs into an elasticsearch instance.

In short, the problem occurs as I'm planning to add GeoIP statistics for our DNS logs. I'm using mmdb files from GeoLite repository, and the startup logs of go-dns-collector shows that the GeoIP databases seemes loaded correctly. However, I can't find any recognized ASNs, country codes, or cities in the output stream, no matter I'm using text output nor fluentd output.

Here shows my go-dns-collector configuration.

multiplexer:
  collectors:
    - name: tap
      dnstap:
        listen-ip: 0.0.0.0
        listen-port: 10053
        tls-support: false
        cache-support: true
      transforms:
        geoip:
          mmdb-country-file: "GeoLite2-Country.mmdb"
          mmdb-city-file: "GeoLite2-City.mmdb"
          mmdb-asn-file: "GeoLite2-ASN.mmdb"

  loggers:
    - name: console
      stdout:
        mode: text
        text-format: "timestamp-rfc3339ns identity country city as-number operation rcode queryip queryport family protocol length qname qtype answer aa latency"
    - name: fluentd
      fluentd:
        transport: tcp
        remote-address: 127.0.0.1
        remote-port: 24224
        retry-interval: 5
        tag: "dns.collector"
        tls-support: false
        tls-insecure: false
  routes:
    - from: [ tap ]
      to: [ fluentd, console ]

Here's my Unbound configuration related to DNSTAP:

dnstap:
    dnstap-enable: yes
    dnstap-ip: 10.201.10.33@10053
    dnstap-tls: no
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-resolver-query-messages: yes
    dnstap-log-resolver-response-messages: yes
    dnstap-log-client-query-messages: yes
    dnstap-log-client-response-messages: yes
    dnstap-log-forwarder-query-messages: yes
    dnstap-log-forwarder-query-messages: yes

Here're startup logs of go-dns-collector.

INFO: 2022/11/10 16:34:03.561112 main - version 0.25.0-beta1
INFO: 2022/11/10 16:34:03.561169 main - starting dns-collector...
INFO: 2022/11/10 16:34:03.561171 main - loading loggers...
INFO: 2022/11/10 16:34:03.561290 [console] logger stdout - enabled
INFO: 2022/11/10 16:34:03.561424 [fluentd] logger to fluentd - enabled
INFO: 2022/11/10 16:34:03.561459 main - loading collectors...
INFO: 2022/11/10 16:34:03.561571 [tap] dnstap collector - enabled
INFO: 2022/11/10 16:34:03.561583 main - routing: collector[tap] send to logger[fluentd]
INFO: 2022/11/10 16:34:03.561984 main - running all collectors and loggers...
INFO: 2022/11/10 16:34:03.562034 [tap] dnstap collector - starting collector...
INFO: 2022/11/10 16:34:03.562169 [tap] dnstap collector - running in background...
INFO: 2022/11/10 16:34:03.562237 [console] logger to stdout - running in background...
INFO: 2022/11/10 16:34:03.562435 [fluentd] logger to fluentd - running in background...
INFO: 2022/11/10 16:34:03.562493 [tap] dnstap collector - is listening on [::]:10053
INFO: 2022/11/10 16:34:03.562506 [fluentd] logger to fluentd - connecting to 127.0.0.1:24224
INFO: 2022/11/10 16:34:03.562810 [fluentd] logger to fluentd - connected
INFO: 2022/11/10 16:34:03.566863 [tap] dnstap collector - <MASKED>:55960 - new connection
INFO: 2022/11/10 16:34:03.566897 [tap] dnstap processor - initialization...
INFO: 2022/11/10 16:34:03.567236 [tap] dnstap processor - dns cached enabled: true
INFO: 2022/11/10 16:34:03.567440 [tap] dnstap collector - receiver framestream initialized
INFO: 2022/11/10 16:34:03.567533 Subprocessor GeoIP - country database loaded (919324 records)
INFO: 2022/11/10 16:34:03.567607 Subprocessor GeoIP - city database loaded (5026343 records)
INFO: 2022/11/10 16:34:03.567697 Subprocessor GeoIP - asn database loaded (939652 records)
INFO: 2022/11/10 16:34:03.567722 [tap] dnstap processor - geoip is enabled
INFO: 2022/11/10 16:34:03.567998 [tap] dnstap processor - running... waiting incoming dns message

Well, here's some text output of actual DNSTAP messages being parsed.

2022-11-10T08:48:51.799571Z sh-dns-02 - - - FORWARDER_QUERY NOERROR - - INET UDP 49b gslb-pek01.zhihu.com A 0.000000
2022-11-10T08:48:51.801018Z sh-dns-02 - - - FORWARDER_QUERY NOERROR - - INET UDP 57b e4094fc1d98c915a.ksyunad.com A 0.000000
2022-11-10T08:48:51.802444Z sh-dns-02   0 CLIENT_RESPONSE NOERROR 10.201.18.17 57490 INET UDP 219b captcha.zhihu.com A 0.008948
2022-11-10T08:48:51.813236Z sh-dns-02   0 CLIENT_QUERY NOERROR 10.201.33.11 56326 INET UDP 32b mesu.apple.com HTTPS 0.000000
2022-11-10T08:48:51.813579Z sh-dns-02   0 CLIENT_RESPONSE NOERROR 10.201.33.11 56326 INET UDP 190b mesu.apple.com HTTPS 0.000343
2022-11-10T08:48:51.942343Z sh-dns-02   0 CLIENT_QUERY NOERROR 10.201.18.17 49624 INET UDP 32b da.dun.163.com A 0.000000
2022-11-10T08:48:51.942724Z sh-dns-02 - - - FORWARDER_QUERY NOERROR - - INET UDP 43b da.dun.163.com A 0.000000
2022-11-10T08:48:51.944953Z sh-dns-02   0 CLIENT_RESPONSE NOERROR 10.201.18.17 49624 INET UDP 48b da.dun.163.com A 0.002610

Interestingly, from the output I noticed there is no "AA" nor "answer" field has been displayed.
So what field does dnstap-collector takes to resolve GeoIP information? Is this a problem with Unbound?

missing rdata in answers

Hi!

When i try to query gmail.com MX records i get:

dig @127.0.0.1 gmail.com MX

; <<>> DiG 9.16.22-Debian <<>> @127.0.0.1 gmail.com MX
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58347
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: dac713c60d021e8e01000000619dfbb31521462b3eaaf61d (good)
;; QUESTION SECTION:
;gmail.com. IN MX

;; ANSWER SECTION:
gmail.com. 1916 IN MX 10 alt1.gmail-smtp-in.l.google.com.
gmail.com. 1916 IN MX 5 gmail-smtp-in.l.google.com.
gmail.com. 1916 IN MX 40 alt4.gmail-smtp-in.l.google.com.
gmail.com. 1916 IN MX 30 alt3.gmail-smtp-in.l.google.com.
gmail.com. 1916 IN MX 20 alt2.gmail-smtp-in.l.google.com.

;; Query time: 4 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Nov 24 09:45:39 CET 2021
;; MSG SIZE rcvd: 192

but in syslog json output there is no rdata:

{
"operation": "CLIENT_RESPONSE",
"identity": "dns-bind",
"family": "INET",
"protocol": "UDP",
"query-ip": "127.0.0.1",
"query-port": "49191",
"response-ip": "127.0.0.1",
"response-port": "-",
"length": 276,
"rcode": "NOERROR",
"qname": "gmail.com",
"qtype": "MX",
"latency": "0.000000",
"timestamp-rfc3339": "2021-11-24T08:33:03.922233802Z",
"answers": [
{
"name": "gmail.com",
"rdatatype": "MX",
"ttl": 2672,
"rdata": "-"
},
{
"name": "gmail.com",
"rdatatype": "MX",
"ttl": 2672,
"rdata": "-"
},
{
"name": "gmail.com",
"rdatatype": "MX",
"ttl": 2672,
"rdata": "-"
},
{
"name": "gmail.com",
"rdatatype": "MX",
"ttl": 2672,
"rdata": "-"
},
{
"name": "gmail.com",
"rdatatype": "MX",
"ttl": 2672,
"rdata": "-"
}
],
"country-isocode": "-"
}

bind9 native dnstap-output file:

type: MESSAGE
identity: dns-bind
version: bind
message:
type: CLIENT_RESPONSE
response_time: !!timestamp 2021-11-24T08:41:02Z
message_size: 192b
socket_family: INET
socket_protocol: UDP
query_address: 10.2.66.88
response_address: 10.2.66.88
query_port: 35263
response_port: 0
response_message_data:
opcode: QUERY
status: NOERROR
id: 39250
flags: qr rd ra
QUESTION: 1
ANSWER: 5
AUTHORITY: 0
ADDITIONAL: 1
OPT_PSEUDOSECTION:
EDNS:
version: 0
flags:
udp: 1232
COOKIE: fd13c2857898ee9d01000000619dfa9e9c7b03c68cfac6b7
QUESTION_SECTION:
- gmail.com. IN MX
ANSWER_SECTION:
- gmail.com. 2193 IN MX 5 gmail-smtp-in.l.google.com.
- gmail.com. 2193 IN MX 20 alt2.gmail-smtp-in.l.google.com.
- gmail.com. 2193 IN MX 10 alt1.gmail-smtp-in.l.google.com.
- gmail.com. 2193 IN MX 40 alt4.gmail-smtp-in.l.google.com.
- gmail.com. 2193 IN MX 30 alt3.gmail-smtp-in.l.google.com.
response_message: |
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39250
;; flags: qr rd ra ; QUESTION: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: fd13c2857898ee9d01000000619dfa9e9c7b03c68cfac6b7
;; QUESTION SECTION:
;gmail.com. IN MX

;; ANSWER SECTION:
gmail.com.          2193    IN      MX      5 gmail-smtp-in.l.google.com.
gmail.com.          2193    IN      MX      20 alt2.gmail-smtp-in.l.google.com.
gmail.com.          2193    IN      MX      10 alt1.gmail-smtp-in.l.google.com.
gmail.com.          2193    IN      MX      40 alt4.gmail-smtp-in.l.google.com.
gmail.com.          2193    IN      MX      30 alt3.gmail-smtp-in.l.google.com.

Extend statistics by domain level

Thank you for this really useful tool which give possibility to control dns servers and get statistics, but I would like to ask about extend current statistic by new feature:

Add statistics for count number of queries for first domain level:
com, net and etc...

EDNS decoder issue?

hey,

EDNS decoder seems to be bit optimistic about the OPT RRs, go-dns-collector produces a lot of log like this:

Mar 27 13:44:07 asd.estpak.ee go-dnscollector[119352]: ERROR: 2022/03/27 13:44:07.290620 procesor dnstap parser - malformed edns options in DNS packet: edns, packet contained too many OPT RRs - {{INET UDP - - 45.57.8.1 53 - -} {REPLY [123 151 132 0 0 1 0 2 0 2 0 5 15 111 99 99 45 48 45 49 53 48 48 45 49 53 48 49 1 49 6 110 102 108 120 115 111 3 110 101 116 0 0 1 0 1 192 12 0 1 0 1 0 0 0 30 0 4 45 57 36 151 192 12 0 1 0 1 0 0 0 30 0 4 45 57 36 148 192 30 0 2 0 1 0 1 81 128 0 7 1 101 2 110 115 192 30 192 30 0 2 0 1 0 1 81 128 0 4 1 102 192 92 0 0 41 4 176 0 0 0 0 0 0 192 90 0 1 0 1 0 1 81 128 0 4 45 57 8 1 192 90 0 28 0 1 0 1 81 128 0 16 42 0 134 192 32 8 0 0 0 0 0 0 0 0 0 1 192 109 0 1 0 1 0 1 81 128 0 4 45 57 9 1 192 109 0 28 0 1 0 1 81 128 0 16 42 0 134 192 32 9 0 0 0 0 0 0 0 0 0 1] 212 31639 0 NOERROR occ-0-1500-1501.1.nflxso.net net nflxso.net A {true false true false false} {[{occ-0-1500-1501.1.nflxso.net A 1 30 45.57.36.151} {occ-0-1500-1501.1.nflxso.net A 1 30 45.57.36.148}] [{nflxso.net NS 1 86400 e.ns.nflxso.net} {nflxso.net NS 1 86400 f.ns.nflxso.net}] [{e.ns.nflxso.net A 1 86400 45.57.8.1} {e.ns.nflxso.net AAAA 1 86400 2a00:86c0:2008::1} {f.ns.nflxso.net A 1 86400 45.57.9.1} {f.ns.nflxso.net AAAA 1 86400 2a00:86c0:2009::1}]} 1} {0 0 0 0 0 []} {RESOLVER_RESPONSE mus-dns4.estpak.ee 2022-03-27T13:44:06.780074Z 1.648388646780074e+09 1648388646 780074000 0 -} {- - -}}

Mostly originated by certain auth NSes:

root@asd:/tmp# journalctl -u dnscollector.service -n 10000 | grep "packet contained too many OPT RRs" | grep --color -P "\S+ NS \d+ \d+ \S+" --only-matching  | sort | uniq -c | sort -nr | head -10
   3261 {dradis.netflix.com NS 1 86400 f.ns.nflxso.net}]
   3261 [{dradis.netflix.com NS 1 86400 e.ns.nflxso.net}
   1786 {nflxso.net NS 1 86400 f.ns.nflxso.net}]
   1786 [{nflxso.net NS 1 86400 e.ns.nflxso.net}
    200 {ftl.netflix.com NS 1 86400 f.ns.nflxso.net}]
    200 [{ftl.netflix.com NS 1 86400 e.ns.nflxso.net}
    111 {c10r.instagram.com NS 1 60 d.ns.c10r.instagram.com}]
    111 {c10r.instagram.com NS 1 60 c.ns.c10r.instagram.com}
    111 [{c10r.instagram.com NS 1 60 b.ns.c10r.instagram.com}
    111 {c10r.instagram.com NS 1 60 a.ns.c10r.instagram.com}

It would probably also make sense to handle this as malformed packet. Currently, with log-malformed and verbose set to false, those errors are still logged.

prom logger: panic with invalid prefix in metric name

panic: descriptor Desc{fqName: "dnscollector-dev_build_info", help: "Build version", constLabels: {}, variableLabels: [version]} is invalid: "dnscollector-dev_build_info" is not a valid metric name

rest api: get list of domains queried for the client ip provided

Show the EDNS Client Subnet (ECS) option

Hi @dmachard ,
first of all, thank you for this tool, it's really powerful.

I am interested in seeing the EDNS Client Subnet (ECS) option of the queries I log. Can you add it this info to the loggers? Alternatively, can you add support to see the full "query_message" (so I can see the ECS inside the message) or the DNS query options?

Thanks,
Carlo

0.12.0 crash - stack overflow

Running 0.12.0 binary I get a stack overflow after a few seconds of runtime. Tested on Fedora 35 and vanilla Ubuntu 20.04

Nov 30 22:49:11 protobuf go-dnscollector[2238]: runtime: goroutine stack exceeds 1000000000-byte limit
Nov 30 22:49:11 protobuf go-dnscollector[2238]: runtime: sp=0xc020380390 stack=[0xc020380000, 0xc040380000]
Nov 30 22:49:11 protobuf go-dnscollector[2238]: fatal error: stack overflow
Nov 30 22:49:11 protobuf go-dnscollector[2238]: runtime stack:
Nov 30 22:49:11 protobuf go-dnscollector[2238]: runtime.throw({0x9cb67f, 0xe00220})
Nov 30 22:49:11 protobuf go-dnscollector[2238]: #11/opt/hostedtoolcache/go/1.17.3/x64/src/runtime/panic.go:1198 +0x71
Nov 30 22:49:11 protobuf go-dnscollector[2238]: runtime.newstack()
Nov 30 22:49:11 protobuf go-dnscollector[2238]: #11/opt/hostedtoolcache/go/1.17.3/x64/src/runtime/stack.go:1088 +0x5ac
Nov 30 22:49:11 protobuf go-dnscollector[2238]: runtime.morestack()
Nov 30 22:49:11 protobuf go-dnscollector[2238]: #11/opt/hostedtoolcache/go/1.17.3/x64/src/runtime/asm_amd64.s:461 +0x8b
Nov 30 22:49:11 protobuf go-dnscollector[2238]: goroutine 21 [running]:
Nov 30 22:49:11 protobuf go-dnscollector[2238]: runtime.heapBitsSetType(0xc009804f50, 0x10, 0x10, 0x901880)

config file:

trace:
verbose: false

collectors:
dnstap:
enable: true
listen-ip: 0.0.0.0
listen-port: 6000
tls-support: false

loggers:
logfile:
enable: true
file-path: "/var/log/dnstap.log"
max-size: 100
max-files: 20
mode: json

Rename hyphen in qtypes to another name

Right now, for all unknown dns types dnscollector uses hyphen, I would like to suggest rename hyphen to "unknown" or "unrecognized" name.

Add collector for PowerDNS protocol buffers

Unfortunately DNSTAP support is quite limited in PowerDNS. Would it be possible to add a collector for PowerDNS protocol buffers ?

Thanks in advance,
Carlo

docker build : error obtaining VCS status: exec: "git": executable file not found in $PATH

https://github.com/dmachard/go-dns-collector/runs/5984956957?check_suite_focus=true

go1.19 support

Got an error at runtime on latest main branch.

panic: Something in this program imports go4.org/unsafe/assume-no-moving-gc to declare that it assumes a non-moving garbage collector, but your version of go4.org/unsafe/assume-no-moving-gc hasn't been updated to assert that it's safe against the go1.19 runtime. If you want to risk it, run with environment variable ASSUME_NO_MOVING_GC_UNSAFE_RISK_IT_WITH=go1.19 set. Notably, if go1.19 adds a moving garbage collector, this program is unsafe to use.

dnscollector and version output

I tried to get version information but got 0.0.0
Also, it will be useful to have version information as prometheus metric and in output during start new process, right now, I didn't find any place with version information

root@dnstap:/compose-go_dnscollector# docker exec --user root -it dnscollector_ext /bin/sh
/ # ps uaxfwwwww
PID   USER     TIME  COMMAND
    1 dnscolle  0:01 /bin/go-dnscollector -config /etc/dnscollector/config.yml
   19 root      0:00 /bin/sh
   25 root      0:00 ps uaxfwwwww
/ # /bin/go-dnscollector -v
flag provided but not defined: -v
Usage of /bin/go-dnscollector:
  -config string
        path to config file (default "./config.yml")
  -version
        Show version
/ # /bin/go-dnscollector -version
0.0.0

TLS MinVersion too low

Minimum version will be be TLS1.2 for next release

&tls.Config{
        MinVersion: tls.VersionTLS12,
}

Loggers and collectors impacted:

loggers/webserver
loggers/prometheus
collectors/dnstap

new prometheus metrics for number connected clients

It will be cool to have metric with number of connected clients
like as:
dnscollector_clients_total

INFO: 2021/12/24 05:38:54.218184 collector dnstap - 10.5.2.5:51318 - new connection
INFO: 2021/12/24 05:38:54.218221 dnstap processor - initialization...
INFO: 2021/12/24 05:38:54.218834 Subprocessor GeoIP - country database (%!s(uint=1630373273)) loaded (1203001 records)
INFO: 2021/12/24 05:38:54.218961 Subprocessor GeoIP - city database (%!s(uint=1639486540)) loaded (5265672 records)
INFO: 2021/12/24 05:38:54.219099 Subprocessor GeoIP - asn database (%!s(uint=1639486240)) loaded (881682 records)

Filter based on DNS rcode

Hi @dmachard,

Could you please extend the filtering feature to exclude/include replies based on the DNS return code? For example, I would like to filter out all the NOERRORs.

Thank you very much in advance,
Carlo

Filter to drop logs for specific query ip

to implement for next release

Soft reload configuration

I am not sure, have been it implemented or haven't, but it will be useful to have some soft reload configuration if configuration file was changed.

If configuration file was changed, just do automatic restart dns-collector process to apply new changes, this feature can be enabled over some configuration option.