bio-routing / tflow2 Goto Github PK

This project forked from google/tflow2

License: Apache License 2.0

Go 93.61% CSS 0.70% HTML 3.10% JavaScript 2.49% Shell 0.09%

tflow2's Introduction

tflow2

tflow2 is an in memory netflow version 9, IPFIX and Sflow analyzer. It is designed for fast arbitrary queries and exports data to Prometheus.

Usage

Quick install with go get -u github.com/bio-routing/tflow2 and go build github.com/bio-routing/tflow2 or download a pre-built binary from the releases page.

The release binaries have an additional command, tflow2 -version, which reports the release version.

Once you start the main binary it will start reading netflow version 9 packets on port 2055 UDP and IPFIX packets on port 4739 on all interfaces. For user interaction it starts a webserver on port 4444 TCP on all interfaces.

The webinterface allows you to run queries against the collected data. Start time and router are mandatory criteria. If you don't provide any of these you will always receive an empty result.

Config file

There is YAML file as config. Defaults can be found in config-example.yml. You'll at least need to add your Netflow/IPFIX/Sflow agents and adjust (if you don't want to work with interface IDs) your SNMP RO community.

Command line arguments

-alsologtostderr

Will send logs to stderr on top.

-channelBuffer=int

This is the amount of elements that any channel within the program can buffer.

-dbaddworkers=int

This is the amount of workers that are used to add flows into the in memory database.

-log_backtrace_at

when logging hits line file:N, emit a stack trace (default :0).

-log_dir

If non-empty, write log files in this directory.

-logtostderr

log to standard error instead of files.

-samplerate=int

Samplerate of your routers. This is used to deviate real packet and volume rates in case you use sampling.

-sockreaders=int

Num of go routines reading and parsing netflow packets (default 24).

-stderrthreshold

logs at or above this threshold go to stderr.

-v value

log level for V logs.

-vmodule value

comma-separated list of pattern=N settings for file-filtered logging.

Limitations

Please be aware this software is not platform independent. It will only work on little endian machines (such as x86)

License

This is not an official Google product.

tflow2's People

Contributors

Stargazers

Watchers

Forkers

kant indigos33k3r fatman00 superq seandunlap itzoey

tflow2's Issues

Filter not working

I have a problem with filtering information.
If I make a query for all source ports I get the following graph:

And as shown there are a lot of traffic with source port 443. If change my filter to only include source port 443 it shows there is no traffic. I would expect there to be at least the amount shown in the previous graph.

Any suggestions?

Feature request - Netflow5

Hi
It would be great with support for NetFlow Version 5.

[annotation.go:88] Unable to annotate

Hi,

Can you share an example bird.conf? From what I read through bird.go I understand that the protocol should be named: nf_(ip of peer).replace(".", "_"):

template bgp Neighbors {
        local as 1234;          # our own as

        import all;
        export none;
	multihop;
}

protocol bgp nf_12_12_12_12 from Neighbors {
        description "bb01.fra";

        neighbor 12.12.12.12 as 1234;
}

But I keep getting unable to annotate.

How to use Prometheus feature

One can guess that the tflow2 server exposes basic metrics on /metrics. But there is no documented way of retrieving actual sample data as Prometheus metrics. Please, write a documentation for that!

I already tried an example from pull request #15 - but that didn't give any results other than the base metrics.

Feature request: Filter by / Breakdown by port

It would be nice to have an option to filter by or breakdown by port similar to dst port or src port, but not minding the direction. While for breakdown that would count each flow twice I think it would still be nice if you wanted to see for example http vs https.

Use ExporterAddr where applicable

Hi,
please add the functionality to use the netflow field exporteraddr where set and differend from the source address.
The current implementation either tries to snmpwalk the wrong device or cannot associate incoming traffic.

Nico

Feature Request: show graphs in pps instead of bps

It would be nice if there was an option to display graphs in pps instead of bps.

Bundle static files

HTML, CSS and Javascript files should be bundled in the binary for easier deployment:
https://github.com/shurcooL/vfsgen#comparison

Build broken

please fix it @taktv6
broken by b0e351c

example config file missing

In the latest commit bab0f63, the examples config file is missing.

Filter DstAsn and SrcAsn not working

Hi,

the query is searching for fields named DstAsn and SrcAsn, although they are named DstAs and SrcAs.
Changing the parameters in the URL fixes the problem.

Nico

Mapper error when trying to run tflow2

Here is the error that I receive:

F0331 16:59:36.969631 3319 tflow2.go:59] Unable to initialize interface mappper: Unable to get interface mapping for localhost: Walk error: Error reading from UDP: read udp 127.0.0.1:50142->127.0.0.1:161: read: connection refused

Here is my config.yml. What am I missing?

aggregation_period: 60
default_snmp_community: "public"
debug: 0
compression_level: 6
data_dir: "data"
anonymize: false
cache_time: 1800

netflow_v9:
enabled: true
listen: ":2055"

ipfix:
enabled: false
listen: ":4739"

sflow:
enable: false
listen: ":6343"

frontend:
enable: false
listen: ":4444"

bgp_augmentation:
enabled: false
bird_socket: "/var/run/bird/bird.ctl"
bird6_socket: "/var/run/bird/bird6.ctl"

annotators:

name: "BGP Annotator"
target: "localhost:9090"

agents:

name: "localhost"
ip_address: "127.0.0.1"
snmp_community: "public"
samplerate: 1000

Thank you!