Quick Start | Contribute

⚠️ PewView is currently being actively developed. Until it reaches v1.0.0 breaking changes may occur in minor versions.

PewView is a self-hosted network visualization tool. It listens for incoming network traffic flow in the commonly used IPFix (NetFlow v9), NetFlow v5 and sFlow formats, as well as a simple HTTP-based API endpoint. The traffic data is condensed, anonymized and enriched by adding location data to the observed network connections. The network traffic is then exported in a simple API. A web-based frontend then uses the API to visualize the network traffic in realtime on a 3D globe, shown in the picture above.

The service comes in two parts, an optional frontend and a high-throughput server based on Cloudflare's goflow.

Whilst the two are intended to be used together, one may chose to deactivate the frontend in order to use PewView as a high-throughput consumer of IPFIX, NetFlow, sFlow and more.

First, download the latest release for your architecture.

You'll also need a GeoIP service to enable PewView to resolve IP addresses to locations. For instructions on setting some of them up, see IP Geolocation configuration. A free service without config values is used in the example below.

The service can now be started like so:

pewview --consumer=netflow --location-provider=ipapi

PewView is incredibly configurable, please refer to the documentation below for instructions on how to use other conumers and location providers.

Quickstart
Features
Installation
Usage
Contributing

• Intuitive web interface with a 3D visualization
• High performance and scalable consumer
• Supports NetFlow v5, Netflow v9 / IPFIX, sFlow and more
• Stateless and usable via a single Docker container
• Supports many location providers, offline, online, free and paid

brew install alexgustafsson/tap/pewview

Download the latest release from here.

Clone the repository.

git clone https://github.com/AlexGustafsson/pewview.git && cd pewview

Optionally check out a specific version.

git checkout v0.2.0

Build the application and frontend.

make build

Note: This project is still actively being developed. The documentation is an ongoing progress.

# Run PewView, listening for incoming IPFix (Netflow v9) data, getting location data from ipapi.io
pewview --consumer ipfix --location-provider ipapi

# Lookup some addresses using the configured location provider(s)
pewview --location-provider ipapi --lookup-address <ip>

Usage:
  pewview [OPTIONS]

Application Options:
      --lookup-address=                               Print the location of the address and exit. May be used more than once
      --consumer=[ipfix|netflow|sflow|webhook|random] Enable a consumer. May be used more than once
      --geo=[geolite|ipgeolocation|ipapi|file]        Enable a location provider. May be used more than once

Logging:
      --log.level=[debug|info|warn|error]             Log level (default: info)

IPFix Consumer:
      --consumer.ipfix.address=                       Listening address (default: <unset>)
      --consumer.ipfix.port=                          Listening port (default: 2055)
      --consumer.ipfix.workers=                       Worker count (default: 1)

Netflow Consumer:
      --consumer.netflow.address=                     Listening address (default: <unset>)
      --consumer.netflow.port=                        Listening port (default: 2056)
      --consumer.netflow.workers=                     Worker count (default: 1)

SFlow Consumer:
      --consumer.sflow.address=                       Listening address (default: <unset>)
      --consumer.sflow.port=                          Listening port (default: 6343)
      --consumer.sflow.workers=                       Worker count (default: 1)

WebHook Consumer:
      --consumer.webhook.address=                     Listening address (default: <unset>)
      --consumer.webhook.port=                        Listening port (default: 8081)

GeoLite Location Provider:
      --geo.geolite.path=                             Path to GeoLite2-City.mmdb

ipgeolocation.io Location Provider:
      --geo.ipgeolocation.key=                        API key [$PEWVIEW_IPGEOLOCATION_KEY]

File-based Location Provider:
      --geo.file.path=                                Path to JSON file containing patterns and locations

Web:
      --web.disable                                   Disable the built-in web interface
      --web.address=                                  Listening address (default: <unset>)
      --web.port=                                     Listening port (default: 8080)
      --web.origin=                                   Origin to allow (default: *)

Pipeline:
      --pipeline.queue=                               Length of the pipeline's message queue (default: 1024)

Metrics Tuning:
      --metrics.window=                               Duration of a window (default: 1m)

Metrics to Expose:
      --metrics.expose.bytes                          Expose number of bytes sent in a connection
      --metrics.expose.source-address                 Expose source address of a connection
      --metrics.expose.source-port                    Expose source port of a connection
      --metrics.expose.destination-address            Expose destination address of a connection
      --metrics.expose.destination-port               Expose destination port of a connection

Prometheus:
      --prometheus.enable                             Enable /metrics endpoint

Help Options:
  -h, --help                                          Show this help message

MaxMind's database is available offline, either free or paid. The paid version comes with more and newer data.

1. Create a free account on https://dev.maxmind.com/geoip/geoip2/geolite2/
2. Log in and go to the Download Files page under GeoIP2 / GeoLite 2
3. Right click on Download GZIP of the GeoLite2 City row and copy the link
4. Download the file using wget --output-document geoip.gzip "<copied path>"
5. Untar the file using mkdir -p data/GeoLite && tar --strip=1 --directory data/GeoLite -xzvf geoip.gzip

For evaluation, you can download test data from maxmind/MaxMind-DB and follow the same procedures as above.

Specify --location-provider=geolite --geolite.path=./data/GeoLite/GeoLite2-City.mmdb when starting PewView.

The ipgeolocation.io service has a free tier offering 1K requests a day, 30K a month. There are alternatives for up to 20M requests a month.

1. Create an account on https://ipgeolocation.io
2. Log in to get your API key

Specify --location-provider=ipgeolocation --ipgeolocation.key=<key> when starting PewView. The key can also be set in the PEWVIEW_IPGEOLOCATION_KEY environment variable.

The ip-api.com service is free, but it is served over HTTP and limited to 40 requests per minute. The paid alternative is currently not supported.

Specify --location-provider=ipapi when starting PewView.

Any help with the project is more than welcome. The project is still in its infancy and not recommended for production.

# Clone the repository
https://github.com/AlexGustafsson/pewview.git && cd pewview

# Show available commands
make help

# Build the project for the native target
make build

Note: due to a bug (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93082, https://bugs.llvm.org/show_bug.cgi?id=44406, https://openradar.appspot.com/radar?id=4952611266494464), clang is required when building for macOS. GCC cannot be used. Build the server like so: CC=clang make server.

This project was made possible by Cloudflare's goflow which is used for consuming network traffic.