When running queries, /tmp appears to be used to store .rwf file output from rwfilter which is later processed but another tool like rwcut, rwstats, etc. When querying a large about of data, these working files can be very large. Our /tmp directory is on a rather small file system and can become full when running large queries. Our data directory for SiLK however is relatively large. We'd like to either have a configurable option to set the directory where the temporary .rwf files are stored of have them default to the SiLK data directory.

Build in support for a pull-down to multiple SiLK deployments. For instance, if you have 10 SiLK deployments, each deployment has multiple sensors, and it isn't feasible to forward the flows to a singular location, this would allow the user to specify a "deployment" where data will be pulled from. This is as simple as changing the remoting configuration to another "saved" configuration.

This is not a perfect instruction.

Before run: silkonabox.sh & install_flowbat_ubuntu.sh:

1. https://www.itechlounge.net/2017/10/linux-how-to-add-rc-local-in-debian-9/

2. wget https://install.meteor.com > meteor.sh and edit line 33: 1.9 -> 1.6

3. Edit /install_flowbat_ubuntu.sh:

Line 26 replace curl https://install.meteor.com/ | sh with
cat meteor.sh | sh
Line 185: replace *xenial* with *buster*

4. https://docs.mongodb.com/manual/tutorial/install-mongodb-on-debian/

5. Run : silkonabox.sh

6. Run : install_flowbat_ubuntu.sh

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04 LTS
Release: 16.04
Codename: xenial

./install_flowbat_ubuntu.sh: line 141: npm: command not found
To manually start, stop, or check status of FlowBAT:
sudo service flowbat [start/stop/status]

Attempting startup. Check http://127.0.0.1:1800.
Failed to start flowbat.service: Unit flowbat.service not found.

Logfiles attached
logfile.txt
silkinstall.log.txt
silkonabox.sh.txt
install_flowbat_ubuntu.sh.txt

Can an option to display an interpreted IP protocol instead of the number?

Maybe the keyword or protocol columns from the link.

When I finish the installation with the mothod where offered on http://www.flowbat.com/installation.html,and remained me flowbat start.
but when I use sudo service flowbat status, I see the service flowbat stop/waitting,what should I do to solve the problem?

Doing a clean install. Upon pressing "Create" nothing happens and the browser returns

Exception in delivering result of invoking 'addNewUser': e.makeErrorType/r@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:3:3334
._livedata_result@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:70:22670
c/o@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:70:9772
._launchConnection/t.socket.onmessage/<@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:70:395
A.forEach@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:1:863
._launchConnection/t.socket.onmessage@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:70:351
h</r.prototype.dispatchEvent@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:68:1010
h</T.prototype._dispatchMessage@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:68:14587
h</T.prototype._didMessage@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:68:15474
h</T.websocket/o.ws.onmessage@http://10.10.10.74:1800/b55cd89ecbf81d1372b4e1ed02e0e2ca813ec38f.js:68:17569

when at this process:fetchmetadata sill install loadAlldepsintodeaiTree,it shows that:
Processing triggers for libc-bin (2.19-0ubuntu6.9) ...
Processing triggers for ureadahead (0.100.0-16) ...
npm ERR! code EPROTO
npm ERR! errno EPROTO
npm ERR! request to https://registry.npmjs.org/node-gyp failed, reason: write EPROTO 140565006227328:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:
npm ERR!

npm ERR! A complete log of this run can be found in:
npm ERR! /home/xxxxx/.npm/_logs/2018-03-23T07_45_37_007Z-debug.log
flowbat start/running, process 54704

Currently people want to be able to run FlowBAT against custom generated SiLK files. This would be possible with a switch that disabled the automatic "--type=all" addition to the filter query. I suggest adding a --type=none that when selected will trigger the type field to disappear. This, or if there are multiple input fields, automatically take out the --type input selector.

I see in install_flowbat_ubuntu.sh that SecurityOnion installation is detected when installing on Trusty. Doug has since moved on to Xenial as the base OS for SecurityOnion. I assume this should be handled differently than a vanilla install when installing FlowBAT on Xenial...

I can do sort in rwcount.such as : rwcount --delimited --no-titles --bin-size=300 --skip-zeroes --site-config-file=/data/silk.conf /tmp/S8nPHvQchqMMbomD9.rwf | sort --field-separator=| --key=+2nr | head --lines=100 | tail --lines=100.it do sort on records.but the chart isn't change.
111.txt

Do you wish to have FlowBAT start on boot in the background? [y/n] y
cp: unable to create normal file "/etc/init/": not a directory
To manually run FlowBAT, cd to /root/FlowBAT and run:
meteor --port 1800 run --settings settings/dev.json "$@"
or to run FlowBAT in the background:
nohup meteor --port 1800 run --settings settings/dev.json "$@" &

It seems that the installation shell script do not work well on Centos 7.4.

This sounds like #23, but I have new information, so I'm creating a separate issue.

I followed the FlowBAT installation instructions on my Ubuntu 14.04.3 box. Both silkinabox.sh and the FlowBAT installation script worked as expected and without surprises. (Congratulations!)

SiLK seems to be working: rwfilter --proto=0-255 --type=all --pass=stdout | rwcut shows information about traffic.

After the successful FlowBAT install, I tried to go to http://localhost:1800, but couldn't connect. When I issue sudo service flowbat start, /var/log/syslog shows:

Apr 14 01:28:14 ubuntu kernel: [ 1791.166427] init: flowbat main process (12715) terminated with status 1
Apr 14 01:28:14 ubuntu kernel: [ 1791.166436] init: flowbat main process ended, respawning
Apr 14 01:28:14 ubuntu kernel: [ 1791.218369] init: flowbat main process (12721) terminated with status 1
Apr 14 01:28:14 ubuntu kernel: [ 1791.218379] init: flowbat main process ended, respawning
Apr 14 01:28:15 ubuntu kernel: [ 1791.274604] init: flowbat main process (12727) terminated with status 1
Apr 14 01:28:15 ubuntu kernel: [ 1791.274614] init: flowbat main process ended, respawning
Apr 14 01:28:15 ubuntu kernel: [ 1791.325147] init: flowbat main process (12733) terminated with status 1
Apr 14 01:28:15 ubuntu kernel: [ 1791.325160] init: flowbat main process ended, respawning
Apr 14 01:28:15 ubuntu kernel: [ 1791.376849] init: flowbat main process (12739) terminated with status 1
Apr 14 01:28:15 ubuntu kernel: [ 1791.376857] init: flowbat main process ended, respawning
Apr 14 01:28:15 ubuntu kernel: [ 1791.429980] init: flowbat main process (12748) terminated with status 1
Apr 14 01:28:15 ubuntu kernel: [ 1791.429988] init: flowbat main process ended, respawning
Apr 14 01:28:15 ubuntu kernel: [ 1791.481163] init: flowbat main process (12754) terminated with status 1
Apr 14 01:28:15 ubuntu kernel: [ 1791.481172] init: flowbat main process ended, respawning
Apr 14 01:28:15 ubuntu kernel: [ 1791.531945] init: flowbat main process (12760) terminated with status 1
Apr 14 01:28:15 ubuntu kernel: [ 1791.531955] init: flowbat main process ended, respawning
Apr 14 01:28:15 ubuntu kernel: [ 1791.584680] init: flowbat main process (12766) terminated with status 1
Apr 14 01:28:15 ubuntu kernel: [ 1791.584692] init: flowbat main process ended, respawning
Apr 14 01:28:15 ubuntu kernel: [ 1791.635308] init: flowbat main process (12772) terminated with status 1
Apr 14 01:28:15 ubuntu kernel: [ 1791.635317] init: flowbat main process ended, respawning
Apr 14 01:28:15 ubuntu kernel: [ 1791.687643] init: flowbat main process (12778) terminated with status 1
Apr 14 01:28:15 ubuntu kernel: [ 1791.687652] init: flowbat respawning too fast, stopped

What other information could I provide? Thanks.

Hoping to test it, but it would be great to have proper, manual installation instructions. In my case I want to try it on FreeBSD.

Besides, manual instructions help debug sudden installation problems due to unexpected changes in the OS distributions.

I miss the opportunity to add exclusions an easier way. E.g based on the flow output. So when I click an IP-address in the output it should also include "Add to exclusions as Source/Destination IP". (together with "Add to query as Source IP"++)

Is this possible add alarms and thresholds to email you when they are reached like Scrutinizer?

I had earlier reported a problem with installing FlowBAT (#25), but realize that my report was flawed. I spent some time to create a reproducible test case using a Docker instance that shows where things go wrong. I put the Dockerfile as well as the build procedure in https://github.com/richb-hanover/SiLK-in-Docker

To see the installation process, read the Dockerfile in that repo. After using the "To start debugging" steps, I see this error:

node /home/flowbat/FlowBAT/private/bundle/main.js

/home/flowbat/FlowBAT/private/bundle/programs/server/node_modules/fibers/future.js:245
            throw(ex);
                  ^
TypeError: Cannot read property 'isDebug' of undefined
    at lib/app.coffee:162:39
    at lib/app.coffee:1:1
    at /home/flowbat/FlowBAT/private/bundle/programs/server/boot.js:222:10
    at Array.forEach (native)
    at Function._.each._.forEach (/home/flowbat/FlowBAT/private/bundle/programs/server/node_modules/underscore/underscore.js:79:11)
    at /home/flowbat/FlowBAT/private/bundle/programs/server/boot.js:117:5

My github repo shows how to reproduce this failure. Any thoughts? Many thanks.

Is it possible to disable UTC time zone and get everything on eastern time
Searching is very difficult when i need to convert time

Good Afternoon,

We are requesting that the Country Codes be translated to Country Names to make it easier to identify source and destination countries. Also, ASN number population would be cool.

If a user requests a CSV from the output of rwcount, he/she gets an endless spinning wheel and no data.

the install script provided is for ubuntu. Is there another install script you have to install this on other distros. Everything we use in my org is CentOS.

Thanks

Drop down is below the next section when a single row is in view.
One must scroll to see it.

I get the following error while installing on Ubuntu 16.04 with your installing script:

[...]
Done.
Adding user `mongodb' to group `mongodb' ...
Adding user mongodb to group mongodb
Done.
Processing triggers for libc-bin (2.23-0ubuntu10) ...
Processing triggers for systemd (229-4ubuntu21.1) ...
Processing triggers for ureadahead (0.100.0-19) ...

> [email protected] install /home/flowbat/FlowBAT/private/bundle/programs/server/node_modules/fibers
> node build.js || nodejs build.js

`linux-x64-57` exists; testing
Binary is fine; exiting

> [email protected] install /home/flowbat/FlowBAT/private/bundle/programs/server
> node npm-rebuild.js

events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: spawn npm ENOENT
    at _errnoException (util.js:1024:11)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:190:19)
    at onErrorNT (internal/child_process.js:372:16)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
    at Function.Module.runMain (module.js:678:11)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:608:3
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node npm-rebuild.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/flowbat/.npm/_logs/2018-02-27T16_56_38_332Z-debug.log
Created symlink from /etc/systemd/system/default.target.wants/flowbat.service to /lib/systemd/system/flowbat.service.

It'd be handy if IP Set files can have user-defined names (perhaps the same names as the IP Set itself?)

The reason is that sometimes you might want to use multiple IP Sets in an Exclusion and it would be much simpler (and more meaningful) to say "--dipset=/tmp/setOne.rws OR --dipset=/tmp/setTwo.rws" than to work out what the random names are.

I just set up a new flowbat instance on our network but it seems like the ipfix flows aren't being decoded properly. I can see the flows within the packets coming in correctly using tcpdump and manually opening them. Flowbat only shows the packets themselves not the flows within.

Currently only a-z, A-Z, 0-9 work in file names. This needs extended character support.

Title says it all. We're a couple of versions behind in the installer.

Dear
I setting up flowbat for my system with remote sensor. So when i type querry and see a Warning "Oh snap!"
Permission denied, please try again.

Permission denied, please try again.

Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password)

I can't see chart, pie v...v
So, what can i do fix this Error? My account can ssh to sensor and have permission sudoer
I sent a image shooted

Running the installation script 'install_flowbat_ubuntu.sh' on a freshly
installed Ubuntu Xenial server results in the following error message:

...
Setting up mongodb-server (1:2.6.10-0ubuntu1) ...
Adding system user `mongodb' (UID 107) ...
Adding new user `mongodb' (UID 107) with group `nogroup' ...
Not creating home directory `/var/lib/mongodb'.
Adding group `mongodb' (GID 111) ...
Done.
Adding user `mongodb' to group `mongodb' ...
Adding user mongodb to group mongodb
Done.
Processing triggers for libc-bin (2.23-0ubuntu3) ...
Processing triggers for systemd (229-4ubuntu4) ...
Processing triggers for ureadahead (0.100.0-19) ...

> [email protected] install /home/flowbat/FlowBAT/private/bundle/programs/server
> node npm-rebuild.js

events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: spawn npm ENOENT
    at _errnoException (util.js:1024:11)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:190:19)
    at onErrorNT (internal/child_process.js:372:16)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
    at Function.Module.runMain (module.js:678:11)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:608:3
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node npm-rebuild.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/flowbat/.npm/_logs/2018-02-08T12_51_35_981Z-debug.log

The log file is attched
2018-02-08T12_51_35_981Z-debug.log

Hello
I do try to install and make it work on Centos7 - It 's seem not working at all.
The closest to work that I had was METEOR UI.
Flowbat never come up.

When auto-execute is enabled, a previously large query can fill the /tmp directory or begin a long running query that is unwanted. This is because FlowBAT retains the previous form data (which is nice).

An option to disable auto-execute would be beneficial.

sudo installs might result in a non-functioning install from the install scripts. Those installs require significant manual efforts to finish installing. This addition will add a warning in the beginning, or require an "option" to proceed with a sudo install.

I'm a newer
I'm learning flowbat.I only know that the rwcut output is a document that File extension is .csv.
But I can't find it.I want to know the path of output data.I can't find it from table.js and chart.coffee.
I need your help.Can tell me where's the output data