iqtlabs / gamutrf Goto Github PK

An SDR orchestrated scanner and collector.

License: Apache License 2.0

Dockerfile 1.47% Python 86.84% Shell 9.03% HTML 2.22% CSS 0.44%

sdr scanner collector iq-samples iq-signal docker edge raspberry-pi rpi ettus

gamutrf's Introduction

gamutRF

gamutRF is a gnuradio-based SDR-based scanner, I/Q signal collector and identifier (using a image or I/Q based pyTorch model).

While it can run on Pi4 and Pi5 machines (and its components can be distributed over a network), it is more typically deployed on a single x86_64 machine with an nvidia GPU (see deployment instructions).

gamutRF's scanner container connects to a local SDR and sweeps over a configured frequency range or ranges collecting samples. When a configurable number of valid I/Q samples are received the SDR is retuned to a new interval (see blocks in gr-iqtlabs). The samples are processed and sent to a waterfall container for display, and optionally to a Torchserve instance for identification. Recording, and basic parameters (such as the frequency range to scan) can be controlled from the waterfall container.

License

Distributed under the Apache 2.0. See LICENSE for more information.

Contact IQTLabs

Twitter: @iqtlabs
Email: [email protected]

See our other projects: https://github.com/IQTLabs/

AIR-T support

GamutRF has legacy support for the Deepwave AIR-T

Development

Development with GamutRF requires familiarity with gnuradio, an SDR, a x86_64 host running Ubuntu 24.04 with Docker installed (and ideally an nvidia GPU, though this is not required).

Local development

Install gnuradio, 3.10 or later
Install gr-iqtlabs
Make modifications, and install with poetry install
Run tests with pytest

Docker development

Follow above local development instructions, and then build containers (tests will be run inside the containers).

docker build -f docker/Dockerfile.base docker -t iqtlabs/gamutrf-base:latest
docker build -f Dockerfile . -t iqtlabs/gamutrf:latest

gamutrf's People

Contributors

Stargazers

Watchers

Forkers

anarkiwi mfkiwl robotastic cglewis emmair ltindall rashley-iqt lk-iqt meadej kaiwhata bigdatamatta jimzgchow monag-ucberkeley

gamutrf's Issues

BladeRF can't find it's lib for A4

[WARNING @ host/libraries/libbladeRF/src/board/bladerf2/bladerf2.c:456] FPGA bitstream file not found.
[WARNING @ host/libraries/libbladeRF/src/board/bladerf2/bladerf2.c:457] Skipping further initialization...

  Error: Command requires FPGA to be loaded

Can't specify multiple recorders

Streamline docker builds

Currently docker builds are taking on the order of 6 hours to complete and frequently timeout. Please:

profile the builds to determine where the bottleneck is occurring
Further modularize build workflow

potentially seperate the build of amd64 and arm64 images

refactor builds or images to improve speed

Can Pluto SDR be used？

Is it possible to implement only Pluto SDR and Ubuntu systems？

ZMQ Client Library

Currently ZMQ functionality is tightly coupled to the sigfinder module. This functionality should be abstracted out into a separate library.

Support non-linear scans

Allow users to specify a list of scan ranges to be monitored.

Remove sigfinder's png output in favor of waterfall

Possibly no longer required (though peak finding still necessary to command workers).

Tar up scanner output directories on demand

Easier file management with dealing with a lot of scanner samples/output.

When using Ettus, the container has to be restarted on the first try

The USB connection isn't properly detected on the first attempt at boot

Improve Test Coverage

remove obsolete:

scan2mp4

Increase coverage for:

grsource currently at 59% coverage
scan2rtlpow currently at 55% coverage
sigfinder currently at 57% coverage

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

WARN: Unexpected range error

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Update dependency numpy to v2.1.1

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Update python Docker tag to v3.12

Detected dependencies

dockerfile

Dockerfile

ubuntu 24.04

ubuntu 24.04

iqtlabs/gnuradio 3.10.11.0

docker/Dockerfile.airspyfm

ubuntu 24.04

ubuntu 24.04

docker/Dockerfile.base

ubuntu 24.04

iqtlabs/gnuradio 3.10.11.0

ubuntu 24.04

iqtlabs/gnuradio 3.10.11.0

docker/Dockerfile.driver

ubuntu 24.04

iqtlabs/gnuradio 3.10.11.0

ubuntu 24.04

docker/Dockerfile.sigmf

ubuntu 24.04

ubuntu 24.04

docker/Dockerfile.test

docker/Dockerfile.vkfft

ubuntu 24.04

ubuntu 24.04

docker/Dockerfile.waterfall

ubuntu 24.04

ubuntu 24.04

utils/mavlink-api/Dockerfile.mavlink-api

python 3.11-slim

utils/mavlink-api/Dockerfile.mqtt-publisher

python 3.11-slim

github-actions

.github/workflows/ci-test.yml

actions/checkout v4

actions/setup-python v5

snok/install-poetry v1

ubuntu 22.04

.github/workflows/codeql-analysis.yml

actions/checkout v4

github/codeql-action v3

github/codeql-action v3

github/codeql-action v3

.github/workflows/docker-extras.yml

actions/checkout v4

docker/setup-qemu-action v3

docker/setup-buildx-action v3

docker/build-push-action v6

actions/checkout v4

docker/setup-qemu-action v3

docker/setup-buildx-action v3

docker/build-push-action v6

.github/workflows/docker-test.yml

actions/checkout v4

actions/checkout v4

actions/checkout v4

actions/checkout v4

.github/workflows/docker.yml

actions/checkout v4

docker/setup-qemu-action v3

docker/setup-buildx-action v3

docker/build-push-action v6

docker/build-push-action v6

docker/build-push-action v6

docker/build-push-action v6

docker/build-push-action v6

actions/checkout v4

docker/setup-qemu-action v3

docker/setup-buildx-action v3

docker/build-push-action v6

.github/workflows/pypi.yml

actions/checkout v4

snok/install-poetry v1

.github/workflows/secrets-scan.yml

actions/checkout v4

actions/github-script v7

actions/upload-artifact v4.4.0

.github/workflows/semgrep.yml

actions/checkout v4

actions/setup-python v5

actions/github-script v7

actions/upload-artifact v4.4.0

actions/checkout v4

actions/setup-python v5

actions/github-script v7

actions/upload-artifact v4.4.0

.github/workflows/stale.yml

actions/stale v9

pep621

gamutrflib/pyproject.toml

poetry-core >=1.0.0

gamutrfwaterfall/pyproject.toml

poetry-core >=1.0.0

pyproject.toml

poetry-core >=1.0.0

utils/mavlink-api/pyproject.toml

poetry

gamutrflib/pyproject.toml

numpy 2.0.1

scipy 1.14.1

pandas 2.2.2

python >=3.10,<3.13

pyzmq 26.2.0

zstandard 0.23.0

gamutrfwaterfall/pyproject.toml

Flask 3.0.3

cairocffi 1.7.1

matplotlib 3.9.2

numpy 2.0.1

pandas 2.2.2

pycairo 1.27.0

python >=3.10,<3.13

pyzmq 26.2.0

scipy 1.14.1

zstandard 0.23.0

requests 2.32.3

pyproject.toml

Flask 3.0.3

Jinja2 3.1.4

bjoern 3.2.2

falcon 3.1.3

falcon-cors 1.1.7

gpsd-py3 0.3.0

numpy 2.0.1

paho-mqtt 2.1.0

pandas 2.2.2

pbr 6.1.0

prometheus_client 0.20.0

python >=3.9,<3.13

pyzmq 26.2.0

requests 2.32.3

sigmf 1.2.2

sysrsync 1.1.1

webcolors 24.8.0

zstandard 0.23.0

scipy 1.13

attr 0.3.2

attrs 24.2.0

black 24.8.0

docker 7.1.0

pdbpp ^0.10.3

pylint 3.2.7

pytest 8.3.2

pytest-cov 5.0.0

pytype 2024.4.11

utils/mavlink-api/pyproject.toml

python >=3.10,<3.13

Flask ^3.0.0

pymavlink ^2.4.40

pyserial ^3.5

httpx 0.27.2

paho-mqtt ^2.0.0

Check this box to trigger a request for Renovate to run again on this repository

sigfinder won't open new connections to a collector if it tries too many times and fails

2021-11-19 22:26:20,350 HTTPConnectionPool(host='192.168.111.12', port=8000): Max retries exceeded with url: /v1/info (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffff7c0cee90>: Failed to establish a new connection: [Errno 111] Connection refused'))

It should try to establish a new connection each time, not hold on the initial failure and not try again.

The gamutrF fails to connect to the BladerF

Hello, how does GamuTRf connect to Bladerf? Bladerf is running correctly on the VM. Run the "docker run -v/TMP :/ TMP -ti iqtlabs/gamutrf gamutrf-scan -- SDR bladerf "command to return "[ERROR] bladerf_open_with_devinfo() Returned -7 - No device(s) available" Please see the screenshot for details

Standalone install for NUC

Setup a method for creating an instance of GamutRF that doesn't need to be split across multiple Raspberry Pis

SDR Test Harness

Create a test harness compatible with supported SDRs to test signal detection and performance tune scanning and accuracy.

Parallelize Sigfinder scans

allow sigfinder to connect to multiple scanners

Add GUI for configuration

Please add a graphical user interface allowing the modification of the gamutRF scanner configuration during operation.

This crashes sigfinder if the response doesn't have that key for some reason, this should be properly handled

gamutRF/gamutrf/sigfinder.py

Line 106 in 13055ec

excluded = json.loads(req.text)['freq_excluded']

Waterfall bug (bad db min/max + bad data)

I've observed unexpected behavior where the waterfall dB limits do not seem to update properly. In image 1 the minimum dB on the y-axis seems to be about -95 dB despite the true minimum from visual inspection being closer to -84 dB. This behavior appears after restarting the scanner with a different --peak_fft_range value (in this case 0 -> 1) which results in the noise floor being raised and thus the expectation of the minimum dB increasing.

Additionally, I've noticed what appears to be incorrect FFT values in the first column as shown in images 2 and 3. I've included these in the issue since I speculate they could be the root cause of the issue above.

db_min/db_max are dependent on db_data

gamutRF/gamutrfwaterfall/gamutrfwaterfall/waterfall.py

Lines 654 to 655 in 1d58be6

 state.db_min = np.nanmin(state.db_data) 

 state.db_max = np.nanmax(state.db_data)

db_data should roll in new data as the scanner restarts and eventually all the db_data from the first run of the scanner container will be rolled out. This behavior remains even when all db_data has been replaced.

gamutRF/gamutrfwaterfall/gamutrfwaterfall/waterfall.py

Lines 621 to 622 in 1d58be6

 state.db_data = np.roll(state.db_data, -1, axis=0) 

 state.db_data[-1][idx] = scan_df.db.values.flatten()

Image 3 appears to show a single non-nan value for the entire first column. This by itself may be a separate issue but could also be the root cause of this issue. My initial guess is that there could be a mistake in the following lines which calculate the indexing used to populate db_data.

gamutRF/gamutrfwaterfall/gamutrfwaterfall/waterfall.py

Lines 610 to 616 in 1d58be6

 idx = ( 

 ((scan_df.freq - config.min_freq) / config.freq_resolution) 

 .round() 

 .clip(lower=0, upper=(state.db_data.shape[1] - 1)) 

 .values.flatten() 

 .astype(int) 

 )

Image 1

Image 2

Image 3

To replicate:

Run waterfall and scanner containers. Set --peak_fft_range=0 (value is arbitrary, this value is used for clarity of the resulting difference)
Set --peak_fft_range=1 and restart scanner container.

	state.db_min = np.nanmin(state.db_data)
	state.db_max = np.nanmax(state.db_data)

	state.db_data = np.roll(state.db_data, -1, axis=0)
	state.db_data[-1][idx] = scan_df.db.values.flatten()

	idx = (
	((scan_df.freq - config.min_freq) / config.freq_resolution)
	.round()
	.clip(lower=0, upper=(state.db_data.shape[1] - 1))
	.values.flatten()
	.astype(int)
	)

iqtlabs / gamutrf Goto Github PK

gamutrf's Introduction

gamutRF

License

Contact IQTLabs

AIR-T support

Development

Local development

Docker development

gamutrf's People

Contributors

Stargazers

Watchers

Forkers

gamutrf's Issues

Repository problems

Open

Ignored or Blocked

Detected dependencies

Recommend Projects

Recommend Topics

Recommend Org