radioml / dataset Goto Github PK
View Code? Open in Web Editor NEWOpen RadioML Synthetic Benchmark Dataset
Open RadioML Synthetic Benchmark Dataset
Can you please guide me with the steps how u are executing this file in gnu radio?
Thanks in advance
I am trying to build the datasets but "import mediatools" fails.
I tried to install manually gr-mediatools but It doesn't work.
Traceback (most recent call last):
File "/home/winlab/PycharmProjects/GeneralGNU/dataset-master/generate_RML2016.10a.py", line 42, in
mod = mod_type()
File "/home/winlab/PycharmProjects/GeneralGNU/dataset-master/transmitters.py", line 35, in init
transmitter_mapper.init(self, mapper.BPSK,
AttributeError: 'module' object has no attribute 'BPSK'
There is a error. I've tried several versions of mapper but it still failed. What is the proper and correct version of mapper?
What is the carrier frequency of the modulation modes in the RML201610A dataset?
nathan@saffron:~/code/dataset$ git pull origin master
From github.com:radioML/dataset
* branch master -> FETCH_HEAD
Already up-to-date.
nathan@saffron:~/code/dataset$ git push origin master
ERROR: Repository not found.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
nathan@saffron:~/code/dataset$ git remote -v
gh-https https://github.com/radioML/dataset.git (fetch)
gh-https https://github.com/radioML/dataset.git (push)
origin [email protected]:radioML/dataset.git (fetch)
origin [email protected]:radioML/dataset.git (push)
The classes.txt file in the dataset is in the following order:
classes = ['32PSK',
'16APSK',
'32QAM',
'FM',
'GMSK',
'32APSK',
'OQPSK',
'8ASK',
'BPSK',
'8PSK',
'AM-SSB-SC',
'4ASK',
'16PSK',
'64APSK',
'128QAM',
'128APSK',
'AM-DSB-SC',
'AM-SSB-WC',
'64QAM',
'QPSK',
'256QAM',
'AM-DSB-WC',
'OOK',
'16QAM']
I'm not sure if that is the same order as in the dataset.
Has someone figured this out?
When I run generate_RML2016.10a.py, this error come out:
$ ./generate_RML2016.10a.py
snr is -20
len(interp_taps) = 287
[mp3 @ 0x7fcfb8000b60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7fcfb8002040] Header missing
error decoding file
...
error decoding file
WARN: fractional_interpolator is deprecated. Please use fractional_resampler instead.
[mp3 @ 0x7fcff0000b60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7fcff0096c80] Header missing
error decoding file
all done. writing to disk
Is there something that my computer missed?
By the way, it seems that the RML2016.10a_dict.dat
has also been generated successfully. The .dat file size is 640.9MB.
I'm trying to run the code with the Anaconda - Python (windows). It says "module gnuradio is missing". There is no "gnuradio" package available - Windows version at:
https://anaconda.org/ryanvolz/gnuradio
Should I use it with Linux? Any other solution would be appreciated.
I use the generate_RML2016.10a.py to generate the data set, which is the same number as the data set published by the author. And I use the same network to train the two data set, but the test accuracy are different. The accuracy of the data set generated by the code is 4-5 percentage points lower. Why? Are the parameters in the code provided by the author different from those in the public dataset?
Sorry, when I tried to run the generate_RML2016.10a.py, this error came out. Could you please give me a hand?
Traceback (most recent call last):
File "e:\python\dataset_master\generate_RML2016.10a.py", line 3, in
from source_alphabet import source_alphabet
File "e:\python\dataset_master\source_alphabet.py", line 3, in
import mediatools
ImportError: No module named mediatools
Hi there!
I've realized that in the dataset generation, the energy of the 128-sample data vectors is being normalized to unity as follows (lines 65 and 66):
However, to the best of my knowledge, the energy Es of a discrete-time signal x(n) is defined mathematically as:
Once you have calculated the Es, the sampled_vector must be divided by the square root of the energy, not only by the energy itself. In code, it should be something like this:
energy = np.sum(np.abs(sampled_vector) ** 2)
sampled_vector = sampled_vector / math.sqrt(energy)
I've plotted both versions and these are the results.
Before:
After:
Therefore the signals are being unnecessarily compressed, which can make it harder for (some) models to extract meaningful information, or even prevent it altogether.
Do my findings make sense to you or is there anything that I may have not understood properly? Please, check it and update us with your conclusions if you are so kind.
I look forward to hearing from you.
Regards,
Ramiro Utrilla
I think the noise amplitude calculation should be:
10^(-snr_db/20.0), but the code has: 10^(-snr_db/10.0)
It looks like all the transmitters scale to an average magnitude squared P_avg=1. Then SNR_lin = P_avg/sigma^2, so sigma^2 = 1/snr_lin = 1/(10^(-snr_db/10)) => sigma = sqrt(1/(10^(-snr_db/10))) = 10^(-snr_db/20). The dynamic channel model takes in the standard deviation not the variance.
What am I missing?
When running the data generation script, I am missing files
source_material/gutenberg_shakespeare.txt: No such file or directory
Traceback (most recent call last):
File "/home/aaron/src/radioml/dataset/generate_RML2016.10a.py", line 39, in <module>
src = source_alphabet(alphabet_type, tx_len, True)
File "/home/aaron/src/radioml/dataset/source_alphabet.py", line 13, in __init__
self.src = blocks.file_source(gr.sizeof_char, "source_material/gutenberg_shakespeare.txt")
File "/home/aaron/src/gr/lib/python2.7/dist-packages/gnuradio/blocks/blocks_swig0.py", line 436, in make
return _blocks_swig0.file_source_make(*args, **kwargs)
RuntimeError: can't open file
I do not appear to have access to the dependent submodule.
$ git submodule update
Cloning into 'source_material'...
ERROR: Repository not found.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:radioML/source_material.git' into submodule path 'source_material' failed
Are these files necessary or there is a way to bypass them (use random number generator maybe?)
Could someone tell me how to generate my own dataset with different channel parameters ??
I couldn't find any documentation on this.
(dp1
(S'QPSK'
p2
I2
tp3
cnumpy.core.multiarray
reconstruct
p4
(cnumpy
ndarray
p5
(I0
tS'b'
tRp6
(I1
(I1000
I2
I128
tcnumpy
dtype
p7
(S'f4'
I0
I1
tRp8
(I3
S'<'
NNNI-1
I-1
I0
tbI00
S'!a\xc1\xbbK\xbc\x19\xbb7PC\xba0+\xaf\xbb(\xb5\xbd\xbb\x99Fr\xbbB$\xa3\xbbR%\xd7\xbb\x9cC\x14\xbc\x11}\x9e\xbb)6%\xbc\xd2\xb4\x94\xbb\xbfd\x8d\xbb\x11=\xa8\xbbp\x16\xc7\xbb\xaa\x89\x9b:=\x97\x98\xbb\xbd\xd8\xe3\xbb4\x81\xda\xbb\xba\x04\xd3\xbbD\xb2z\xbb\xbdn\t\xbc\xcd\n&\xbc\x89\x1f\xe2\xbbh\xc0\x1d\xbcY\xa0\xcb\xba$\r=\xbb\xcb\n\x94\xbb\xf2~;\xf8\xd3\xf2;\xbf\x0c\;\xdd\xe5\xf9;B\x9e\x10<m\xac;WH\xf1:\x1c?\xd2;\r=+\xb9\xc2\xe3\x12\xbb\xdb#\xaa\xbb\xcf\x7fn\xbb&\xff%\xbc\x04\xb6\xd1\xbbl\xb9\xc6\xbb\x84/\xfb\xbb\xcf\xfda\xbbs\x03\xe89o>(;\x9f}\x9b;$\xcf\x98;\x86@\xd4;a\xe9\x0b<\x9cU\xa3;\x83w@9T\xbd\x849\xc1\x19>\xbb-\x9b\xaf\xbb4\x81\x18\xbc%\x93\xa5\xbb\xda\x97\x9e\xbb\xc9h\xf78j\xe2\x0f\xba\xe8\x9df:h\x18\xdf;\xcao\x03<\xf4\xfc\x05<a@\xe0;io[;\x99\xb7+;^y$\xbb\xcb\xb8\x1f\xbaJ"\x03\xbcL\x1a/\xbc\x80"\x06\xbc\x0f\xfb\x1b\xbc\x8e\\\x98\xbb\xba\x89\x97\xba\x0c\xa2\x10;.]\xa3;\'\x0f\x0e;\xf7H\xd0;SY+<\x05E\xf5;\xb0X\xcc;\xd8\x7f@;p\x9c\x98:\x15\xf0r9MR\xa0\xbb\xed\xd2Y\xbbo\x8d\xd8\xbb_\xfe\xa0\xbb,\xf4\xee\xbb\x1e\x1c\xc5\xbb\x17\xc4\xa8\xba\xa9Z\xdd9}\x0e\x8a93\x93I;\xd1\xa8\x82;\xeb\xc1
;^\xa20;\x06\xe9\xe9;t\x9e\xcc;\xf1\x97\xc8;I\xba\xb1;:\xfe\x83:5\xcd\x80:\xcc\x8c\xb3:H\xbcg;\x99\x10\x8d;T\x99\t<\xf8\x14\x03<\xef\x80\xda;\xcf%\x07<\x06\xd3;Z[0<>\xf1\xde;\xae?2;a\xa4\x8c8\x872\x10;f\xbez:{\x87\xba\x87U\xae\xbb\xc5\xde\xf2\xbb\xdc\xee\xee\xbbO\xdd\xe58\xfa__\xbb\xe4/V\xbb\xf4RG\xbb\xccu\xba;\xc2q\xff\xbb@\x10\x00\xbci\xb7\x83\xbb7\x8f\xa7\xbbT\xa0\xc2\xbbQH\xd7\xbb\x9f\x00\xb7\xbb@\x98\xad\xbb\xcfs\9\x00(\xc6\xbb\xcd\x1c\xe7\xbb\x15\xc8\xfb\xbbs\xc9\xdf\xbb/w\xac\xbb?d\x8c\xbbT\xed\x06\xbcD{\x11\xbc\xaa\x1c\xda\xbb^\x13\x0f\xbc\xf1\x16\x88\xbbY\xaa\x05\xbc\x15\x7f\xc5\xbb<\xb5T\xbc\xfc\x0b\xe1\xbb\xaf(\xdd\xbb\xa0J\x84\xbb\xa6\x11\xff\xbb\xc4\x1f\xae\xbbL~R;A\xff\xa3\xbb-D\x9f:\xcf\xe4=:\x89R\x91;'1\x08<m\xb4\x10<\xad*\xc2;\xfd\x9c\x08<\n\x89\xeb;\x87\xb6\x05<c\x9d\x0e<\r!\xd5;\x98\xfe\x8f;;!9;S\xcc\r;d\x80\xaa;X\xae\xf2;W\x04\x0b:b\xb4\xa4;\xb6*\x9d;\x92'\x12<\x9d\x89\xee;\xd8\xa3\x86;\xea\x13\xd7\xba\xb7\xa4\xa89\x11\x8e\x8e\xbbs\xfd\xae\xbb\xf1A\xdc\xbb2\x8f\x86\xbb\xa1\x1b\xae\xbb\xf6\x9f\x95\xbb\xf9\x03\x9c\xbb\xec\xc4+\xbbd-;\xf5\xb4\x01<\\xbc\x04<\x16C\x0c<\x82O\xc0;u\xd0\x14<!B6;\xd7"\x96\xbb/\xacM\xba\xcfJ\xf9:R\xae\xa8\xbb\xde\x85\x1e\xbc\xd6\x16\xf2\xbb\x8bY\x0f\xbcU\xc6\x90\xbb\xcc\xda\xe1\xba\xed\xa8J\xbb\xc5S]\xba\xfd:\xe6;WA\x1b<GJp;){\x16<\x8d\xe1\x8e;\x05\xbd\x06<GO\t<\x95(\xed;\x92\xccH;U\xef\xf4;>\x8b?;7\x90\xa5;$\xcc\xad9L\x85\x18<\xef\xaf\x97;\xc5\xcf\xd7;\x99\xc4\xb3;w\x82\xd1;"\x12\xa6;?b\xb6;'w\xdf;w+\x15<O\x81\t<\xca\xdd\x0e<\x9c,&<\xd5\x8c\xf8;q\xc8\x9d;\x16R\xb9;\xc3\x1c.;\x1bi\x1a\xbb\xfb1\xa4\xbbF\x177\xbbA\x10\x9e\xbbQ\x06\x92\xbb\xca\x17\x0c\xbbA\x06\xe0\xba\xe5\x834\xbb\x8f\xdb\x0b\xbb\x18Y\xae\xb9!\xe0\x11\xba\xd2\x87\x04<8V\xda;\xf5\xbf\xef;9\x9f\xb5;\xfc>\xaf;r\x1d)<O\xdf\t<\x03p\xb2;\xa3\xe8\xa4;Viw:q&\xf8\xba\x7f\xf2\xac\xbb\x0bi\x17\xbaQ\xb2\xba\xbb\xaf\xa1\xcd\xbb\xd8\x9a\x1f\xbc\x12"\x1f\xbc\xc4\x15\x11\xbc\xaaD\xec\xbb\xac\xfd\xe3\xbb!\xef\x01\xbcb\xc9\xd7\xb8\t\x14+\xbb\xb3{_\xbb|\xb8D;\x10\x85\xed\xb9\xafw\xbe:\xd3\t\xf5;\x9a\xadk;j\x9b\xf3;\xd2\xe1\xbd;\xddv\x0c<\xdd\x19\xb9;M\xef\xa2;\xd0\x96\xe9;\xc3e3;']\xe0;\x158\xfd:\x8b\xe3#;ii\x93;\xc8\xb9\xdc;%\xf6\xc7;\xd0Y\x92;<rx;"\xe05;\xb6\r\x98;K\xb4\xb0;\xee\xb5;\xb7\xa5\xe1;K\xf7\xa1;\xf31\xf0;`\x07\x02<sP\x93;pe;;
the data is what?
I got this error when I run generate_RML2016.10a.py, who can help me, many thanks
Traceback (most recent call last):
File "generate_RML2016.10a.py", line 42, in
src = source_alphabet(alphabet_type, tx_len, True)
File "/home/niejinbo/PycharmProjects/dataset-master/source_alphabet.py", line 35, in init
self.src = mediatools.audiosource_s(["source_material/serial-s01-e01.mp3"])
AttributeError: 'module' object has no attribute 'audiosource_s'
Has anyone designed a blind equalization algorithm on this dataset, I have tried algorithms such as CMA/MMA, but the results are not ideal. Can anyone share the experience?
Hi Tim,
I was wondering if you've published (or plan to publish) the dataset + codebase associated with "End-to-End Radio Traffic Sequence Recognition with Deep Recurrent Neural Networks." I'd very much like to tinker with that kind of data (mostly look more closely at injecting additional frontend non-idealities and comparing classification performance with a baseline), and it'd be super helpful if I could use your work as a starting point.
Thanks!
Angie
When I run the program generate_RML2016.10a.py, the data can't be generated, so I checked and found snk.data() is None, it was cased by follows:
snk = blocks.vector_sink_c()
raw_output_vector = np.array(snk.data(), dtype=np.complex64)
I print the snk.data and I got a '()' and the length of raw_output_vector is 0, who can help me to fix this problem, many thanks
Hi developer,
I was trying to run generate_RML2016.10a.py, and the following run time error occurs:
Mapper warning: Could not load the module “fastcluster”.
The module “scipy.cluster.hierarchy“ is used instead, but it will be slower.
The 'cmappertools' module could not be imported.
The 'cmappertools' module could not be imported.
Intrinsic metric is not available.
The 'cmappertools' module could not be imported.
Traceback (most recent call last):
File "generate_RML2016.10a.py", line 3, in
from source_alphabet import source_alphabet
File "/root/dataset/source_alphabet.py", line 3, in
import mediatools
File "/usr/local/lib/python2.7/dist-packages/mediatools/init.py", line 45, in
from mediatools_swig import *
ImportError: No module named mediatools_swig
I have already installed gr-mediatools from source (saw a similar issue there but no response exists).
Please help.
I'm interested in determining the dataset's data rate or transmission rate. Can you assist with this?
After cloning the repo, when I try to run build_dataset
command from the terminal (I'm using ubuntu 18.04), I get the following error:-
I think the radioml/minsdr
image on the docker hub either doesn't exist or is still private.
I'm trying to run build the dataset using yonidavidson/minsdr
docker image from @yonidavidson's docker hub.
When I run generate_RML2016.10a.py, this error come out(only when generate am&amssb&fm)
$ ./generate_RML2016.10a.py
snr is -20
len(interp_taps) = 287
[mp3 @ 0x7fcfb8000b60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7fcfb8002040] Header missing
error decoding file
...
error decoding file
[mp3 @ 0x7fcff0000b60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7fcff0096c80] Header missing
error decoding file
all done. writing to disk
Is there something that my computer missed?
these three modulations signal data is wrong when I inspect frequency spectrum and power spectrum。
The document in gnuradio.channels.dynamic_channel_model indicates that the -noise_amp – Specifies the standard deviation of the AWGN process .
So, I think there should be noise_amp = np.sqrt(10**(- snr/10)), original code is the variance.
Besides, the Normalization of the energy may be the energy=np.sqrt(np.sum(np.abs(sample_vector)**2)),sample_vector = sample_vector / energy, rather than the following form, the former formula can promise after be normalized the total energy of a complex vector equals to one.
When I excecuted 'python generate_RML2016.10a.py', errors occurred when generating am, fm and amssb mod:
[mp3 @ 0x7f8130000c60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7f8130006380] Header missing
error decoding file
I think the my gnuradio 3.7 config is probably not right. How to deal with this?
Thanks!
Has anyone tested with real data?
I found that the prediction result is very inaccurate, does this only exist in the theoretical stage
Hey guys,
I'm trying to run the dataset generation tools from outside of the docker environment that you have setup I am getting an error in the transmitters.py:
ImportError: No module named mapper
I presume there is a dependency that I'm missing here. I noticed in the docker environment / build_dataset script that your running a setup_env.sh which I presume is adding some paths to the python path for execution.
Thanks for any help
Hello, i am working on signal classification problem using RADIOML 2016 dataset.I need to calculate signal features like gamma max , kurtosis and many of the other expert features. Since the signal points are complex , i have a doubt if i should take absolute value . I am new to the area of signal processing and a undergraduate student . It would really helpful if u could help me with calculating the expert features using these complex signal points.
thank u
I want to how to plot the constellation diagram of the energy units signals?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.