Collecting data from the TI AWR1642 via its serial port, this setup allows the user to choose one of several neural network architectures - convolutional, ResNet, LSTM, or Transformer. The selected network is then used for the recognition and classification of specific gestures:
- None (random non-gestures)
- Swipe Up
- Swipe Down
- Swipe Right
- Swipe Left
- Spin Clockwise
- Spin Counterclockwise
- Letter Z
- Letter S
- Letter X
- python 3.8+
- unzip (optional)
- curl (optional)
unzip and curl are used by the fetch script.
Install mmwave_gesture package locally:
git clone https://github.com/vilari-mickopf/mmwave-gesture-recognition.git
cd mmwave-gesture-recognition
pip install -e .You can run ./fetch script to download and extract:
-
data (20k samples - 2k per class) ~120Mb
-
models (Conv1D, Conv2D, ResNet1D, ResNet2D, LSTM and Transformer models) ~320Mb
To access the required data manually, follow the provided links to download the files. Once downloaded, manually extract the contents to the directories mmwave_gesture/data/ and mmwave_gesture/models/ as appropriate.
End result should look like this:
mmwave_gesture/
│ communication/
│ data/
│ │ ccw/
│ │ cw/
│ │ down/
│ │ │ sample_1.npz
│ │ │ sample_2.npz
│ │ │ ...
│ │ └ sample_2000.npz
│ │ left/
│ │ none/
│ │ right/
│ │ s/
│ │ up/
│ │ x/
│ │ z/
│ │ __init__.py
│ │ formats.py
│ │ generator.py
│ │ loader.py
│ │ logger.py
│ └ preprocessor.py
│ models/
│ │ Conv1DModel/
│ │ │ confusion_matrix.png
│ │ │ history
│ │ │ model.h5
│ │ │ model.png
│ │ └ preprocessor
│ │ Conv2DModel/
│ │ LstmModel/
│ │ ResNet1DModel/
│ │ ResNet2DModel/
│ └ TransModel/
│ utils/
│ __init__.py
│ model.py
...
The group name can differ from distribution to distribution.
gpasswd -a <username> uucpgpasswd -a <username> dialoutThe change will take effect on the next login.
The group name can be obtained by running:
stat /dev/ttyACM* | grep Gidchmod 666 /dev/ttyACM*The firmware code used for AWR1642 is just a mmWaveSDK demo provided with the version 02.00.00.04. Bin file is located in firmware directory.
- Close SOP0 and SOP2, and reset the power.
- Start the console and run flash command:
python mmwave-console.py
>> flash xwr16xx_mmw_demo.bin- Remove SOP0 and reset the power again.
If the board was connected before starting the console, the script should automatically find the ports and connect to them. This is only applicable for boards with XDS. If the board is connected after starting the console, autoconnect command should be run. If for some reason this is not working, manual connection is available via connect command. Manual connection can also be used for boards without XDS. Type help connect or help autoconnect for more info.
If the board is connected, the prompt will be green, otherwise, it will be red.
After connecting run plotter and prediction with following commands:
python mmwave-console.py
>> plot
>> predictUse Ctrl-C to stop this command.
The console can be used for easy data collection. Use log command to save gesture samples in .npz format in mmwave/data/ directory (or custom directory specified by set_data_dir command). If nothing is captured for more than a half a second, the command will automatically be stopped. redraw/remove commands will redraw/remove the last captured sample.
python mmwave-console.py
>> listen
>> plot
>> set_data_dir /path/to/custom/data/dir
>> log up
>> log up
>> redraw up
>> remove up
>> log down
>> log ccwpython mmwave-console.py
>> set_data_dir /path/to/custom/data/dir
>> trainor
python mmwave_gesture/model.pyNote: Default data dir is mmwave_gesture/data.
By default, conv2d model is used. Other models can be selected using set_model option.
python mmwave-console.py
>> set_model conv1d
>> set_model lstm
>> set_model transUse help command to list all available commands and get documentation on them.
python mmwave-console.py
>> help
>> help flash
>> help listen- Thanks to NOVELIC for providing me with sensors
- Filip Markovic
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent