jpsml / 6-dof-inertial-odometry Goto Github PK

IMU-Based 6-DOF Odometry

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

6-dof-inertial-odometry's Introduction

IMU-Based 6-DOF Odometry

By João Paulo Lima, Hideaki Uchiyama, Rin-ichiro Taniguchi.

This repository contains the code for the paper "End-to-End Learning Framework for IMU-Based 6-DOF Odometry". You can find a demonstration video here.

Prerequisites

Python 3
TensorFlow
Keras
NumPy
Matplotlib
scikit-learn
Pandas
SciPy
numpy-quaternion
tfquaternion

Training

We provide training code that can use OxIOD or EuRoC MAV datasets.

Download the desired dataset and unzip it into the project folder (the path should be "<project folder>/Oxford Inertial Odometry Dataset/handheld/data<id>/" for OxIOD and "<project folder>/<sequence name>/mav0/" for EuRoC MAV)
Run python train.py dataset output, where dataset is either oxiod or euroc and output is the model output name (output.hdf5).

Pretrained models

Pretrained models can be downloaded here:

Testing

We provide code for trajectory prediction and visual comparison with ground truth trajectories from OxIOD or EuRoC MAV datasets.

Download the desired dataset and unzip it into the project folder (the path should be "<project folder>/Oxford Inertial Odometry Dataset/handheld/data<id>/" for OxIOD and "<project folder>/<sequence name>/mav0/" for EuRoC MAV)
Run python test.py dataset model input gt, where:

dataset is either oxiod or euroc;
model is the trained model file path (e.g. 6dofio_oxiod.hdf5);
input is the input sequence path (e.g. "Oxford Inertial Odometry Dataset/handheld/data4/syn/imu1.csv" for OxIOD, "MH_02_easy/mav0/imu0/data.csv\" for EuRoC MAV);
gt is the ground truth path (e.g. "Oxford Inertial Odometry Dataset/handheld/data4/syn/vi1.csv" for OxIOD, "MH_02_easy/mav0/state_groundtruth_estimate0/data.csv" for EuRoC MAV).

Evaluation

We provide code for computing trajectory RMSE for testing sequences from OxIOD or EuRoC MAV datasets.

Download the desired dataset and unzip it into the project folder (the path should be "<project folder>/Oxford Inertial Odometry Dataset/handheld/data<id>/" for OxIOD and "<project folder>/<sequence name>/mav0/" for EuRoC MAV)
Run python evaluate.py dataset model, where dataset is either oxiod or euroc and model is the trained model file path (e.g. 6dofio_oxiod.hdf5).

Citation

If you use this method in your research, please cite:

@article{lima2019end,
        title={End-to-End Learning Framework for IMU-Based 6-DOF Odometry},
        author={Silva do Monte Lima, Jo{\~a}o Paulo and Uchiyama, Hideaki and Taniguchi, Rin-ichiro},
        journal={Sensors},
        volume={19},
        number={17},
        pages={3777},
        year={2019},
        publisher={Multidisciplinary Digital Publishing Institute}
}

License

BSD

6-dof-inertial-odometry's People

Contributors

Stargazers

Watchers

6-dof-inertial-odometry's Issues

window size and stride

The window size and stride are set to be

window_size = 200
stride = 10

May I know how to determine these numbers? Thanks!

I am facing following error when I run test.py as
python3 test.py oxiod 6dofio_oxiod.hdf5 "/home/aisl/AI/6-DOF-Inertial-Odometry-master/Oxford Inertial Odometry Dataset/handheld/data3/syn/imu1.csv" "/home/aisl/AI/6-DOF-Inertial-Odometry-master/Oxford Inertial Odometry Dataset/handheld/data3/syn/vi1.csv"
2021-11-30 15:46:23.512258: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/aisl/catkin_ws/devel/lib:/opt/ros/noetic/lib
2021-11-30 15:46:23.512297: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
/home/aisl/.local/lib/python3.8/site-packages/quaternion/numba_wrapper.py:23: UserWarning:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Could not import from numba, which means that some
parts of this code may run MUCH more slowly. You
may wish to install numba.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

warnings.warn(warning_text)
2021-11-30 15:46:24.556396: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2021-11-30 15:46:24.556433: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (Z370M-S01): /proc/driver/nvidia/version does not exist
2021-11-30 15:46:24.556594: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING:tensorflow:No training configuration found in the save file, so the model was not compiled. Compile it manually.
Traceback (most recent call last):
File "test.py", line 69, in
main()
File "test.py", line 34, in main
[yhat_delta_p, yhat_delta_q] = model.predict([x_gyro[0:200, :, :], x_acc[0:200, :, :]], batch_size=1, verbose=1)
File "/home/aisl/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/aisl/.local/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 58, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'CudnnRNNV2' used by {{node model_3/bidirectional_3/forward_cu_dnnlstm_3/CudnnRNNV2}} with these attrs: [dropout=0, seed=0, T=DT_FLOAT, input_mode="linear_input", direction="unidirectional", rnn_mode="lstm", is_training=true, seed2=0]
Registered devices: [CPU]
Registered kernels:
device='GPU'; T in [DT_DOUBLE]
device='GPU'; T in [DT_FLOAT]
device='GPU'; T in [DT_HALF]

 [[model_3/bidirectional_3/forward_cu_dnnlstm_3/CudnnRNNV2]] [Op:__inference_predict_function_1578]

Trained model not found

Hello

The URL for the trained models are not found, give me error

Run the model in real time

Hi,
Have you tried running the model in any real time setup?
For example running it using tensorRT on nvidia Jetson?
If yes how did you perform the conversion?
Thanks

Error while training

Hi,
When I am running the command "python3 train.py euroc euroc_model_trained_by_me.hdf5" I am getting the following error

Traceback (most recent call last):
File "train.py", line 137, in
main()
File "train.py", line 114, in main
train_model = create_train_model_6d_quat(pred_model, window_size)
File "/mah/AI/6-DOF-Inertial-Odometry-master/model.py", line 115, in create_train_model_6d_quat
out = CustomMultiLossLayer(nb_outputs=2)([y1_true, y2_true, y1_pred, y2_pred])
File "/home/aisl/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/aisl/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py", line 699, in wrapper
raise e.ag_error_metadata.to_exception(e)
TypeError: Exception encountered when calling layer "custom_multi_loss_layer" (type CustomMultiLossLayer).

in user code:

File "/mah/AI/6-DOF-Inertial-Odometry-master/model.py", line 74, in call  *
    loss = self.multi_loss(ys_true, ys_pred)
File "/mah/AI/6-DOF-Inertial-Odometry-master/model.py", line 66, in multi_loss  *
    loss += precision * quaternion_mean_multiplicative_error(ys_true[1], ys_pred[1]) + self.log_vars[1][0]
File "/mah/AI/6-DOF-Inertial-Odometry-master/model.py", line 36, in quaternion_mean_multiplicative_error  *
    return tf.reduce_mean(quat_mult_error(y_true, y_pred))
File "/mah/AI/6-DOF-Inertial-Odometry-master/model.py", line 29, in quat_mult_error  *
    q = tfq.Quaternion(y_pred).normalized()
File "/home/aisl/.local/lib/python3.8/site-packages/tfquaternion/tfquaternion.py", line 30, in scoped_func  *
    return func(*args, **kwargs)
File "/home/aisl/.local/lib/python3.8/site-packages/tfquaternion/tfquaternion.py", line 341, in normalized  *
    return Quaternion(tf.divide(self._q, self.abs()))
File "/home/aisl/.local/lib/python3.8/site-packages/tfquaternion/tfquaternion.py", line 30, in scoped_func  *
    return func(*args, **kwargs)
File "/home/aisl/.local/lib/python3.8/site-packages/tfquaternion/tfquaternion.py", line 394, in abs  *
    return tf.sqrt(self.norm(keepdims))
File "/home/aisl/.local/lib/python3.8/site-packages/tfquaternion/tfquaternion.py", line 30, in scoped_func  *
    return func(*args, **kwargs)
File "/home/aisl/.local/lib/python3.8/site-packages/tfquaternion/tfquaternion.py", line 389, in norm  *
    return tf.reduce_sum(tf.square(self._q), axis=-1, keep_dims=keepdims)

TypeError: Got an unexpected keyword argument 'keep_dims'

Call arguments received:
• inputs=['tf.Tensor(shape=(None, 3), dtype=float32)', 'tf.Tensor(shape=(None, 4), dtype=float32)', 'tf.Tensor(shape=(None, 3), dtype=float32)', 'tf.Tensor(shape=(None, 4), dtype=float32)']

requirements.txt with specific module versions

Hello,

I would like to run your train.py but I am encountering some problems. In a different issue, you provided tensorflow and keras versions, but even with these versions there are still problems. Could you provide a requirements.txt with specific versions of all the python modules? And which python version should be used? It seems that python 3.8 and upwards do not support TensorFlow 1.13.

ValueError: Unknown layer: CustomMultiLossLayer

While running the test.py file with model and the input and ground truth file paths, I am getting the following error

Traceback (most recent call last):
File "test.py", line 69, in
main()
File "test.py", line 24, in main
model = load_model(args.model)
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/engine/saving.py", line 492, in load_wrapper
return load_function(*args, **kwargs)
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/engine/saving.py", line 584, in load_model
model = _deserialize_model(h5dict, custom_objects, compile)
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/engine/saving.py", line 274, in _deserialize_model
model = model_from_config(model_config, custom_objects=custom_objects)
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/engine/saving.py", line 627, in model_from_config
return deserialize(config, custom_objects=custom_objects)
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/layers/init.py", line 168, in deserialize
printable_module_name='layer')
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py", line 147, in deserialize_keras_object
list(custom_objects.items())))
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/engine/network.py", line 1056, in from_config
process_layer(layer_data)
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/engine/network.py", line 1042, in process_layer
custom_objects=custom_objects)
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/layers/init.py", line 168, in deserialize
printable_module_name='layer')
File "/Users/Gurtaj/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py", line 140, in deserialize_keras_object
': ' + class_name)
ValueError: Unknown layer: CustomMultiLossLayer

Would greatly appreciate your help on this.

Will there be pytorch version?

Looking forward to your reply. Thank you very much.

Version in Prerequisites

Dear authors, thanks for this great work. It would be great if we have the version for the prerequisites, specifically

Python 3
TensorFlow
Keras
NumPy
Matplotlib
scikit-learn
Pandas
SciPy
numpy-quaternion
tfquaternion

Thanks!

Requirements version incompatibility

Hi!

I am trying to reproduce your results, but cannot find requirements version, and this is leading to several bugs while running train and test scripts (testing with pretrained models).

Can you provide keras and tensorflow versions that support this code?

Thanks!

jpsml / 6-dof-inertial-odometry Goto Github PK

6-dof-inertial-odometry's Introduction

IMU-Based 6-DOF Odometry

Prerequisites

Training

Pretrained models

Testing

Evaluation

Citation

License

6-dof-inertial-odometry's People

Contributors

Stargazers

Watchers

Forkers

6-dof-inertial-odometry's Issues

Recommend Projects

Recommend Topics

Recommend Org