Data Comets is a interactive PX4 flight log analysis tool. With Data Comets you can encode flight data onto the flight path, filter and brush the data by time, and much more! Data Comets was designed and is in development as a part of a information visualization design study from the Visualization Lab @ Northeastern University’s Khoury College of Computer Sciences. The tool is ready to be used but the project and the research is ongoing and seeking user feedback. Learn how you can help Data Comets and our design study in the contributing section below.

To try out Data Comets for yourself you can visit datacomets.com. The hosted version is limited to logs smaller than 32mb. For larger logs and better performance/load times please use the local version. Uploaded logs are not saved once they are parsed and displayed. Refreshing the page will delete the log data from your browser as well.

You can use Data Comets locally with python 3. Clone or download this repo, then using the terminal run the following commands in the directory.

pip install -r requirments.txt
python DataCometsLocal.py

Data Comets will then run on your local host and should open automatically or it can be accessed by going to http://localhost:5000/ with your browser.

The map shows the recorded flight path, you can also encode the estimated flight path and set points from the settings menu that you can access with the gear icon at the top left of the map

The flight path uses color to encode any data selected from the flight log. Select data such as altitude, velocity, or jamming to see where data was recorded or expected not just when. You can change the selected data by clicking the map icon found on each line chart

You can prowse all the data in the log using the line chart tree. The overview tab show a highlevel view of some important charts similar to the charts found on the flight review tool. You can look through all the data from the flight in the all tab where the data is orgainzed into a drop down tree by the message file they belong to. You can pin data you want to compare or get a closer look at with the pin icon found on every linechart. Pinned data will apear in the pin tab.

Data Comets is fully interactive! Brush the timeline at the bottom to filter the amount of time shown. Use this to reduce visual cluter on longer complex flight paths, or to drill down into certain trouble areas of a flight. The line charts will update to show the time windo selected. You can also use the brushing window to play back the flight by scrubing the window as you please. All the charts are linked! Hover over the line charts or map to see that position on the other charts.

Interested in contributing? Even just giving your feedback is a huge help! Below are a few of the ways anyone can help contribute to this project.

1. Give Us Your Feedback! We are currently in the evaluation phase of our design study and are seeking user feedback
   - Leave questions, comments, ideas, and feedback (positive or negative) over on the discussion topic here
   - If you have been using the tool please take this survey to help us evaluate the project
   - Have you found anything interesting in your logs with Data Comets? We would love to hear your stories using
   using the tool and potential use them as case studies for our publication.
   - Raising issues on this repo with any bugs or suggestions or questions you may have.
2. A JavaScript and or web-based ULog parser. This would be a huge help for not only this project but other and future PX4/Dronecode projects on the web. Currently we are using PyUlog package to parse the logs into a suitable format for the web. A native JS parser would simplify and speed up things quite a bit.
3. Adding to the Lexicon. In the Lexicon folder there is a json file which contains information about all the possible data you can find in any given ULog. This was gererated by parsing through the .msg file source code and is not nearly complete. Filling in info about proper titles, descriptions, and units will help future versions of Data Comets. The lexicon will eventually be implemented to give all charts more accurate labels and encoding. The lexicon will also help serve ULog documentation in general.
4. If you are confident enough for changes to the code feel free to submit a pull request and I will be more than happy to review it and merge good changes or additions.

What can you expect from Data Comets in the future? I still have a lot planned for this tool and similar research projects so stayed tuned! Things to expect in the nearish future include

1. Changes, fixes, and additions based on community feedback
2. Performance optimizations to better handle very large logs (1+ hours flights)
3. Improved code styles, architecture, and documentation
4. Implementing the Lexicon to give all plots accurate titles and unit labels
5. Search functionality
6. 3D drone orientation and position plots
7. Pure web-based implementation with out the need of a python backed for parsing