px4 / flight_review Goto Github PK

The idea is that when the user uploads the file, adds the issue description on the Additional Feedback field, and then checks an option like Create Github issue on PX4/Firmware, popping up a Github login if the user is not logged in yet. I assume that the backbone of the Flight Review would have to include one API like https://github.com/paulirish/git-open or https://github.com/node-gh/gh or similar and the binds between issue creation and the Flight Review capabilities. We could also iterate over this and present options for choosing what plots to show on the Github issue, parameter values, etc.

For example, for nodegh, would be something like a script that runs something like:

gh is --new --title <the_log_description_field> --message <the_additional_feedback_field> --user <the_github_user> --repo <master> --log <the_uploaded_log_file> --plots .... --parameters ...

We can add an option for link the flight log with an existent issue also. For example, in nodegh, that could be something like:

gh is <issue_number_or_name> --comment <log_description> --log <the_uploaded_log_file> --others ...

In the Flight Review side, we can add an option Link log to Github isse, which pops a dropdown list of open issues and/or having the option to the user write the issue number, though I prefer the first option.

Note: to keep track of the suggested in PX4/PX4-Autopilot#7701.

Each plot title should have a HTML anchor so you can create direct links to a specific plot. This is super important for issue reports / discussions.

The stabilized mode color for the google map is incorrectly shown as purple.

Here is the altitude graph which correctly shows stabilized mode with blue background:

Here is the google map view of the log, which show stabilized mode incorrectly as purple:

And here is the link to the log in question:
https://review.px4.io/plot_app?log=77a2d3d3-1834-40d0-bb0c-61b22aad9214

Here is another log with the same problem:
https://review.px4.io/plot_app?log=80daaa0a-6f12-467d-add2-27599c8fda38

How is the delta time (in blue) calculated here? The reason I ask is that we need to make very clear if the system slips that delta or just the logging process. That makes a huge difference. If it is merely the logging process we might consider moving all these quantities to a new, separate plot and do one for system performance (potentially including control system delay) and one for logging performance.

My primary concern is that people think that the system is unreliable in timing when its actually not.

Can we plot filtered current (battery_status.current_filtered_a) instead of current? The remaining percentage is also useful (what most users actually see).

http://logs.px4.io/plot_app?log=25f46bfb-4389-40e7-b06d-07f96094121e#Nav-Power

it would be beneficial to have the ability to define your own functions, such as subtracting mean of logs

The current plot on http://review.px4.io/plot_app?stats=1 does include all releases but not master. Given that a lot of testing is against master it would be great to list it in front of all releases.

@bkueng It would be great if we can display the geotag points on the map so that we know where the camera was triggered. They are currently logged in the camera_capture message.

Log : https://drive.google.com/file/d/0B3WIncZkmRb6Y0piUms0RjkyeTQ/view?usp=sharing

Should flight review use vehicle_status.nav_sate instead of commander_state.main_state for the vehicle mode?

Hello,
When I try to download parameters from a log, the .ulog file is downloaded not the paramters file.
This worked few days ago. Maybe since the KLM track download function ?

Thank for this very usefull flight log analysis.

Vivien

i setup a server with ubuntu 16.04 lts,but it can not plot anything.in chrome dev console,it show me websocks not work,and console show me

list index out of range
but i don't know where is wrong.can you give me some advice?thanks!

@bkueng Now that we start to have fixed wing logs we should try to plot airspeed vs. ground speed:
http://review.px4.io/plot_app?log=68e01eee-23ba-43ec-95d1-4e87b47213b8

This is from the SITL system, completely unreadable:

Zooming in and panning even worse:

Here's the best explanation:

It stays like this even after moving and zooming...
I use Chrome 56.0.2924.87 (64-bit) on Windows.

@bkueng In this log (which I haven't analyzed yet) there is either a topic stopping to update or the output becomes NaN. We need to have clear markers for that:
http://logs.px4.io/plot_app?log=d72b30cd-d713-4c43-9f32-30510657f2e6#Nav-Raw-Acceleration

Otherwise most developers do not realize what the issue is and think the topic stopped updating (or gave a constant value).

PX4 version 1.6.2 adds the feature:

• "New .ulg uLog logging format which directly logs uORB topics and offers more accurate time stamping. Supported in FlightPlot"

I tested 1.6.2 with a VTOL QuadRanger. I loaded 1.6.2 firmware, loaded my parameters from 1.6.1, calibrated the sensors etc. and did a few short flights.

The resulting logs load fine in FlightPlot under Windows (I cannot seem to get FlightPlot working on Ubuntu, but that's a different issue).

But when I try to upload to review.px4.io, I get "HTTP Error 400: Failed to parse the file. It is most likely corrupt."

I attach a sample log: log_8_2017-6-13-08-42-46.zip.

The logs are fine in FlightPlot so should be loadable by Flight Review?

I know space is at a premium, but I'm wondering if we can display these differently. For example this FW vehicle was having completely different issues with both magnetometers and airspeed sensors.

https://review.px4.io/plot_app?log=fe5f07bc-93ba-4ca1-a99d-fd3437847e7d

Could we have them all plotted individually, but only showing them if non zero?

@bkueng Here are a few potentially helpful plots we could generate. Some of them might require additional questions for the user.

Really need this for: PX4/PX4-SITL_gazebo-classic#82

Will try to do a PR if you don't have time.

@bkueng The system currently reports an internal server error on the browse tab:
http://logs.px4.io/browse

This was potentially the lag log that got uploaded and might be related to the breakage (or one of the SITL logs):

http://logs.px4.io/plot_app?log=e322bc62-56e4-48e9-8dd3-136e2799e789

It would be beneficial to have fft function.

In case of High rate log the Spectral Density fails and this is how the plot looks like:

Currently the graphs do not align in time - e.g. if messages from one topic stop updating it crops the graph and they are unaligned. All graphs should start and stop at the min and max times for which we have any data, not just the topic they are graphing.

The current state can make crash analysis quite confusing, because you carefully need to read the time labels.

These are really helpful, but there are a few cases where NAN is used normally. For example an IDLE position setpoint will have a lat/lon set to NAN.

Hello guys,
Thank you for this amazing tool! Is it possible to access/download the whole public flight log database (instead of writing a scraper that pulls it)? This would make it easier to experiment with all the timeseries available so far and feed them to some LSTMs etc.

Would be nice if we could display the date and time from the log where the logging duration etc. is

The graph below would be a lot more readable if we have SITL in a separate graph. I would not remove it because its important validation.

@bkueng We should try to load the GPS track immediately (cache the KML export on upload on the server) and then offer the other plots later. That way people can start looking at some of the data already and the 20 seconds wait time isn't feeling like forever.

@bkueng This service carries sensitive information. I think it should only be provided over SSL. You can generate a free certificate with letsencrypt. An auto-renewal and easy installer is found here: https://certbot.eff.org/all-instructions/

/cc @LorenzMeier

Run locally, got this error.
This is the ulog:
https://review.px4.io/plot_app?log=2f9610e9-87fa-409a-bb8f-0fab6c61971a

% ./serve.py -f log_206_2017-12-14-16-22-02.ulg                                                                                                                                                                                 eric@eric-Lenovo
[8157:8196:1215/152545.162167:ERROR:browser_gpu_channel_host_factory.cc(107)] Failed to launch GPU process.
Created new window in existing browser session.
GET[log]=/home/eric/all_code/flight_review/log_206_2017-12-14-16-22-02.ulg
Downloading http://px4-travis.s3.amazonaws.com/Firmware/master/airframes.xml
<class 'IndexError'> (vehicle_local_position_groundtruth): list index out of range
Traceback (most recent call last):
  File "/home/eric/all_code/flight_review/plot_app/main.py", line 185, in <module>
    plots = generate_plots(ulog, px4_ulog, db_data, vehicle_data)
  File "/home/eric/all_code/flight_review/plot_app/configured_plots.py", line 187, in generate_plots
    mark_nan=True, use_step_lines=True)
  File "/home/eric/all_code/flight_review/plot_app/plotting.py", line 467, in add_graph
    p.step(x='timestamp', y=field_name, source=data_source,
AttributeError: 'Figure' object has no attribute 'step'

It's very difficult to see the dark gray altitude control section over a satellite map.

http://logs.px4.io/plot_app?log=4011b562-64a3-494e-8685-782414499d52

Since setpoints are values that are effective until the next setpoint is issued, a plot of them should be more like a bar graph than a line. With the current graphs, connecting the setpoint updates with lines makes it visually appear as though the setpoint is changing linearly with time between updates.

This characteristic is more readily apparent with flights and/or vehicles that have slow setpoint updates. For example, my vehicle's yaw rate setpoint changes only once a minute or so, and only by large values. The graph plots end look like a sawtooth, rather than a square wave. See the following plot for an example.

It would be useful when downloading KML to make the attachment name based off the uploaded log name rather than track.kml.

We should have a column in the log list for the PX4 version (e.g. 1.5.2) and the OS version (on Linux the kernel version, on NuttX the NuttX version, e.g. 2.4.2. or 7.2.1).

@bkueng I would like to ensure we improve the visibility of the auto test system and show all relevant branches (master, beta stable) in an overview page. The overview page should show the 2D ground trajectory plots of all tests and then link to the main plots.

Under statistics we only seem to show a short list of airframes, is there a way to show them all?

It's not clear when a VTOL is in MC or FW mode.
http://review.px4.io/plot_app?log=d22eb960-a60c-44de-a643-11070a868408

Could we add lines indicating any change in vtol state (MC, TRANSITION, FW), or would that be too much clutter?

See this plot:

http://logs.px4.io/plot_app?log=b3dc79d6-6c5a-4178-a459-fbc8291d62b8

Clearly the ground truth is just the GPS since there is so much random noise in it.

Drive costs on AWS are somewhat expensive, we should move our storage to S3, there's lots of python bindings for S3 nowadays, I'll submit a PR as soon as I can, or if there's anyone interested in helping just let me know I can set you up with S3

The link to the detailed plot should be something other than the number, because its a single character: http://logs.px4.io/browse

I'm testing a new estimator here (PX4/PX4-Autopilot#6045), and it always shows the logs as EKF2.

http://logs.px4.io/plot_app?log=8d8211a2-93cb-4b22-86b3-ee04730e51e5

Do I need to set a field somewhere?

Tried to upload a log today, there is some Problem, as it always returns HTTP 500 on logs.px4.io.

I don't know who to contact on this issue or if you also operate the px4.io infrastructure.

From @kd0aij: I just discovered that my description for upload #6 is very wrong. It turns out the the parameter file load did not work correctly, and I wasn't even running the right attitude estimator.

Simplest option is probably just to delete that flight report.

@bkueng I think you need to subsample the KML track to 5 Hz or lower as the current data seems to overwhelm Google. The map is often broken because of it.

@bkueng Now that the firmware side is in (PX4/PX4-Autopilot#6074), we need plotting for vision topics.

There should be new plots :

1. External position, velocity (vehicle_vision_position and relevant fields from vehicle_local_position_groundtruth)
2. External atttiude, angular velocity (vehicle_vision_attitude and relevant fields from vehicle_attitude_groundtruth)

It would also be great if you can add ground-truth to the existing local position / velocity plots. I will add plumbing the Firmware side to log data from ATT_POS_MOCAP and in the future from mavlink/mavlink#660 into vehicle_groundtruth_position.

The plotting system should check whether these topics exist and plot them if available. I'm not sure if it does this already.

A few suggested changes to the email sent by Flight Review:

• Add the title to the subject
• Add the description to the body

This will make searching and archiving easier

We need to be able to see the number of flights, the number of different airframes (identified by the unique ID of the MCU) and the number of hours per release and accumulated (all previous releases up to this release).

This is required to get an idea about the testing coverage we have achieved for a particular release. We might need some fuzzy matching and have two different hour counts: One for a particular minor release (1.5.2) and one for the major release (1.5), which is generally a bit more informative.

Currently flight review plots rc_channels as "Raw Radio Control Inputs" to show user input. This is the calibrated RC input which is then mapped to manual_control_setpoint x,y,z,r, etc.

In PX4 RC enters the system through input_rc -> rc_channels -> manual_control_setpoint.
The message manual_control_setpoint is what the controllers actually use and it's more helpful for debugging. Additionally it's the only thing logged when using a gamepad/joystick over mavlink instead of RC.

In summary, I think we should be logging and plotting manual_control_setpoint instead of rc_channels. I can help make the transition if we're in agreement.

From http://logs.px4.io/plot_app?log=00b0b048-39bd-45cd-910b-c3d54b0f39ad (PX4/PX4-Autopilot#7463)

Flight review estimator watchdog doesn't show anything.

This log has filter fault flags and innovation check flags.

00b0b048-39bd-45cd-910b-c3d54b0f39ad.ulg (2).pdf

@bkueng i see you added average speed. For VTOL it would be great to have average speed separated for MC and FW mode. The sum of these doesn't say much but individually they do