We can add a feature in setup.cfg to install mcap and mcap-protobuf-support if you ask for it with pip install foxglove-data-platform[mcap,protobuf]. ROS1 support is more complicated.

ROS1

there's no way in setup.cfg to point to a specific index for dependencies for a package. That means that we basically have to ask users to manually pip install genpy --index-url https://rospypi... before installing mcap-ros1-support. This would be true also if we tried adding mcap-ros1-support as an optional dependency of foxglove-data-platform.

IMO we should vendor genpy ( foxglove/mcap#547 )and block this issue on that.

API now supports sorting, pagination, and basic filtering on the devices endpoint: https://github.com/foxglove/console/pull/691

Description
The python client or the web client does not handle datetime.min when querying events

• Version: 0.13.0
• Platform: Ubuntu 22.04.3 LTS

Steps To Reproduce

import os
from datetime import datetime, timedelta

from foxglove_data_platform.client import Client

if __name__ == "__main__":
    foxglove_api_key = os.environ.get("FOXGLOVE_API_KEY")
    device_id = os.environ.get("FOXGLOVE_DEVICE")
    client = Client(token=foxglove_api_key)
    events = client.get_events(
        device_id=device_id,
        start=datetime.min,
        end=datetime.now(),
    )

Expected Behavior
Would expect it to get all events from start to end. Instead it gives me an error:

    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: data/start must match format "date-time" for url: https://api.foxglove.dev/v1/events?...

This was requested by a user. I am not sure what the conventional way to do this in python is, but the client should expose a mechanism to close the connection so the caller can retry.

It would be nice if the library supported a "synchronous" argument. This could work by using the request ID returned from the import endpoint and polling the /pending-imports endpoint until either a success or failure appears.

Currently the behavior of the backend is to display a failure on retryable failures (which could resolve to a success eventually if it succeeds before the retry count). We probably need to coordinate around that somehow, probably by indicating in the pending-imports response when a failure status is terminal.

From the private data platform repo (https://github.com/foxglove/data-platform/issues/676):

In addition to the current client.get_messages() method, it would be useful in some cases such as message statistics gathering to skip decoding and just return tuples of (topic, raw mcap record) from a client.get_raw_messages() method.

Description
Although I see it in some of the pypi files, typing extensions doesn't seem to be getting downloaded when pip installing foxglove-data-platform

• Version: 0.1.0
• Platform: Ubuntu 20.04, Mac

Steps To Reproduce
Make sure that typing-extensions isn't already installed (pip list, pip uninstall)
Run pip install foxglove-data-platform
Verify with pip list that typing-extensions isn't installed

Expected Behavior
It's installed

Actual Behavior
It's not

Ideas for solutions

• Upgrade to python 3.8, where Protocol is in the typing package
• Debug issue with pip not installing the dependency
• have a try/except statement that tries to import it from typing first, then tries typing-extensions if that fails

e.g. https://github.com/foxglove/py-data-platform/blob/main/examples/downloading_data.py does not show updated signature

Another alternative is to remove altogether, and just point to the updated docs on the website from the README

Can you add an example of using the client to upload a bag to the data platform in the README? I'd like to try it out but want to make sure i'm using the client correctly.

Description

• Add device_name argument to create_event method, to correspond with REST API's request signature

If API responds with a 400-level status code, we should relay the message to the client. It sounds like currently we're relaying the status code but using default messages for the status.

• Should reflect REST API endpoint: https://foxglove.dev/docs/data-platform/api/data#list-coverage-ranges

Filter imports & pending imports based on file name.

Depends on https://github.com/foxglove/console/issues/750

Several customers have asked for the ability to iterate ROS 2 data directly using the get_messages() API.

Currently, get_messages() is only supported for ROS 1 data. For ROS 2, the only workaround is to download the raw mcap data using download_data(), then parse it using mcap and rclpy, which is pretty cumbersome. This would be slightly nicer if we at least had a way to iterate raw message data for unsupported serialization formats.

In some methods, error messages for a 4xx client error are ignored. We should include these when available.

I have lots of devices that need to be created via API. Would it be possible to get a method on the data platform client for creating devices so I don't have to do it via requests?

https://github.com/foxglove/rosbag#stay-in-touch

Topics, imports, and coverage will soon support pagination and sorting:

• https://github.com/foxglove/console/pull/714
• https://github.com/foxglove/console/pull/715
• https://github.com/foxglove/console/pull/716

Description

• topics arg for downloading data should be optional (to correspond with REST API)
  Code in question: https://github.com/foxglove/py-data-platform/blob/main/foxglove_data_platform/client.py#L284

Description
The Python Client API Reference states that the deviceID argument to delete_import() is deprecated. But in fact, it is required by the existing code base.

• Version: Release 3
• Platform: All

Steps To Reproduce

from foxglove_data_platform.client import Client
api_key = 'XXXXXXX'
client = Client(api_key)  
devices = client.get_devices() 
files = client.get_imports(device_id='XXXXXXX')
client.delete_import(import_id=files[0]['import_id'])

Expected Behavior
Deletes import and returns 200.

Should mirror REST API: http://localhost:3000/docs/data-platform/api/data#list-imports

Add optional args to get_imports call:

• deviceId
• start
• end
• dataStart
• dataEnd
• includeDeleted

e.g. list_events(sort_by=timestamp_nanos, ...)

Description

• get_topics call should have an optional includeSchemas arg, to correspond with REST API call

Code in question: https://github.com/foxglove/py-data-platform/blob/main/foxglove_data_platform/client.py#L450

It looks like calls to upload_data require the bytes to be fully loaded into memory. For large files this is undesirable - and maybe impossible. Having an api where the file can be read from disk and streamed would allow me to upload files without reading into memory first.

https://github.com/foxglove/py-data-platform/blob/main/foxglove/data_platform/client.py#L187
https://docs.python-requests.org/en/latest/user/advanced/#streaming-uploads