voxel51 / fiftyone-plugins Goto Github PK

I wanted to add a new plugin. in execute() function I select some samples. I want to show them when the operation is done. For this purpose in execute() function I added a line like this:

ctx.trigger("set_selected_samples", params=dict(samples=[ids]))

ids is a list of sample ids that have been selected. In the end, nothing happens. These samples cannot be shown to the user by using this operation on App. Do you have an idea what might be the problem?

Hey there :)
Thanks for the awesome work!
I'd like to create a fiftyone plugin to compute samples' timestamps, either from filename or file's metadata, to add the following filter options:

1. by timerange in general (e.g. 2023-01-23 to 2023-05-21). Could be just the unix timestamp.
2. by weekday (e.g. Mondays to Wednesdays). Could be 0 - 6.
3. by time (e.g. 8 am - 11 am). Could be 0.00 - 23.99

Usecase: I have a big dataset from construction sites and want to create a dataset from it that's made up evenly from different seasons, weekdays, daytimes,... So I need to filter by weekday for example, as I really don't need data from sundays...

Could someone helping me with the first steps and guide me a little?
Where do I start? I guess first step would be to compute a new metadata-field to give fiftyone access to the sample's timestamp?
Having this data I could compute other fields like time ( 0.00 - 23.99 ) and weekdays (0 - 6).
From there I might be able to find out the rest from browsing other plugins' repositories :)

Features

Rendering point cloud data stored in .pcd format

• allow user to specify orientation of lidar (similar to orientation of camera)
• use filepath as the source for pcd data by default, allow override for advanced use cases

Coloring point clouds by intensity or z-value using a configurable colorscale

• color by z value
• color by intensity
• configurable colorscale

Rendering 3D detections and polylines stored in FiftyOne’s default 3D Label types

• remove support for 2d types + additional 3d data
• add support for new 3D Detection types
• improve polygon support

Perspective camera model (pinhole) with support for position/rotation/zoom adjustments

• allow for customizing initial position

Support for mapping App field colors onto objects in the scene

• use existing colorscales (requires new @fiftyone/state module)

Support for filtering objects in the scene via the App’s filters

• inherit state from app + render what matches app filters

Support for selecting and tagging objects in the scene

• support label selection
• support label clearing
• fix label tagging

Publishing

• setup npm team/org
• publish initial version / release candidate for install testing
• publish v1.0.0

Docs

• install instructions
• settings

Hi Everyone, I would like to create a plugin which displays a Plotly image and executes some action based on the result. Here is a minimal example which hopefully makes it clearer what I am trying to do. I have also attached the output of this plugin - as you can see, the plot is not displayed.

from plotly.io import to_json
import json
import fiftyone.operators.types as types
import fiftyone.operators as foo
import plotly.graph_objects as go

class AcceptRejectModifiedImage(foo.Operator):
    @property    
    def config(self):
        return foo.OperatorConfig(
            name="segment_anything_declustering",
            label="Apply Segment Anything for Declustering"
        )

        
    def  resolve_input(self,ctx):
        inputs=types.Object()
        style_choices = types.RadioGroup()
        style_choices.add_choice("Accept", label="Accept")
        style_choices.add_choice("Reject", label="Reject")
        inputs.enum(
            "style",
            style_choices.values(),
            default="Accept",
            view=style_choices,
        )
        
        fig=go.Figure()
        fig.add_trace(
            go.Scatter(x=[10,10,30,30,10], y=[30,40,40,30,30], mode='lines')
        )
        plotly_json=json.loads(to_json(fig))
        plotly = types.PlotlyView(data=plotly_json['data'],layout=plotly_json['layout'],config=None,label="My Plotly")


        inputs.define_property(
            "component",
            types.List(types.Object()),
            view=plotly,
        )
        return types.Property(inputs)
    
    def execute(self,ctx):
        #do something...
        yield ctx.trigger("reload_dataset")

Thanks for any help!

I really like the functionality the plugins can bring to 51 but I have a setup where I run 51 remote in a docker.
When I run a plugin demanding paths like the add_samples, a user like myself, would expect to be able to add samples from my client computer and not need to know the file system structure of the docker image.
So, is there implemented (or is it feasible to do) a drag n'drop or a file picker for this to work more like how you usually interact with web page uploads?

Describe the problem

An error occurs when using the Compute Visualization plugin from FiftyOne brain in the app. This issue arises specifically when the view in the app is set to "patch view." The objective is to view images at the patch level, create embeddings for the patches, and make a subset of these patches for reclustering to perform more in-depth data curation. However, Compute Visualization fails when computing the visualization while on "patch view".

Code to reproduce issue

The issue arises when using the compute_visualization plugin in the app.

Steps to reproduce

1. Create FiftyOne detection dataset from an image dataset and a COCO JSON file with annotations.
2. Launch the app from a python script.
3. Navigate to the "patch view"
4. Run the compute_visualization plugin on patch level.

The following error will occur after a few seconds.

▼Error occurred during operator execution

Traceback (most recent call last):
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/operators/executor.py", line 277, in execute_or_delegate_operator
    result = await do_execute_operator(operator, ctx, exhaust=exhaust)
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/operators/executor.py", line 319, in do_execute_operator
    result = await (
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/core/utils.py", line 2343, in run_sync_task
    return await loop.run_in_executor(_get_sync_task_executor(), func, *args)
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/vintecc/fiftyone/__plugins__/@voxel51/brain/__init__.py", line 77, in execute
    fob.compute_visualization(
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/brain/__init__.py", line 405, in compute_visualization
    return fbv.compute_visualization(
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/brain/visualization.py", line 108, in compute_visualization
    embeddings, sample_ids, label_ids = fbu.get_embeddings(
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/brain/internal/core/utils.py", line 759, in get_embeddings
    embeddings = samples.compute_patch_embeddings(
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/core/collections.py", line 3188, in compute_patch_embeddings
    return fomo.compute_patch_embeddings(
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/core/models.py", line 1470, in compute_patch_embeddings
    return _embed_patches_data_loader(
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/core/models.py", line 1657, in _embed_patches_data_loader
    ctx.save(sample)
  File "/home/vintecc/.local/lib/python3.10/site-packages/fiftyone/core/collections.py", line 152, in save
    sample_ops, frame_ops = sample._save(deferred=False)
TypeError: cannot unpack non-iterable NoneType object

Extra information

I wanted to note that when computing visualization via the plugin, the following arguments are passed to the compute_visualization function:

fob.compute_visualization(
    target_view,
    patches_field=patches_field,
    embeddings=embeddings,
    brain_key=brain_key,
    model=model,
    method=method,
    batch_size=batch_size,
    num_workers=num_workers,
    skip_failures=skip_failures,
)

When calling the function when the app is in "image-view", the target_view variable is of the class 'fiftyone.core.view.DatasetView'.
However, when calling the function from "patch-view", target_view is of the class 'fiftyone.core.patches.PatchesView'.

The code fails when running sample_ops, frame_ops = sample._save(deferred=False).
I believe this is because sample is None, which makes sense as the code tries to iterate over the samples of the dataset, but the dataset consists of patches.

System information

• OS Platform and Distribution: Linux Ubuntu 22.04
• Python version : Python 3.10.6
• FiftyOne version: 0.24.1

Willingness to contribute

The FiftyOne Community encourages bug fix contributions. Would you or another
member of your organization be willing to contribute a fix for this bug to the
FiftyOne codebase?

• Yes. I can contribute a fix for this bug independently
• Yes. I would be willing to contribute a fix for this bug with guidance
  from the FiftyOne community
• No. I cannot contribute a bug fix at this time

The 'plugins' plugin has an operation to generate a plugin skeleton.
When selecting the 'Has Placement' option, the generated code (at least the one which can be previewed) does not contain valid code.

E.g., the preview code shows

def resolve_placement(self, ctx):
        return types.Placement(
            types.Places.SAMPLES-GRID-SECONDARY-ACTIONS,
            label="My placement label",
            icon="/path/to/icon.svg",
            prompt=False
        )

While it should be

def resolve_placement(self, ctx):
        return types.Placement(
            types.Places.SAMPLES_GRID_SECONDARY_ACTIONS,
            types.Button(
                label="My placement label",
                icon="/path/to/icon.svg",
                prompt=False,
            ),
        )

Note, the invalid type types.Places.SAMPLES-GRID-SECONDARY-ACTIONS
and also the returned type is a single tuple instead of a tuple of tuples, which in my case does not produce any icon at all.
Only when I change my plugin code to the second variant (tuple of tuples) an icon will be shown.

The cause of the error should be in the function
https://github.com/voxel51/fiftyone-plugins/blob/5c800f1ded53c285f8e17f37e1ad9b2472fa93e7/plugins/plugins/__init__.py#L1630C1-L1656C18

Tried to draw labels on top of video samples using frame-level labels, however frame_labels are not displayed. You can see in the screenshot below that I have a bunch of labels but none of them is available in the dropdown.

Do I understand correctly that this line has to be changed? for field in _get_fields_with_type(target_view, fo.Label):

In resolve_placement(..) calling fiftyone.operators.types.Placement(..) with incorrect types

• fiftyone.operators.types.Placement(..) expects param1=place, param2=view but is called wth param1=place, param2=button
  Is here an error? should Placement init not be expecting place and button?

Possible TODO

• Add ProgressBar.start_msg property in voxel51-eta
• Update all usages of ProgressBar in fiftyone to use start_msg rather than separate logging messages
• Update the progress callbacks introduced in #101 PR to use ctx.set_progress(label=pb.start_msg, progress=pb.progress)

Why? This would provide a bit more info to users about what is actually happening in our builtin operations. This may be especially useful if there is a builtin operation that uses multiple progress bars, in which case the naive implementation may involve two separate progress bars that "reset" progress back to 0% when the second progress bar starts.

One operation that could "want" to have multiple progress bars is @voxel51/io/import_samples when uploading media to a new location before adding samples:

Currently if the upload step happens, there is no progress bar, so the progress indicator in #101 is not particularly useful in this case as it will sit at 0% for awhile and then rapidly jump to 100% as add_samples() is a faster operation than copy_files().

An alternative to the double progress bar problem is this pattern:

progress1 = lambda pb: ctx.set_progress(progress=0.5 * pb.progress)
progress2 = lambda pb: ctx.set_progress(progress=0.5 + 0.5 * pb.progress)

I checked the operation of the io plugin with command "fiftyone operators list", but got "Failed to register operators for plugin @voxel51/io".

Fiftyone is the best dataset management tool I've ever used. I have a small piece of advice on label drawing.
The APP lacks interactive tools to save the beautiful labels on the image, and it will be more convenient if this plugin is added.
I now use the API foua.draw_labeled_image and set the complicate config to draw labels on an image. However, the quality of the label is not very high like the view in the app, like the following figure. I hope you can consider my suggestion.

I am sorry if I submitted this in a wrong place or category. I can't really find any information on this problem that I'm having and would appreciate any help. Basically, whenever I try to apply any augmentations using the Albumentations library in FiftyOne, the created images don't load at all.

Here is what I did and the outputs I got:

1. Created a custom dataset in FiftyOne and labelled it with CVAT
2. Followed this tutorial https://docs.voxel51.com/tutorials/data_augmentation.html to create 3 augmented images based on selected samples.
3. Run 'View last Albumentations run'. The output was a message: 'Failed to execute operator @voxel51/operators/set_view. See console for details.'
4. Opened the 'augmented' tag in FiftyOne. Output:
   
5. When opened each of those displayed a message: 'This image failed to load. The file may not exist, or its type (image/jpeg) may be unsupported.'
   The labels though seem to be correct, but the image doesn't load.
6. Went to the folder where the dataset is located. The size didn't change. I'd expect it to contain 3 extra images.

I tried deleting the dataset and importing it again as a fresh new one, but the problem persist.

Hi,

Iam developing a Plugin for Fiftyone, but unfortunately I can't set any view in the execute function.
Running execute gives med the following error popup:

"Failed to execute operator @voxel51/operators/set_view. See console for details."

I also tried the view plugins contained in the basic examples, but experience the same issue.

Thanks in advance for any advice,

cheers,

ML