faustomorales / qsl Goto Github PK

It would be nice to be able to see / filter by label values on index screen.
Our specific use case it quickly pulling up images that were tagged as 'unsure' to rereview as a group.

Since labelling configs could have multiple label configuration and types I understand this would not be trivial to have a good universal solution.
Initial Ideas:

• Filtering just does a plain string search of the label blobs.

Current work arounds are either for us to do it outside of QSL, or instruct labellers to 'skip' labelling those images, have to make assumptions to disambiguate skipped due to running out of time, vs skipped due to user error vs skipped due to unsure label.

Hi again! For one activity we had a lot of files to ignore, and this was a top request to make it easier to speed up labeling.

Description

Keyboard shortcuts are awesome and make labeling go so fast! For datasets that are a bit messier and have a lot of images to ignore, an "Ignore" keyboard shortcut would let you stay in this flow and not have to move to the mouse or use tab a bunch of times to make it to the Ignore button.

steps

• i ran make develop to contribute
• it used port 5002 because the default 5000 is occupied.
• when i open http://localhost:5002 it redirects to http://localhost:5000/auth/login

expected behaviour: redirect to http://localhost:5002/auth/login

There's no way to check what version of qsl you're using. Users should be able to check this.

It appears that newer versions of eel do not work properly with qsl. See #19

Hi, I was taking QSL for a spin and ran into some issues when accessing objects in a bucket outside my default region.

It might be nice to specify the region in the config—that way I could segment my data by region and then run multiple labeling sessions sequentially.

There might be a smarter way to do this, but from my quick look through the codebase it looks like checking the config could happen right about here and not be too intrusive:

Thoughts?

Currently when you load a batch the images are all selected. It would be nice to have this customisable, so that new batches can be un-selected by default.

When in batch labelling and I change the Target Batch Size it doesn't do anything until you submit the page, and sometimes it takes another submit.

Not completely clear what triggers it to take effect. Things I tried

• Using the arrow keys to go to the next batch and back does not change the batch size.
• Refreshing the page removes all the settings

After using 'enter' to save and progress twice the new batch size took effect.

ideally:

• changing the slider would instantly load more or fewer images, but I could see that being a pain with the static urls. If it forced a page reload I'd be ok with that
• Would be nice if it remembered my batch sizes between leaving the batch review page and coming back.

When I labelled the last 9 and hit enter UI stayed at 87% and did not show a new batch or anything and did not go to 100%

It looked/felt like it was hitting an error

Confirmed all labels are done:

ideally:

• After labelling the last batch to progress updates to 100%.
• Some UI message "Labelling complete. click here to return to the project page"

When i install via pip install --upgrade qsl click and eel are missing.
I'm probably doing something simple wrong since i see it in the Pipfile and lock file.

I also found if i manually install eel * i get v 0.16 and weird behaviour (page / images don't load or only semi load) , where as if i install v 0.14, the one in the lock file, QSL appears more reliable.

https://pypi.org/project/qsl/

$ pip install --upgrade qsl
Requirement already satisfied: qsl in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (0.2.38)
Requirement already satisfied: ipywidgets>=7.0.0 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from qsl) (8.1.2)
Requirement already satisfied: typing-extensions in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from qsl) (4.9.0)
Requirement already satisfied: filetype in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from qsl) (1.2.0)
Requirement already satisfied: comm>=0.1.3 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipywidgets>=7.0.0->qsl) (0.2.1)
Requirement already satisfied: ipython>=6.1.0 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipywidgets>=7.0.0->qsl) (8.21.0)
Requirement already satisfied: traitlets>=4.3.1 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipywidgets>=7.0.0->qsl) (5.14.1)
Requirement already satisfied: widgetsnbextension~=4.0.10 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipywidgets>=7.0.0->qsl) (4.0.10)
Requirement already satisfied: jupyterlab-widgets~=3.0.10 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipywidgets>=7.0.0->qsl) (3.0.10)
Requirement already satisfied: decorator in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (5.1.1)
Requirement already satisfied: jedi>=0.16 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (0.19.1)
Requirement already satisfied: matplotlib-inline in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (0.1.6)
Requirement already satisfied: prompt-toolkit<3.1.0,>=3.0.41 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (3.0.43)
Requirement already satisfied: pygments>=2.4.0 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (2.17.2)
Requirement already satisfied: stack-data in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (0.6.3)
Requirement already satisfied: exceptiongroup in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (1.2.0)
Requirement already satisfied: pexpect>4.3 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (4.9.0)
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (0.8.3)
Requirement already satisfied: ptyprocess>=0.5 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (0.7.0)
Requirement already satisfied: wcwidth in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from prompt-toolkit<3.1.0,>=3.0.41->ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (0.2.13)
Requirement already satisfied: executing>=1.2.0 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (2.0.1)
Requirement already satisfied: asttokens>=2.1.0 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (2.4.1)
Requirement already satisfied: pure-eval in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (0.2.2)
Requirement already satisfied: six>=1.12.0 in /user/local/home/thorn/virtualenv/lib/python3.10/site-packages (from asttokens>=2.1.0->stack-data->ipython>=6.1.0->ipywidgets>=7.0.0->qsl) (1.16.0)

I am working on Active Machine Learning. My model is able to make a top 5 class prediction for images.
E.g.:

• image1, top 5 ranking is A B C D E.
• image2, top 5 ranking is A E F D Z.

From what I understand, each MediaLabeler can only have one configuration, so if I want to label images that do not have the same five labels, I need to create multiple MediaLabelers.

Would it be possible to pass a list of options, s.t. for each image[i] it uses the options=options[i]?

Thanks!

Hi again! Also repeating, thanks for this awesome tool and it's so easy to use! This one was very easily gotten around by just changing our label names, but we didn't notice til a bit into the activity.

Description

If your labels contain a subset of one another filtering for the word contained in both will show results for both labels

Reproduction

• Have labels that contain each other
  • ex: "Cat" and "Not Cat"
• Filter for just "Cat"
• Notice that both "Cat" and "Not Cat" labels show up

Hi Fausto! What a great tool! I dunno if this falls under bug, quirk that needs some call out in the docs, or something else, but it was enough to cause some (mild) panic that filtering wasn't going to work until we figured out what was happening!

Description

On the "View Index" page, if your filtered results are fewer than what would show on the page you're currently viewing, it looks like there are zero filtered results. To view filtered results, you need to manually page back to a page that is low enough to contain the number of results that are in your filter.

Reproduction

• Label enough images so that you have multiple pages in the "View Index" window
  • in my testing this was more than 7 images
• Click the "View Index" button, notice you're on a high page number
• Filter for a term that has less than 7 results (or few enough to not make it to the page you're on)
• Notice that it looks like there are no results for the filter
• Page backwards a few times, and there are your results!

**Request: ** Stable Support for a larger number of items OR guidance on a good max.

I added 18k s3:// image paths and labelling ended up being very slow and a little unreliable (UI getting in stuck states)
The act of labelling 10 goes pretty slowly specifically during the update to the DB and load the new images.

Using the UI to add this many was also slow, being able to provide a txt file with URLs would be nice, Chrome almost froze twice with the large copy paste.

I've realized I only really want to label 5k anyway, so I'm about to retry with 5k and will report back on the experience. Then the fix may be suggested max per project.

(I was using it running on EC2 with portforwarded to my local machine, I don't expect this to be the cause, but for complete info)

Trying to use your amazing tool in jupyterlab and having problems starting it.

import qsl

params = dict(
    config={
        "image": [
            {"name": "Location", "multiple": False, "options": [{"name": "Indoor"}, {"name": "Outdoor"}]},
            {"name": "Flags", "multiple": True, "freeform": True},
            {"name": "Type", "multiple": False, "options": [{"name": "Cat"}, {"name": "Dog"}]},
        ],
        "regions": [
            {"name": "Type", "multiple": False, "options": [{"name": "Eye"}, {"name": "Nose"}]}
        ]
    },
    items=[
        {"target": "https://picsum.photos/id/1025/500/500", "defaults": {"image": {"Type": ["Dog"]}}},
    ],
)
qsl.MediaLabeler(**params)

returns me this
Error displaying widget: model not found

> jupyter labextension list
JupyterLab v3.2.9
/home/mya/.local/share/jupyter/labextensions
        qslwidgets v0.0.42 enabled OK (python, qsl)
        jupyter-matplotlib v0.11.1 enabled OK

/usr/local/share/jupyter/labextensions
        jupyterlab-plotly v5.3.1 enabled OK
        nbdime-jupyterlab v2.1.1 enabled OK
        jupyterlab-topbar-extension v0.6.1 enabled OK (python, jupyterlab-topbar)
        jupyterlab-system-monitor v0.8.0 enabled OK (python, jupyterlab-system-monitor)
        @jupyter-server/resource-usage v0.6.1 enabled OK (python, jupyter-resource-usage)
        @jupyterlab/git v0.34.2 enabled OK (python, jupyterlab-git)
        @jupyterlab/server-proxy v3.2.0 enabled OK
        @jupyter-widgets/jupyterlab-manager v3.0.1 enabled OK (python, jupyterlab_widgets)

Other labextensions (built into JupyterLab)
   app dir: /usr/local/share/jupyter/lab
        @jupyterlab/toc v5.2.9 enabled OK
        jupyter-gmaps v0.9.0 enabled OK
        jupyterlab-dash v0.4.0 enabled OK

Spoiler, I don't have access to root installation

I am able to view images when batchSize is set to 1. But, when increased to anything above 1, images are not getting displayed. Here's a google colab notebook where the bug can be observed - https://colab.research.google.com/drive/1YyuJW-6s_xHvssIJ-Cc5O6Aooh0GJ63O?usp=sharing#scrollTo=sStB-wQuRYrR