vitrivr / vitrivr-ng Goto Github PK

Currently, the user has no way of knowing that an mlt query was executed and what was used a basis for that mlt query. There are multiple ways to implement this, we haven't decided on one yet.

The late filter option based on the 5 color tags which can be assigned to individual results does not appear to have any effect.

For some use cases, it would be nice to have the possibility to query for a subelement of a retrieved element, be that a thumbnail of a video or an image. This could be implemented for example by adding a cropping functionality to the QuickViewerComponent.

Currently, the metadata filters just generate a checkbox for every existing value for a given key (on the lsc branch). This could use two improvements:

• if metadata values are found to be numbers (special consideration for length), use sliders
• limit the number of checkboxes to some reasonable number (1000 per key?) so the UI doesn't crash

The current implementation leads to huge performance problems on various browsers (firefox, chrome, safari) and operating systems (mac, windows). Real Pagination might be very helpful here.

Hi,

I created a custom feature module that assigns each image a random score(one float number). I additionally created a boolean slider in the vitrivr-ng config file, to query all images that are in a specific range. How can I specify which feature to query for in the vitrivr-ng config file, i.e. what name should I use to reference the feature module I created?
Specifically, I created the feature module RandomAestheticScore and extracted the generated value for each image to the db. Furthermore, I added a new field in the cineast.json file referencing the feature module, under the features property called aestheticscore(see attachment) and added a slider in the vitrivr-ng config file under the boolean property(see attachment).

Best,

Ribin
vitrivr-config-files.zip

currently, if there are two concepts with the same name but a different description (e.g. crane), they are not distinguishable by the user.

On dev, you can currently access individual objects at /mediaobject/:objectID, and it would be nice to have a similar UI for individual segments. One can imagine both /mediasegment/:segmentID and /mediasegment/:objectID/:segmentID as sensible endpoints.

Currently, the FusionFunction interface is responsible for scoring both segments and objects. Ideally, this is separated into two interfaces with an extension of the config so users can choose which scoring / fusion function to use at runtime.

The OpenSeaDragon inclusion which enables support for IIIF and zoomable images in general, added in https://github.com/LukasStoeckli/vitrivr-ng/tree/dev should be merged here as well. Maybe some cleanup is necessary first. Any thoughts @silvanheller @ppanopticon ?

The only thing i see keeping us from upgrading are the es6 / es7 in polyfill.js. Drawing was broken on firefox nightly for me - the fix proposed in a related github issue:zloirock/core-js#412 did not work for me.

As there's no critical reason to upgrade to angular 8, i'll just leave this issue here as a reminder.

Once we've upgraded, we should upgrade all dependencies (videogular prevents us from upgrading to 7 before we have Angular 8)

Dear devs,

Is it possible to have the Range slider with floating-point values( for example in the range between 0 and 1)?

Best,

Ribin

For debugging purposes, it would be rather useful to have a text-field where you can late filter by segmentID.

One ngx-videogular has deployed their version update, we can upgrade to Angular 10.

During VBS (or other competitions) one might realise that a certain tag is not useful - a not filter for such particular tags might prove useful

The checkboxes for late filters based on metadata are not updated when new options become available via query results. Adjusting a weight slider triggers the update.

When switching views, it would be very convenient to be able to filter segments which were marked in a previous view

Two issues in the temporal scoring view:

• Currently, as you scroll down the checkbox to hide a specific item disappears
• If you simply close all visible items, vitrivr-ng does not automatically continue scrolling.

Since the last release, nobody updated the pre-built version of the UI in the github releases. Could someone have a look at this and how to prevent this in the future?

Since Tag Priorization is currently not supported in Cineast, it should be disabled by default

Hi, I am a newer to Vitrivr. After all dependencies installed, there is a error when I try to start the server using ng serve. Both Cineast and Cottontaildb are running at background normally.

Error: openapi/cineast/model/queryConfig.ts:12:21 - error TS2307: Cannot find module './set' or its corresponding type declarations.
12 import { Set } from './set';

See attached screenshot. I think this is possibly an issue in the objectdetails.component.html file - maybe the .segments call then needs to be stringified?

Dear devs,

I want to display certain features in the object detail view. Let's say I have a feature my_custom_feature extracted to the database for an image. Now, when I click on the image, I want to show the value of my_custom_feature in the details view. Given an image, how can I access its my_custom_feature value from vitrivr-ng?

Best,

Ribin

We got as feedback during the evaluation of @cristinailli that users would like some sort of indicator which view (mini-gallery, list) is active

Hi,

I would like to add the ability to view the content sorted by a custom feature that I created.
I succeeded in creating a custom feature and I can retrieve content from the database using the RangeBooleanRetriever.
How can I rearrange how the content is presented in the UI? If I want to add additional UI elements, what is a good place to start, i.e. which files should I modify?

Best,

Ribin

After running ng serve I was getting the following error message in the console:

`Uncaught TypeError: three_obj_loader_1.default is not a function
    at eval (webpack-internal:///./src/app/shared/util/m3d-file-loader.util.ts:8)
    at Object../src/app/shared/util/m3d-file-loader.util.ts (main.bundle.js:1028)
    at __webpack_require__ (inline.bundle.js:55)
    at eval (webpack-internal:///./src/app/shared/components/m3d/m3d-loader.component.ts:15)
    at Object../src/app/shared/components/m3d/m3d-loader.component.ts (main.bundle.js:724)
    at __webpack_require__ (inline.bundle.js:55)
    at eval (webpack-internal:///./src/app/research/containers/m3d/m3d-loader-dialog.component.ts:13)
    at Object../src/app/research/containers/m3d/m3d-loader-dialog.component.ts (main.bundle.js:419)
    at __webpack_require__ (inline.bundle.js:55)
    at eval (webpack-internal:///./src/app/research/containers/m3d/m3d-query-term.component.ts:14)`

I checked the 'three-obj-loader' module but couldn't find "OBJLoaderProto" in the module. Is this a local configuration error or error in Vitrivr UI?
My machine specifications are:
Ubuntu 17.10
Node 8.10
Web Server: python3 http.server
I have set up ADAMPro, Vitrivr-ng and Cineast in the same directory from where I am running the server.
I changed these few lines of code and the UI loads, although data is still not loaded.

t a/src/app/shared/util/m3d-file-loader.util.ts b/src/app/shared/util/m3d-file-loader.util.ts
index 33321ee..a0b63b2 100644
--- a/src/app/shared/util/m3d-file-loader.util.ts
+++ b/src/app/shared/util/m3d-file-loader.util.ts
@@ -1,13 +1,13 @@
 import * as THREE from 'three';
-import OBJLoaderProto from 'three-obj-loader';
+//import OBJLoaderProto from 'three-obj-loader';
 import STLLoaderProto from 'three-stl-loader';
 
 /* UGLY: Load the model-file loaders. */
 
 let Loaders = THREE;
-OBJLoaderProto(Loaders);
+//new THREE.OBJLoader(Loaders);
 let OBJLoader = (Loaders as any).OBJLoader;
-let STLLoader = STLLoaderProto(Loaders);
+//let STLLoader = STLLoaderProto(Loaders);
 
 /* /UGLY */
 
@@ -80,7 +80,7 @@ export class Model3DFileLoader {
:...skipping...
diff --git a/src/app/shared/util/m3d-file-loader.util.ts b/src/app/shared/util/m3d-file-loader.util.ts
index 33321ee..a0b63b2 100644
--- a/src/app/shared/util/m3d-file-loader.util.ts
+++ b/src/app/shared/util/m3d-file-loader.util.ts
@@ -1,13 +1,13 @@
 import * as THREE from 'three';
-import OBJLoaderProto from 'three-obj-loader';
+//import OBJLoaderProto from 'three-obj-loader';
 import STLLoaderProto from 'three-stl-loader';
 
 /* UGLY: Load the model-file loaders. */
 
 let Loaders = THREE;
-OBJLoaderProto(Loaders);
+//new THREE.OBJLoader(Loaders);
 let OBJLoader = (Loaders as any).OBJLoader;
-let STLLoader = STLLoaderProto(Loaders);
+//let STLLoader = STLLoaderProto(Loaders);
 
 /* /UGLY */
 
@@ -80,7 +80,7 @@ export class Model3DFileLoader {
      * @param manager LoadingManager to use together with the loader class.
      */
     public static loadSTLFromPath(path: string, callback: Function, manager?: LoadingManager) {
-        let loader =  new STLLoader(manager);
+        let loader =  new THREE.STLLoader(manager);
         loader.load(path, (geometry: any) => {
             if (geometry instanceof BufferGeometry) {
                 callback(new THREE.Mesh(new THREE.Geometry().fromBufferGeometry(geometry), new THREE.MeshNormalMaterial()));
@@ -127,4 +127,4 @@ export class Model3DFileLoader {
             console.log("File '" + path + "' has an unsupported format.");
         }
     }
-}
\ No newline at end of file
+}

Dear devs,

When I apply two boolean queries, can do a logical "AND" on the results?

Best,

Ribin

After merge of vitrivr/cineast#161 and vitrivr/cineast#162 , as well as #57 and #54 a holistic review and cleanup of the REST api is due.

In particular, reviewing the OpenApi Specification (OAS) of Cineast and the process on how it is included in vitrivr-ng (see #54 as well as enforcinig consistency in naming of endpoints:

• Review OAS client code generation process

Shall be resolved in coordination with vitrivr/cineast#163

It would be a desirable behavior if the host used for the websocket connection would default to whatever host the ui is currently served from, instead of having to specify it explicitly. This would be especially useful in cases where the ui is served from a machine which can be reached by different host names or through a proxy.

We need to add the following categories:

• Audio
• Captioning (NN Captioning)

For flexibility, would it be possible to have the list of text categories configurable? I'd suggest that the config has an array like ["Text on Screen": "text", "Subtitles": "subtitles"] where the key is the text which appears and the value is the category sent to cineast (or introduce objects per element with i.e a flag which makes the boxes ticked by default).
This would give us maximum flexibility in the future and eases load on the UI development side, especially since different datasets will want different categories.

It would be nice to have a late filter for the score similar to the range filters we currently have for metadata. Users would then be able to filter out all elements below (or above, although that is not a likely usecase) a certain score.

This would be implemented in the sidebar on the right which can be toggled by clicking on it.

I on the latest commit of the dev branch and am getting an infinite stream of this error filling up my console. This error stops occurring when I am outside the window but as soon as my mouse even hovers over an angular component, this error start filling up the console.

Uncaught TypeError: ng.probe is not a function
isDebugMode ng-validate.js:1
l ng-validate.js:1
e ng-validate.js:1
invoke zone-evergreen.js:364
runGuarded zone-evergreen.js:133
wrap zone-evergreen.js:117

Both angular-slider/ng5-slider and its supposed successor for angular 9+, m0t0r/ngx-slider are not very active.
We might migrate to ngx-slider if it ever officially becomes part of the angular-slider org (m0t0r/ngx-slider#6). Alternatively, we can hope that range sliders become part of Angular Material Sliders

I am on the latest dev versions of both cineast and vitrivr-ng along with the latest docker image of cottontaildb. When making a image based request from vitrivr-ng to cineast i get the following errors in the cineast console -

WARN o.v.c.c.d.c.CottontailSelector - Error occurred during query execution in getUniqueValues(): INVALID_ARGUMENT: [878, d5612381-6ca1-4fb0-a28c-2b31b0de1580] EXECUTE QUERY failed because of binding error. Failed to bind FROM due to database error: Failed to bind 'warren.cineast.features'. Entity does not exist!

2021-07-11 18:30:13.420 [qtp1955587357-44] ERROR o.v.c.c.u.j.JacksonJsonProvider - Could not map JSON string '{"queries":[{"stages":[{"terms":[{"type":"IMAGE","categories":["globalcolor","localcolor","quantized"],"sliderSetting":1,"data":"data:image/png;base64,iVBORw..."}]}],"config":null}],"config":{"queryId":null,"hints":["exact"]},"timeDistances":[],"maxLength":600,"messageType":"Q_TEMPORALV2"}' to POJO. Please check your object definitions.

com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type 'org.vitrivr.cineast.api.messages.interfaces.MessageType' from String "Q_TEMPORALV2": not one of the values accepted for Enum class: [QR_TEMPORAL, QR_SIMILARITY, QR_START, Q_SEG, Q_SIM, QR_ERROR, M_LOOKUP, Q_NESEG, PING, QR_SEGMENT, SESSION_START, QR_METADATA_O, QR_METADATA_S, Q_MLT, Q_TEMPORAL, QR_END, QR_OBJECT]

As an enhancement, the quick-viewer-component could show metadata for the displayed segment / object.

Steps to reproduce:

• retrieve an object with a segment containing metadata
• load neighboring segments
• filter for metadata which neighboring segments should also have
• Expected behavior: neighboring segments remain visible
• Bug: neighboring segments disappear

When something goes wrong during the loading of the config, there is an exception which is shown in the console but no other indication to the user that something went wrong. This might be confusing, since it then also defaults to an internal config which might be very different from what the user intended. It would be great to add some more user-friendly indicator in case a config cannot be loaded and the default is used instead.

currently, the user has to manually click the reset button in most cases to force to config to load properly

Dear devs,

I would like to change the default padding color(which is currently green)) of the preview results. Which file do I need to modify to do so?

Best,

Ribin

The use case for this feature is as follows: Find as many different objects as possible for your query. This is covered neither with the mini-gallery nor with the list views.

Actually, all views "share" the same scroll position. So if I scroll down, e.g., the list view and then open the gallery view, I'll directly be at the bottom too. In other words, all "movements" are registered by all views, even those currently not displayed. This is quite annoying, because when you go back to the tab you were before, you first have to look for where you left off.

From vitrivr/cineast#108 by https://github.com/mpas97

Dear developers,

I have set up the vitrivr stack according to the getting started tutorials. I could successfully query images via the vitrivr-ng UI , by running cineast locally as a web server on my pc. However, the retrieved images appear to be blank. I assume, this is caused by a wrong setting for the objects and thumbnail paths.
In the attachments you will find a screenshot of the UI, the config file for vitrivr-ng and the cineast.json file.

Best,

Ribin
vitrivir-config-files-ans-screenshot.zip

The ws_secure flag is used instead of http_secure flag to determine whether to use TLS for http in the config model.

This is probably a bug.

Steps to reproduce:

• retrieve object with multiple segments
• tag an element with a color (e.g. red)
• Bug 1: The coloring does not work
• Bug 2: When filtering for Display only objects with tagged segment: this seems to apply a filter on segment-level, not object-level

The UI expects image thumbnail to be at /thumbnail/path/image which is undocumented behavior (and not how Cineast by default extracts thumbnails for images)

Currently, we could already upgrade to Angular 13, we're on 11.