From 2020-09-18 call:

We need to decide exactly what information should be displayed in the summary pop-up window and how it should be displayed.

I like the conciseness of the term -> term display, but we should include each gene or gene product affected and also display the IDs of any terms or entities we display.

@ukemi - other thoughts?

It would be helpful for curators to know, when they're searching for models, what the model state is, e.g. state=production, state=development, etc.

Can this be displayed as metadata in the line for each model in the search results?

Note that this information is available on the current landing page.

Thx.

Ideally, any long list would work as an autocomplete, but if we can't autocomplete the list of curators, the next best thing would be to have an alphabetically sorted list so people can find themselves (or someone else) easily.

I think sorting by first name would be fine as it's a natural way for people to start looking.

From 2020-09-18 call:

When selecting models to review, once the first model is selected the resulting new window covers up some or all of the model metadata listed in the landing page table. (Screenshot from @vanaukenk 15" MacBook)

It seems we have at least two options for improving this:

1. Set a fixed width on the ART window so it doesn't overlap with the model metadata
2. Have curators select their models but only open up the ART window when they've finished selecting

Other possibilities?

@tmushayahama @lpalbou @ukemi

From 2018-10-02 call @tmushayahama

Suggestion from today's call is to highlight in the table the matched search string, e.g. PMID:28114319
or 'receptor recyling' (GO:0001881), so that it is easier for the curator to visually locate the matches to the search.

From 2020-09-18 call:

To allow users to easily clear the values in the search and/or replace box, add an 'x' like we have for the filtering:

I thought this feature existed, but I don't see it. After doing a search/filter, allow the User to select all of the models that have been returned instead of having to go through and select each one individually by checking the boxes at the left.

Reported by Karen Christie. The SheX errors are showing on the search tab but not review tab. It probably should behave the same and show it on both tabs.

e.g. this model doesn't show error on the review tab but did show error on the search tab:

@vanaukenk @ukemi to prioritize. @tmushayahama any issue to fix that ?

@tmushayahama
Can the groups in the 'Filter by Group' list be sorted alphabetically?
This makes it much easier to find the group(s) you're looking for.

Thanks!

In addition to highlighting the matched annotation (see #1 ) it would be useful to be able to scroll through the matching annotations using forward/reverse buttons at the top of the results table.

From 2020-09-18 call:

For example, the natural killer and NK cell entries in the screenshot above.

When curators perform a search, the default display in the table should start with each 'enabled by' GP.

Curators are used to looking at annotations from a gene-centric point-of-view, so will want to be able to see each matching gp when they expand the model.

From 2020-09-18 call:

When selecting models to review, once the first model is selected the resulting new window covers up some or all of the model metadata listed in the landing page table. (Screenshot from @vanaukenk 15" MacBook)

It seems we have at least two options for improving this:

1. Set a fixed width on the ART window so it doesn't overlap with the model metadata
2. Have curators select their models but only open up the ART window when they've finished selecting

Other possibilities?

@tmushayahama @lpalbou @ukemi

The bulk search & replace mode of ART currently requires / restrict the search and the replace to terms / entities that can be resolved by GOLr. In other terms, one can NOT search or bulk replace a value in the With column as it's free text.

In the following example, if one was to search for "PMID:1234", ART would only detect the occurence in the "Reference" column, not in the "With" column. Consequently, a bulk replace of "PMID:1234" by "PMID:X" would only change the value in "Reference" but not in "With":

The same applies for any value in With field, including GPs. And this can be counter intuitive and create mistakes/inconsistencies during the bulk editing models.

To discuss: what should be the proper handling of the "With" column with respect to search & bulk replace
@ukemi @vanaukenk

Right now, I only see the beginning of an (http:identifier...) URL but we will want to show the entity accession/identifier so curators can be sure of what they're viewing.

Creating a ticket for this even though the actual initial work for this is being done elsewhere.

See:

geneontology/minerva#216

During hands on call, Karen was looking at PMID-27340223-krc . On her spreadsheet, there was two models with that same title (the second one having a bit more info). Double check if noctua dev had both models or only one.

This ticket is to reference and keep track of the different developments to create ART, the bulk edit UI of GO-CAMs.

The ART mockup can be find here: https://drive.google.com/file/d/15gIJ85SAHzd6uCO-g7dmpHP-t1yJVt3f/view?usp=sharing

UI part

• build the UI based on the mockup above : follow the sequence of actions and the way the layout are updated after each action
• encourage directional workflow: 1 - activate review mode ; 2 - perform a search to retrieve models of interest ; 3a - selecting/unselecting a model will add it/remove it from the ART panel on the right. any new model selected should be on top of the screen to provide curators of an immediate view of the model and help them decide if it should be part of the models to bulk edit ; 3b - whenever the ART panel is shown, hide the search panel on the left to give a better sense of the unidirectional workflow and help the curators concentrate on the next steps ; 4 - once the models for bulk edit are selected, allow the curators to edit them through both the replace/replaceall functionality and the "edit pen" on each cell of the table. All of those changes are simulated (change the model in memory but not in the store) and should be highlighted to provide a global view of the pending changes that will be either canceled (restore the model from the store and cancel any pending changes on the in memory model) or accepted after showing a summary of all the changes that will take place (stats table)
• highlighting changes in the table : any pending change to models should be highlighted (eg yellow border / background) for quick browsing. A mouse over on any change should show a tooltip to indicate the original value. This original value should be provided by minerva and not kept in the client memory (eg issues if ART closed / reopened)
• summary table of changes : this table should be created by counting the differences based on the "changes functionality" (see in Minerva part). This is an essential step that will provide curators with the ability to get an idea of how many changes will be propagated with their bulk edits
• always show all the models selected by curators : doing a find/replace should not hide the models which don't contain the term/gp/pmid but should just fold them. Models containing the item searched should be unfolded by default to improve visibility
• simple button to fold / unfold all selected models : convenience feature to help curator see only the information they need
• translate shex responses to more user friendly messages : the shex json response usually refers to individuals violating one or more rules. A first level of integration would be to highlight in red any field in the table representation that violates the shex. A second level of integration would be to provide some human readable hints about what was violated. This would probably require 2 ways developments on the UI and minerva side.

Minerva part

• reset functionality: flush the version of a model contained in blazegraph to the permanent in memory version of the model. This is used to cancel any changes done with ART (see Notes below)
• changes functionality: provide a detailed diff of how the in memory version of a model differs from its version in blazegraph. The method would have an optional parameter to detail the diff for a single individual (eg for a goterm node). Another method would call that method for a set of models (while the client could loop through models, this would create separate multiple queries with possible network issues)
• unchanged / up-to-date functionality : simple function returning booleans to indicate if the in memory and blazegraph versions of a set of models are the same (used in UI to warn or prevent users to edit those models with pending modifications)
• selectively access the in memory or the blazegraph version of a model: the get_model() accesses the in memory version of a model, a tag should be added to allow to access the model stored either in memory or in blazegraph (see UI part to highlight changes)
• update queries stacked over multiple models: currently, the bbop library supports the stacking of multiple queries referring to a single model. However ART will be asking to update multiple models, hence the need to support the stacking of queries referring to multiple models
• work with UI team to improve shex json response : in previous calls, it was unclear on how some shex response could be interpreted; this will be an ongoing ticket as this is a new functionality

Unit tests (not specific to this project, but to be encouraged)

• set of permanent models on minerva/dev
• set of queries to modify those models on minerva/dev
• test that all modifications were actually performed
• test the cancel / reset for a single model, or for a list of models

Optional developments (probably for future road maps)

• restrict curators to only edit the models from their group but not all models : we will probably try to get this done before the end of the year. The simplest solution would be to check on the client side if the curator belongs to the same group as the curator who created the model. For better security and avoid errors, a better or additional solution would be to do the check on server side (eg accept edit of a model only if coming from a curator belonging to the same group or being a "super curator" - a tag could be added in the users.yaml to define such "super curator")
• locking mechanism: prevent multiple users to alter the same model at the same time. The "undo" mechanism could then be used more safely and would prevent possible errors arising from the simultaneous edit of a same model. As a reminder all edits are saved in a permanent in memory version of the model, including those coming from two users or two different UIs
• autocomplete: this is a larger task with implications in multiple UIs and with work required also on the back end side / GOLr. However, any development from now on should encourage the creation and use of reusable web components. The behavior of the web component could be the same as the current autocomplete used in NoctuaForm/ART, but the separation as a different library will help this part of the code to evolve and consistently solve issues over time

Notes

• This will be an iterative development, so items can be added / removed over time to these lists
• All changes started with ART must be simulated changes to enable curators to either validate all of their changes or to cancel all of their changes. This ability to simulated changes will be supported by the permanent in memory version of a model vs the version of the model contained in the triplestore
• Currently, any modification done with the bbop library to a gocam with "save" is written to the permanent in memory version of a model. When the "store" command is called, it flushes that memory version of the model to the store
• Due to the lack of locking mechanism when editing a model, the "undo" method can not really be used to cancel and reset the changes done in ART. Hence the task to create a "reset" at the minerva level, to restore the in memory version of the model to its version contained in blazegraph
• As a note, it would be good if any call to "save" could return a unique ID for the transaction / stacked queries, so this specific action could be undo. I don't think this is feasible at the time as it would require a much more advanced "undo" mechanism able to revert changes in a non sequential orders with potential conflits (like in GH)
• To prevent a model that was already changed in memory or to at least warn a user, the "changes" method should be implemented on minerva side to reflect how the in memory model and the store model differs
• A number of models will not pass the shex, either when opening them in ART or when a curator will start its bulk edit.

When searching on GO terms, users will need to be able to clearly see the term name so they know they're selecting the right GO term.

IDs should also be shown, for confirmation, but the term name should be clearly visible.

This is what I currently see in testing on noctua-dev:

Also, as an aside, it would be desirable to have the autocomplete/search results look the same across all the workbenches.

Things I like:

• compact display (the less white space the better)
• term name and id with term name first
• search string highlighted in bold
• link out to AmiGO for more term info
• some indication of GO aspect

We could split this out into a separate ticket, if that's preferable.

This should be a quick, minor fix.

The text in the GO term search box, should say GO term instead of Go term.

It might be worth checking across other workbenches code to make sure this is also uniform.

I tested searching for PMID:12186849.

The search returns four correct models, but when I open the last model in the list 'C. elegans PDI-2 pathway', I don't see that PMID listed as evidence for any of the annotations.

The other three models do show that PMID listed in the table.

From the Swiss-Prot group:

For annotations that are not pathway-like GO-CAMs (ie, not connected), there should be a simpler way to review annotations, more P2GO-like, ie getting the list of annotations either by searching for a PMID or for a gene id.

This would be required for SP to switch from P2GO to Noctua as the main curation tool.

Thanks, Pascale

Related to our discussion on Monday:

• since the minerva API doesn't allow the search of a title with multiple keywords, the field in ART should only accept a single value (currently accept multiple values as badges, but only the first value is used for the search)

• always add "gomodel:" to a model id if the user doesn't input it. The rationale is gomodel:xxx is a CURIE, which resolves to model.geneontology.org/xxx . So gomodel: should always be present in the model id. If not, that should be fixed in the repository

• last time, I try to search for a model and there was 0 result. This was correctly shown in the "Result" box, but ART was still showing the default list of models. When there is 0 result, ART should display 0 models

@vanaukenk @ukemi

From 2020-09-18 call:

When adding models to ART, initially display the title only, as the list of models added could get long and if model annotations are open by default, it will be difficult for curators to keep track of what they're adding.

We should, however, provide an 'expand all' functionality to act on all models, so curators could then open models en masse as well as on a per-model basis.

For the 'expand all' functionality, when acting on all models, we may want to have two levels of expand all:

1. Expand to show all of the 'enabled by gene blah'
2. Expand to show all annotations

I'm not sure that this is the correct place for this ticket, but as I test ART, I am seeing a worrying number of ShEX violations in models. If we are to use ShEX in a meaningful way, we need to address these sooner rather than later. Otherwise they will become like the boy who cried wolf. Many of them seem to be arising because entities cannot be found. we need to have systematic ways of identifying errors and correcting them.

Based on 2020-09-30 and 2020-10-02 workbenches calls:

@tmushayahama Here's what I have on my list. I've tried to check off ones you've already addressed.

Feel free to add more if I've missed anything. Thanks!

• Remove toggle button to the right of Annotation Review Tool

• Replace Annotation Review Tool text with just Annotation Review in menu option

• Indicate that Annotation Review mode is activated by changing background color of menu option

• Move number of models selected to first box in the workflow

• Rename workflow boxes (from L to R): Search, Review, Checkout with appropriate subtext, 'Add/remove models'; 'Edit models'; 'Review and submit changes' ( @tmushayahama - I didn't take a screenshot of your development site today, so didn't quite remember what the current subtext is.)

• Debug horizontal scrollbar not always appearing at the bottom of the table

• Debug table behavior so that clicking the number sequence or up/down arrows moves the results table in the correct order

• Move Cancel - Reset - Review Changes buttons from table to bottom of look-up page

• Change actions buttons at the bottom of the ART lookup page to be: Clear List - Undo Changes - Review Changes

• Remove 'CAMs' row from change summary table

• Upon submission of changes, clear the basket and go back to the default landing page setting with the Search workbench open

From Berkeley meeting 2019-02-01:

Rather than cut off text in each field, wrap text to show the whole entry, e.g. GO term label or With/From column.

From demo at Berkeley meeting - 2019-02-01

This is a not a major bug and we can discuss on the 2021-04-30 workbenches call.

If I finish an ART session, clear the models list, leave ART and then start another search and review from the NLP, when I go back to ART, the previous session's values in the 'Find What' and 'Replace With' fields are still shown.

I think it would be good to clear those two fields when a user exits ART.

Currently, when looking for Gene Product, one can only find gene product. However models also contains small molecules such as CHEBI (here calcium oxalate):

Proposal:

• rename "Gene Product" in the UI by "Molecule" or "Entity"
• extend the closure of the autocomplete to include small molecule

@ukemi @vanaukenk ?

Currently, it's possible to find and to replace compounds / chemical entity, which also authorize a user to potentially change a GP to a compound. We don't want that.

@tmushayahama to fix by removing chemical entity results from both the find & replace boxes.

Note: the autocomplete behavior will have to be reviewed later on, when working on a sharable component across UIs

@tmushayahama
Would it be possible to display the number of matching models after search and/or filtering?

The search for MF works fine, but I'm unable to search for a BP or CC term.

Hi,

Looks like we might need a new pop-up window with a different message if a user saves changes on an individual model and then clears the list.

Right now, the resulting pop-up window says that there are unsaved changes, even if the change menu has disappeared from the models list and the changed term is no longer highlighted on the table.

In this case, we probably just want to show the pop-up that says:

@tmushayahama - let me know if you want to look at this workflow together. Thx.

The table view is showing redundant information for models, as well as relations and inferences that we don't want to see.

For this very basic model:

This is the table view in the ART:

The Noctua form is showing the correct annotations, so we need to match that display in the ART:

Hi,

MGI had an error report for use of invalid properties (extensions):

Use of CL:0008029 in GO:0043524

gomodel:MGI_MGI_1935201
GO:0043524 Pcdhgc3 CL:0008029

gomodel:MGI_MGI_1935203
GO:0043524 Pcdhgc4 CL:0008029

gomodel:MGI_MGI_1935205
GO:0043524 Pcdhgc5 CL:0008029

I tried to use ART to change all three instances of CL:0008029, inhibitory neuron, at one time.

Step 1 - Search - worked to find all 3 models that contain the reference PMID:32633719:

Step 2 - Review - worked great to let me select the CL term to replace and the replacement CL term:

and gave me confirmation that 3 occurrences across 3 models were to be replaced:

Step 3 - Checkout

• Problem 1 - The summary of changes indicates that there are "0 changes across 0 models". Are the CL terms supposed to show up in the Terms section?
  

• Problem 2 - Clicking the Submit button does nothing. I don't know if this is related to the Checkout section thinking that there aren't any changes to save because it ignored CL terms, or if it might be related to the issue that @ukemi submitted a few weeks ago: Bug in saving models? #46

This morning I went to the landing page and:

1. Searched on PMID:16199276- It returned one model
2. Switched to ART and added that model to my basket
3. Clicked on my basket and used the search and replace tool to find PMID:16199276.- The tool found two examples.
4. Clicked on 'replace all'- the tool looked like it worked
5. Clicked on 'review changes' - the summary said no changes were made
6. Clicked on submit anyways
7. Navigated to the graph view and the model had an indication that it had not been saved.
8. Saved the model and checked the annotation preview. I did not see the incorrect ref in any of my annotations.

First, overall looks great and glad to see that finally on dev.

Things to fix / discuss

1. When there are no activities, like in this model, show a message stating so, otherwise it could feel like there was some errors while loading the model

2. the height of that grey bar "ACTIVITIES" is really too big, I would decrease to 30px; still feel a bit clunky but not critical

3. I am surprised as I thought the "ACTIVITIES" bar was to list activities... and yet I find a "BP Annotation" and I imagine CC Annotation can appear there too. Seems like either that grey bar is useless OR you want to have multiple grey bars, one for activity, one for bp annotation and one for cc annotation (to show only when you have such annotations). And if it's the logic, then you don't need to show these badges "Activity Unit", "BP Annotation", "CC Annotation" as they would already be grouped under a same banner.
   

4. I was in review, clear my basket and.. Then decided on the search icon (top left) and it showed me that:
   

I believe clicking on the search icon should switch you back to the search page, otherwise it's confusing.

The culprit is still that bar :

I believe we would have to redesign that later, as both this bar and the left bar control what you should see at the center.

5. in the same aspect, I don't think it's intuitive/clear that after selecting models, you should click on the review button on the top:
   

Not for this V1, but I would suggest that whenever models are selected, a floating colorful button on the bottom right appears and state something like: "X models selected - Click here to review them". The fact the button appears upon model selection would help people know where to click for the next step right away

6. I mentioned above I cleared my basket ("Clear List") in which I modified lipid transport into ion transport. But even though the modal state that any changes would not be saved... When I reselect those models, I still see my modifications:
   

So something is wrong here, maybe you don't call reset on the models ?

And indeed, even after a refresh of the page, you do see there are pending changes:

Note that clicking on "Undo changes" in the list of models to the left does reset the model changes.

7. we don't want to have both "List" (bottom left) and "Basket" (title of modal). I prefer "List" as we are not an ecommerce
   

The Clear button in the search box does not seem to be working to clear any value.

It just seems to restore the generic label, e.g. Go Term, when the field is empty.

When I select a line in the table to edit, the resulting editing box doesn't display anything in the annotated entity field.

Some users want to use the browser search (Ctrl+F / Cmd+F).

On chrome I didn't have any issue, but it seems on firefox, Karen was not able to search for her PMID which was visible on the page.

My guess is that the focus was not on the correct section of the page, maybe on the left panel ? It's still a very strange behavior. @tmushayahama ideas ?

This search/highlight was only working after clicking on the table part of the page.

From 2020-09-18 call:

When a user uses the up and down arrows to find the matches to the search criteria, the match should always appear in the ART window.

In the screenshot below, I've clicked on the arrow, but they match isn't shown: