scottlogic / finos-vuu Goto Github PK

Background

The screenshot feature implemented in #23 works in Chrome and Edge, but not in Firefox.

Reproduction Steps

1. Run the showcase from vuu-ui (npm run showcase)
2. Open the "Shell with New Theme" example in Firefox
3. Open the context menu on the "My Instruments" tab header
4. Click "Save Layout"

Expected Behaviour

The Save Layout dialog shows, displaying a screenshot of the current layout.

Actual Behaviour

• The Save Layout dialog shows, displaying a placeholder for the screenshot with the text "No screenshot available"
• An error is logged to the browser console

Notes

Cypress end-to-end tests have been added to the CI pipeline since this bug was raised. We will want to add a step to test-ui.yml to run them in Firefox as part of this ticket.

Screenshots

Background

We have created a Component for the Save Layout Dialog. We want it to display a screenshot of the layout taken when the Dialog is opened

Work required

• take screenshot on "save Layout" click
• display screenshot in dialog

Note

• Security is not a concern at the moment

Background

We want the e2e automated test to be run across different browsers as part of our checks whenever a PR is raised

Work Required

• add yml file (if not there yet)
• add jobs to the yml for running tests cross browser: chrome, firefox and edge
• add action in GH with trigger being PR raised

Useful Links

https://docs.cypress.io/guides/guides/cross-browser-testing

Background

as part of #54 and #27 we have added error catching for CRUD operation around Layout Management. We now want to add something that informs the user that an error has occurred (currently we are just using console.error())

Work Required

• consider whether we want to create Toast Component (use Salt's as base) or just use Salts
• add new example to showcase. Example should allow to produce an error/warning/success/info Toast
• we could use a pattern like we are doing for the LayoutManagementProvider/Context

Design:

• Toast design
• Location on page

We need to store the layout metadata which is saved by Save Layout Component #17 so that it can be consumed by LayoutList #18.

Work Required

• store layout metadata saved in Save Layout Dialog
• consume layouts metadata in Layout List
• setup lightweight infrastructure for managing Layout metadata state

Background

#75 explored alternative approaches to the screenshot string as it can be very long and posed threats to performance and robustness of client/server interaction. The short term outcome of that prioritised function over form, adding a @Lob annotation to the server entity to accommodate big screenshot strings. There were a few other possible solutions that could be worth exploring so we're not having to handle a large string from client through to server, which would require further investigation.

Possible Solutions

(Courtesy of @pling-scottlogic)

1. Convert the entity to use a byte array:

@Lob
private byte[] screenshot;

Would need to do some manual conversion (I wasn't able to get this working).

Base64.getDecoder().decode(rawScreenshot.replaceFirst("data:image/png;base64,",""))

• I wasn't able to convert it back properly
• Doesn't seem worth it, since everything's a byte array really
• Doesn't seem like we'll gain much from compression either

2. Save as "blob" on client-side, upload as multipart file and store as byte array - looks like the best option

• Would need to restructure metadata on client side, using formData
• Conversion required for local implementation https://thecompetentdev.com/weeklyjstips/tips/57_store_binary_in_localstorage/

3. Client-side compression

• Tried decreasing the "width" parameter when first taking screenshot
• Quality converges on potato pretty quickly with negligible space savings

Background

We want to implement SSL on the layout server so that we can use a secure HTTPS connection when calling from Vuu UI. This will also satisfy the static code analysis and therefore remove the need to reference the calling code in the .semgrepignore file.

Work Required

• Generate a self-signed SSL certificate using the keytool CLI or find an existing one from the Vuu server (looks like we might already be using OpenSSL)
• Enable SSL in the application.properties file for the layout server
• Create a SecurityConfig class to redirect HTTP requests to HTTPS
• Configure the UI to make requests over HTTPS
• chckout SSL usage in Scala code (vuu\src\main\scala\org\finos\vuu\net\http\VuuHttp2Server.scala)
• remove "vuu-ui/packages/vuu-layout/src/layout-persistence/RemoteLayoutPersistenceManager.ts" from .semgrepignore

Testing

• Ensure that calls over HTTPS are picked up successfully
• Ensure that calls over HTTP are redirected to HTTP

Epic to cover the improvements to the quality of Typescript use in the FINOS-VUU repository

Typescript Uplift Project

Background

We currently have 2 different Context Providers around layouts: LayoutManagerProvider and LayoutProvider. LayoutManagerProvider includes state and functionality to manage the layouts and the metadata. LayoutProvider manages the on-screen layout (which hasnt been saved with metadata yet).

Work Required

• move logic for loading, saving layout to useLayoutManager
• delete use-layout-conifg

useLayoutManager should include:

• saveLocation based on env variable
• logic for loading and saving layouts and temp layout
• warning layout for errors

See PR for POC

• resolve typescript errors
• general tidy up of code, i.e. remove unnecessary comments and unused variables
• all variables are appropriately typed, i.e. no use of 'any' type

Some styling needs to be tweaked on the SaveLayoutPanel and LayoutList components to make them match the Figma designs.

1. Font style on inputs should be Nunito Sans
2. Ticked checkbox background colour should be darker
3. On focus, inputs (both text and combo box) should have purple border, instead of dotted blue outline
4. Grey box around dropdown options should be removed
5. All purple borders should be removed from LayoutList

Background

Layout definitions are currently being transferred to and from the server as raw string values in a single JSON field. Since the layout is defined in a JSON format, we want to embed this as a fully formed node within the request/response.

Work Required

• Convert the definition fields in all layout DTOs and entities from String to JsonNode (import from Jackson databind)
• Annotate the entity field with @Convert to make use of the existing JsonNodeConverter

Notes

• This pattern was implemented for the application layouts resource on the same server
• See PR discussion for context

Description

We created a new server for handling layouts in #25. We want to add tests for this.

Work Required

• Write unit tests for controller
• Write unit tests for service
• Write integration tests to cover controller, service and database interaction

Background

Currently the loadLayoutById functionality replaces everything in the layout view with the saved layout. We want the functionality to load the saved layout and add it to what is already in the layout view

Work required

Notes

Background

Currently we are allowing all menu options for non-active tabs (tabs that are not selected). This leads to unwanted behaviour when the user tries to save a non-active tab as we are taking a screenshot of the layout but we cant take a screenshot of the tab that is not being displayed.

Work required

• make "save layout" option disabled for non-active
• check keyboard navigation doesnt allow selecting the disabled option
• introduce concept of disabled options if not there yet
• include tooltip for explaining why the option is disabled (good to have)

• resolve typescript errors
• general tidy up of code, i.e. remove unnecessary comments and unused variables
• all variables are appropriately typed, i.e. no use of 'any' type

• resolve typescript errors
• general tidy up of code, i.e. remove unnecessary comments and unused variables
• all variables are appropriately typed, i.e. no use of 'any' type

• resolve typescript errors
• general tidy up of code, i.e. remove unnecessary comments and unused variables
• all variables are appropriately typed, i.e. no use of 'any' type

Background
We need to create a service which handles the persistence of layouts.

The implementation of persistence has two levels:

1. An interface which directs to a persistence management handler
2. The implementation of the persistence management handler (client / server)

Level 1 is an interface which can direct to multiple persistence handlers, i.e. local or remote. Creating the interface is the priority, as this can be extended with as many persistence handlers as required.

We can implement basic local storage persistence using this interface to showcase the layout management feature.

Server-side persistence is low priority for the Layout epic, as we can easily hook up server level persistence management into the interface at a later stage.

Work Required

• Implement a RESTful interface to direct to a local/remote persistence handler
  • Can make use of use-layout-config.ts
  • Create/Update/Delete
• Implement the local layout persistence handler
  • Can make use of local-config.ts

Acceptance Criteria

• As a Vuu UI user, when I save a layout, then a file defining the layout is saved locally
• As a Vuu UI user, when I save a layout, then a screenshot of the layout is saved locally

Dev Notes

• The current hooks in the layout-config directory and any associated layout-management code is a good starting point but it's quite primitive. It assumes you are saving one thing and doesn't have configuration for options such as layout names, groups, etc.

Questions

• Where should the file be saved? In-browser?
  • Answer: Browser local storage
• (How often) do we clear the saved files?
  • UI option?
  • Answer: Can be dev-managed for local storage as it's not a permanent solution
• How do we couple a screenshot with the layout file?
• How do we handle layout metadata (layout name, group, username, save date, ID)

Background

The layout server was implemented in #57. It currently handles persistence of a user created layout. We need to expand the server to handle application layout(s), as pointed out in the PR for #57 (comment).

Work Required

• Add a controller for application layouts
  • Include GET, POST, PUT and DELETE endpoints
• Add a service
• Add a repository for application layouts (with appropriate entities)

Notes

• Application layouts consist of a (JSON string) definition and a user
• User can be hardcoded for the time being (will eventually be provided by request headers)

• resolve typescript errors
• general tidy up of code, i.e. remove unnecessary comments and unused variables
• all variables are appropriately typed, i.e. no use of 'any' type

Repro steps:

• open App/ShellWithNewTheme in showcase
• select "My layouts" in Navbar
• zoom in/out

The Figma design includes an image of the layout, which can be produced with one of the following packages:

• html2canvas
• html-to-image
• dom-to-image

We need to determine which of these packages is most appropriate to use for saving and browsing layouts.

Acceptance Criteria

• Pick a package

• Proof of concept

• A: I tested with file saving and the biggest file was 25kb, so file size doesn't look to be a concern.

Background

We have a cypress test for the screenshot functionality. the test is currently under e2e directory we want to move it under components/ or integration/

Outcomes

• understand Component Test in Cypress
• difference between Component test in cypress and React unit test
• figure out what to do with screenshot tests currently in e2e/ (move to component/ ?)

Notes

The e2e tests should cover the most obvious, important user flows
Integration tests should cover less important permutations, and interactions between different layers of our application
For example, our e2e might be As a user, I can save a layout. Integration tests related to that might be When I save a layout, I can see a screenshot of the layout and When I save a layout, I see a dialog to confirm details of the layout to be saved

Useful Links

https://docs.cypress.io/guides/core-concepts/testing-types
https://docs.cypress.io/guides/component-testing/overview

• resolve typescript errors
• general tidy up of code, i.e. remove unnecessary comments and unused variables
• all variables are appropriately typed, i.e. no use of 'any' type

Description

In LocalLayoutPersistenceManager, whenever we attempt to read a key from local storage which is not present, we immediately default to returning an empty array. We want to change this behaviour so that an exception is thrown, then caught and handled outside of the persistence manager.

Work Required

• In LocalLayoutPersistenceManager, throw an appropriate exception whenever getLocalEntity returns undefined
• Catch these exceptions outside of LocalLayoutPersistenceManager and handle appropriately (e.g. defaulting to an empty array)

Background

As part of the Layout Management Issue, we want to build a modal component to allow users to save their current layout.

Work Required

Edit the placeholder component SaveLayoutPanel.tsx to match the Figma design (Save Layout 2 > Save Modal w/Overlay > Frame 108)

Testing

Write unit tests (using react-testing-library) for the new component.

Notes

When a layout changes, a layout change event is triggered. This is handled by the Shell/layout management system , which persists the changed JSON representation of the UI.

Until now, the entire JSON structure of the UI has been persisted as one entity. We are moving to an approach that saves layouts individually, where the application level structure references those layouts rather than including them directly.

The layout change handling code needs to know whether any particular change is caused by a change at the application level (switching tabs, opening or closing a layout) or at the layout level (adding or moving a component within the layout, saving props on a component e.g sorting a table, applying a filter)

Modify the layout change callback to provide this information

Work Required

wire it up to be shown when

• layout metadata couldn't be retrieved
• layout couldn't be retrieved
• layout saved successfully
• couldn't save layout

Blocked by #59 and #27

Outcome

• Server running locally
• 1 endpoint
• integration with client
• testable with postman
• discussion on whther using Node.js or Spring for #25

Time 1 or 2 days

Background

We want to add e2e tests to cover the main functionalities

Work Required

• e2e test to cover the save layout functionality.
• happy scenario:
  • customise layout (i.e. rename title)
  • open save layout modal
  • enter details and save
  • verify toast notification is shown
  • customise layout again(so it doesn't match the one we just saved)
  • can find layout in list
  • launch layout
  • layout matches saved layout json

Notes

• A good place to start is the existing screenshot test
• test might need to be updated after #26
• other scenarios to be added as follow-up

Blocked by #49

• resolve typescript errors
• general tidy up of code, i.e. remove unnecessary comments and unused variables
• all variables are appropriately typed, i.e. no use of 'any' type

Background

We want to document our ways of working for anyone in Scott Logic who wants to be involved in the project in the future

Outcome

wiki on GH Repo to cover:

• branching/merging strategy
• testing strategy
• team ceremonies
• key point of contact
• project background (in SL)
• useful links
• setting up environments/ get started steps

Notes

• if there is a concern about sharing this information in the wiki then put this document in google docs and link in the wiki

We're also swallowing these tickets:

• "Developer: I want to be able to easily use the open source code and understand how to implement it so I can implement it in the most effective way"
• "Developer: I want to be able to understand the use cases of the code so that I know if it is fit for purpose and so that I can explain the benefits to the Product Owner" here. Is that appropriate? The description of the ticket was: "UBS users - presenting data in a table . Steve to consider for prioritisation. UX designer coming on filtering - need to know the end goal".
• "Developer: Test Strategy to be agreed" - already being done as part of the in-progress docs

Repro Steps

• open component containing Combobox (i.e. Save Layout Dialog)
• open Combobox Dropdown
• resize window

Expected

the dropdown moves along with the input fields

Actual

input field moves but the dropdown doesnt

Questions

• is this still a problem ❓
• do we need DropdownBase's content to be rendered inside a Portal? (removing the Portal fixes the issue)

Background

In #25, we set up a simple REST service to handle remote persistence of layouts. We want to set up a simple database to use in conjunction with this.

Work Required

• Set up a simple in-memory database (e.g. h2)
• Add a table for storing layouts, with the following columns:
  • ID (PK)
  • Layout (raw JSON?)
• Add a table for storing layouts screenshots, with the following columns:
  • ID (PK)
  • Screenshot
• Add a table for storing layout metadata, with the following columns:
  • ID (PK)
  • Layout group
  • Layout name
  • Version number
  • Create user
  • Create date
  • Last update date

Background

as part of the layout manager feature we have been taking screenshots of the layouts and we are storing them as strings which are very long. We want to investigate what the best way of handling/storing PNGs is.
example length for the PNG string is 13186 characters

Time-boxed: 1 day

Outcome

• shortlist of options with pros and cons for dealing with screenshots
• meeting for discussing

Questions

• format? (i.e. String, Binary)
• compression?
• changes to DTO?
• same format in client side/ local storage?

Background

As part of the Layout Management Milestone we've begun writing some e2e tests with Cypress for the first time in the project.

The tests written in #23 are a good first step, but there's improvements to be done.

As this is the first time we're setting up e2e tests in this project, we should think about best practices to make it easier to implement tests moving forward.

Tasks

Use info in the dev notes to refine a list of tasks.

Dev Notes

1. Specific improvements for screenshot.cy.js
   • Improve the selectors
     • Current selectors are based on classnames. These should be a last resort if the framework doesn't provide other selectors. A combination of Cypress & Testing Library gives support for better selectors (like aria attributes). We should use these.
   • The test was written with one partial user flow in mind (I save the layout, I see a screenshot in the dialog). We should actually consider the full user flow of the save layout feature, how this integrates with the screenshot, and write tests which reflect this flow. We won't be able to get the full user flow tested until the feature is complete, but we can make note of what we expect this full user flow to be so we know whether our e2e tests match up to this at the end. (#50)
2. Improvements for screenshot testing
   • We should also write some Component/Integration tests to cover different permutations not covered in e2e tests
     • The e2e tests should cover the most obvious, important user flows
     • Integration tests should cover less important permutations, and interactions between different layers of our application
     • For example, our e2e might be As a user, I can save a layout. Integration tests related to that might be When I save a layout, I can see a screenshot of the layout and When I save a layout, I see a dialog to confirm details of the layout to be saved (#51)
3. Design Pattern
   • We should consider implementing some kind of testing design pattern, like POM (Page Object Model), to encapsulate different parts of the application
   • E.g. rather than screenshot.cy.js defining all of the locators relevant to it, we have locators defined for each section/page of the application, which can then be used in any tests which rely on that page/section

Mike gave us some useful links to help us in this process:

• https://www.testim.io/blog/end-to-end-testing-best-practices/
• https://www.testim.io/blog/end-to-end-testing-guide/
• https://www.lambdatest.com/learning-hub/end-to-end-testing

Some styling needs to be tweaked on the SaveLayoutPanel component to make it match the Figma design.

1. Dialog title ("Save Layout") should be horizontally aligned with form field labels.
2. Border radius for top corners
3. Font style for placeholder text

Background

As part of the layout management directive, we want to restructure the JSON format that defines layouts.

Work Required

Background

Current selectors are based on classnames. These should be a last resort if the framework doesn't provide other selectors. A combination of Cypress & Testing Library gives support for better selectors (like aria attributes). We should use these.

In Scope:

• SaveLayoutDialog
• LayoutList
• Layout menu
  • save layout option
  • rename
  • title input
• LeftNav options

Work Required

• Add selectors to elements in scope which are used for the e2e test

Notes

We should consider implementing some kind of testing design pattern, like POM (Page Object Model), to encapsulate different parts of the application
E.g. rather than screenshot.cy.js defining all of the locators relevant to it, we have locators defined for each section/page of the application, which can then be used in any tests which rely on that page/section

Useful Links

https://www.testim.io/blog/end-to-end-testing-best-practices/
https://www.testim.io/blog/end-to-end-testing-guide/
https://www.lambdatest.com/learning-hub/end-to-end-testing

• resolve typescript errors
• general tidy up of code, i.e. remove unnecessary comments and unused variables
• all variables are appropriately typed, i.e. no use of 'any' type

• resolve typescript errors
• general tidy up of code, i.e. remove unnecessary comments and unused variables
• all variables are appropriately typed, i.e. no use of 'any' type

Background

In #20, we create a front-end interface to handle all layout persistence. We need to make an implementation of this interface to communicate with the REST service created in #25 to handle remote layout storage.

Base on #20 branch

Work Required

• add environment variable to pick local or remote
• Make a "remote storage" implementation of the layout persistence interface
• Have the load/save/update/delete methods call through to the appropriate GET/POST/PUT/DELETE endpoints
• Handle potential HTTP 500 errors (and any other non-2xx) appropriately

Testing

We will need to mock the API response.

Background

As part of the Layout Management Issue, we want to build a component to allow users to browse layouts that have been saved.

Work Required

Create new Component to match Figma design

Acceptance Criteria

• As a Vuu UI user, when I browse layouts in the 'My Layouts' sidebar, I should see unique screenshots corresponding to the saved layout

Testing

Write unit tests (using react-testing-library) for the new component.

Background

When attempting to operate on a layout that does not exist in local storage, the current behaviour of LocalLayoutPersistenceManager is to continue regardless (see Current Behaviour below). We want to change this so that an exception is thrown whenever we try to access a layout with an non-existant ID.

Current Behaviour

The current behaviour is as follows, when provided with an ID that does not correspond to any layout in local storage:

• updateLayout will add a new layout (and metadata) with the provided ID
• deleteLayout will leave local storage unaltered
• loadLayout will log a warning and return an empty object

Work Required

• Adapt the updateLayout, deleteLayout and loadLayout methods in LocalLayoutPersistenceManager, so that an exception is thrown if there is no layout or metadata in storage with the requested ID
• Consider adding a common method to avoid duplication, e.g. validateId (use a consistent approach for all three methods)
• Handle the new exceptions wherever the methods are being used*

* At time of writing, none of these methods are being used.

Notes

There is a warningLayout, which might be sensible to display when one of these exceptions is thrown.

Links

Initial discussion

Background

As part of the layout management directive, we want to create a backend service to handle remotely saved layouts.

Work Required

• Create a backend service to act as a layout management server with Java and Spring Boot
  • Define an API on the service with the following endpoints:
    • GET for loading saved layouts
    • POST for saving new layouts
    • PUT for updating existing layouts
    • DELETE for deleting layout
• Set up an in-memory H2 database for persisting layouts and metadata
• Create a service and repository layer to connect the API to the database
• • API contract using openAPI (Good to have. Timebox: 0.5 day)
• Add README: include steps to set up Java environment (see what is already covered by steps for setting up Scala)

Background

We have a remote layout manager that makes requests to the layout server for layouts and metadata. After adding a resource for application layouts, we want to call into these new endpoints from the client.

Work Required

From RemoteLayoutPersistenceManager:

• Make a GET request to the application resource when application layout is required
• Make a POST request if the server returns the default layout (check ID/user fields)
• Make a PUT request whenever a change is made to the application layout

Blocked by #27 and #70

Background

In #34 we set up a simple in-memory database for persisting layouts. We want to connect this to the REST service for layout persistence management.

Blocked by #34 and #25

Work Required

• Add a repository layer to the REST service
• Connect the repository layer to the in-memory database

Background

#57 introduced the remote server for layout management. We need to write documentation on how to run the server and publish this to the main Vuu docusaurus site. #27 also integrated remote layout management into the UI using the server, and provides a means of choosing either the local or remote implementation with envars. We should document how to change from local/remote.