Transcribe -> Save Vocabulary -> Delete Vocabulary

From stack overflow:

https://stackoverflow.com/questions/46106037/vuex-computed-property-name-was-assigned-to-but-it-has-no-setter

I'm not sure what's going on with these errors, so just providing screenshots for now:

• CORS error when going from ML Vision tab to Speech Recognition tab:

• The verticle red line in the line chart is not showing up, and there's this error:

• Translation is showing this error:

Create a job to periodically cleanup buckets in test deploys.

We switched from using dropzone to Amplify for file uploads when we merged with MIE 2.0. The Amplify component is a single file upload. We rely on being able to upload multiple files for uploading the side-car subtitles with the input video for analysis.

Custom labels prob won't work on media-insights project as well due to this.

Includes:

• #64 Pre-commit workflow
• #18 PR workflow
• #34 Nightly workflow
• #33 Release workflow
• #67 e2e tests
• #66 Baseline UI Tests
• #65 Code scans

Code for Content Localization solution is delivered and released in stages where it undergoes different types of automated checks driven by GitHub Actions workflows. The stages are: Pre-commit → PR → Nightly → Release. The objective and configuration of each stage is summarized in the table below:

*When update deployment is failing on the Nightly test runs, it may mean there is a breaking change in the current development code base and the next release will need to be a major version. It is not necessarily considered a failure, but surfaces a significant finding. The team will need to decide the course of action for advising users under previous versions for using the new release.

Pre-commit stage

The objective of the Pre-commit stage of testing is to prevent sensitive data or unlicensed third party code from being committed to the public repository. Developers commit feature work to the remote (public) instance of the repository throughout a development sprint using feature branches. Work is committed to feature branches that are not yet integrated with the shared code base on the development and main branches. Even though these changes are not integrated to the shared code base, pushing a feature branch to the public repository is "publishing" the code in the sense that the work in progress is viewable and consumable by everyone on GitHub. We need to ensure that this steps doesn't inadvertently expose the GitHub community to intellectual property or security risks.

Pull-request (PR) stage

The objective of the Pull-request (PR) stage is to ensure the code from a feature branch that is being integrated into the shared code base under development doesn’t introduce instability into the shared code base. At a minimum, the code is "clean", builds and deploys successfully and unit tests are passing. Developers integrate their feature work to the shared code base by creating a PR on the development branch. The PR must pass the automated PR tests as well as being reviewed by a committer for the project. PR testing should support the code review process and help ensure the stability of the code base. It is important to maintain stability of the builds and deployments as problems at this stage can impact the productivity of the entire development team.

Nightly stage

Some testing may not be feasible for every PR due to time and resource constraints. A nightly test run is used to perform longer running tests on the shared code base to ensure ongoing stability.

Release stage

The objective of the Release stage is to test and publish new release of Content Localization on GitHub. Committers for the project periodically create releases with bug fixes and new features. The release workflow includes:

1. creating a release branch from development branch
2. building deployment packages that will be hosted for the release
3. updating one-click deploy button(s) in the README.md to point to the release deployment package
4. testing the deployment packages
5. pushing the release to the main branch
6. labelling the release.

Need to add TranscribeAudio operator to workflow

When I select a custom vocabulary in the Upload page, I can't clear it to make the workflow set to run without a vocabulary.

Propose functional and non-functional requirements for packaging this project as an AWS Solution.

Epic: as a content owner I want to improve the accuracy of the results from the automated localization workflow, so I can publish my content faster and at a lower cost

Story: As a content owner I want to provide customizations for Amazon Translate and Amazon Transcribe when I run my workflow so that editors and translators spend less time correcting the machine generated localization assets.

I created Custom Vocabularies, Custom Language Models, Custom Terminologies and Parallel Data sets outside the Content Localization application and I want to make use of them when I process my videos. I can select from existing Custom Vocabularies, Custom Language Models, Custom Terminologies and Parallel Data that are available in my account when I upload content for processing. The output of the workflow reflects the use of the parallel data.

Task: #51

Task: #17

Story: As a content owner I want a guided experience that uses the results of corrections made by transcriptionists and translators for creating domain specific customizations for Amazon Translate and Amazon Transcribe so I can easily leverage the results of Human in the Loop corrections to create more accurate results for a set of content.

Create a widget that can be included in any application to configure and visualize the results of AWS Transcribe.

See #61

Recommendation generated by Amazon CodeGuru Reviewer. Leave feedback on this recommendation by replying to the comment or by reacting to the comment using emoji.

Analysis of this code determined that this line of code contains a resource that might not have closed properly. A resource leak can slow down or crash your system. Programs are strongly recommended to use the built in with keyword to open a resource or to use a try-finally block to open and close resources explicitly. The contextlib module provides helpful utilities for using the with statement.

Learn more

Originally posted by @aburkleaux-amazon in #44 (comment)

https://github.com/aws-samples/aws-media-insights-content-localization/blob/9a489fd6187b16c46948656c40198ebfa76c0aab/src/views/UploadToAWSS3.vue#L472

//FIXME: filtering source languge from language tag list doesn't refresh
// tag picker in UI. So, if source language is English to start, English is
// removed from the translation target languages. When source language
// is changed to Spanish, Engish is added back to the tags on the Vue
// Translation component but Engish tag is still missing on the tag
// picker. For now, leave the source language in the list.
//.filter(x => x.text !== this.sourceLanguageCode)

Errors from sock.js in the JavaScript console on Analysis and downstream pages. Doesn't appear to be causing a visible issue in the UI.

AbstractXHRObject._start | @ | sockjs.js?9be2:1609
-- | -- | --
  | eval | @ | sockjs.js?9be2:1498
  | setTimeout (async) |   |  
  | AbstractXHRObject | @ | sockjs.js?9be2:1497
  | XHRCorsObject | @ | sockjs.js?9be2:2875
  | InfoAjax | @ | sockjs.js?9be2:356
  | InfoReceiver._getReceiver | @ | sockjs.js?9be2:539
  | InfoReceiver.doXhr | @ | sockjs.js?9be2:556
  | eval | @ | sockjs.js?9be2:525
  | setTimeout (async) |   |  
  | InfoReceiver | @ | sockjs.js?9be2:524
  | SockJS | @ | sockjs.js?9be2:734
  | SockJSClient | @ | SockJSClient.js?0a33:43
  | initSocket | @ | socket.js?e29c:20
  | eval | @ | client?96f8:176
  | eval | @ | index.js?http://10.1…080/sockjs-node:177
  | ./node_modules/webpack-dev-server/client/index.js?http://10.XXX.XXX.XXX:8080/sockjs-node | @ | app.js:26293
  | __webpack_require__ | @ | app.js:833
  | fn | @ | app.js:130
  | 1 | @ | app.js:27315
  | __webpack_require__ | @ | app.js:833
  | (anonymous) | @ | app.js:973
  | (anonymous) | @ | app.js:976

CRUD terminology
CRUD vocabulary
CRUD edits

A Polly audio transcript is generated from the application workflow by default.

Add the option to the upload menu so it can be enabled and disabled as needed

There is a problem in getWorkflowId()- asset_id in path is resolving to undefined for some reason.

The Waveform is an audio frequency graph that is shown on audio components in place of the line chart in video operator components. This graph gives an overview of times in the video when sound is present. Removing the Waverform component will just leave some blank space below the player on the Audio component pages

Currently fails creating the CompleteVideoWorkflow stack:

See #61

This set of tests will check that the high level functions of the deployed UI are not fundamentally broken - it's a sanity test after all! The test will log in to the UI and then use DOM-based validation on different Vue.JS views and components. Tests will prioritize the NLP visualization paths in the application.

Upload View

• Run a workflow with the default configuration + one language selected
• Verify upload completes
• Verify workflow started

Collection View

Setup: workflow has run with a specific test video
Check one or more fields in the table for the expected value

Analysis View (navigate from Collection view)

priority is NLP path

Setup: workflow has run with a specific test video

Transcript tab:

Transcript is length > 0
Transcript contains some high confidence word
Video is playing

Priority 2:

Buttons/modal validation

Subtitles tab:

Subtitles contain data
Subtitle contains high some confidence word
Video is playing

Priority 2:

Buttons/modal validation

Translation tab:

Correct number / language of translations are there (radio buttons)
Subtitles contain data
Subtitle contains high some confidence word
Video is playing

Priority 2

Buttons/modal validation

See #61

Code scans for sensitive data

Trufflehog - searches the codebase for IDs and passwords
XXX - searches the codebase for new third-party inclusions and initiates a license review
XXX - searches the codebase for other copyrighted materials that may require review

Code scans for security

cfn-nag - scans cloudformation for best practice specification of AWS resources
hawkeye (aka Viperlight) - security scan XXX more details here

Code scans for general quality and compliance

verify user-agent is included whenever an AWS runtime is used
linters based on language / tool
verify X-ray trace is enabled for all supported resources

Call GET /workflow/AssetId
Use the configuration from the original workflow for the asset
Disable upstream stages from the WebCaptions stage

Currently, it is using the default workflow settings updated with selected values which is prone to errors

Update READMEs

Refer to aws-media-insights for content guidance

Refer to aws-media-insights for content guidance

See GitHub actions in aws-media-insights-engine for reference.

MIE workflows can be run against existing AssetIds in order to run new operators or update the results of a previously run operator. We need a test to ensure there are no regressions in reprocessing assets. A few things to consider:

• reprocess with new operator
• reprocess with old operators
• operators enabled/disabled differently than the original workflow on the asset

For example, there is a method, getWorkflowStatus(), that is defined separately in UploadToAWSS3 and Translation.vue.

There are many more. Need to go through and pull out duplicated code into components or rename the methods if they are truly local to their component.

Move forms in Transcribe and Translate modals to a separate page that can be navigated to from the job or from the menu.

Using Subtitles as an example flow:

Subtitles -> download: download subtitles in specified format
Subtitles-> upload: upload subtitles in specified format
Subtitles-> save: save subtitles in specified format
Subtitles-> create vocabulary: launch Vocabularies with inputs (current edited subtitles, asset_id)

Vocabularies:

Create Vocabulary

• from scratch
• using edits (assetid, workflow_id) or (select from asset versions) - comparison base is always ML output

Update README and DEVELOPER docs to describe this application.

https://github.com/aws-samples/aws-media-insights-content-localization/blob/9a489fd6187b16c46948656c40198ebfa76c0aab/cloudformation/aws-vod-subtitles-deploy-mie.yaml#L156

The name of the CompleteVideoWorkflow template should be "/aws-vod-subtitles-video-workflow.template"

https://github.com/aws-samples/aws-media-insights-content-localization/blob/9a489fd6187b16c46948656c40198ebfa76c0aab/cloudformation/aws-vod-subtitles-deploy-mie.yaml#L169

This ImageWorkflow resource needs to be removed as it is not used in this applications.

The deployment instructions for the developer builds here should work.

This in the JavaScript console after swtiching to the Analysis view:

xx.js:1 Uncaught ReferenceError: url is not defined
at XXProvider.getVideoData (xx.js:1)
at XXProvider.addVideo (abstract-provider.js:1)
at HTMLVideoElement. (xx.js:1)
at Function.each (jquery-3.2.1.min.js:2)
at r.fn.init.each (jquery-3.2.1.min.js:2)
at XXProvider.search (xx.js:1)
at abstract-provider.js:1

MIE v2.0.2 is API breaking and operator breaking.

• Update /transcribe and /translate API calls to use /service route
• Transcribe operator is renamed to TranscribeVideo

https://aws.amazon.com/translate/details/#Named_Entity_Translation_Customization

See #61

Integration tests:

• run MIE workflow
• run workflow from transcribe with update
• run workflow from translate with update
• upload:
  • Translate enabled / disabled
  • Vision path enabled/disabled
  • english/non-english input
  • with/without VTT file input
  • one, many target languages
  • hindi (validate polly track)
  • with/without Transcribe Custom Vocabulary
  • with/without Translate Custom Terminology
  • with/without Translate ParallelData
  • with/without Transcribe CustomLanguage set

Verify:

• data in ES and dataplane for Transcript, Translations, Polly, Subtitles
• headless load of each view
• headless load of each component

The Media Insights Engine (MIE) project has been split into two separate projects:

• aws-media-insights-engine is the application development framework and APIs. When this framework is deployed it can be referenced by application stacks to provide a media workflow service.
• aws-media-insights is the content analysis search sample application built on MIE.

This application was built on the old, monolithic architecture and needs to be restructured to reference a separately deployed MIE workflow service stack.

New library operators and control plane changes will be delivered to the back-end project.