alexandercerutti / sub37 Goto Github PK
View Code? Open in Web Editor NEWA set of libraries to parse, serve and show subtitles in video medias
Home Page: https://sub37.vercel.app/
A set of libraries to parse, serve and show subtitles in video medias
Home Page: https://sub37.vercel.app/
As per the first version of sub37, the whole architecture expects Adapters always to return a ParseResult
, containing data and perhaps some ParseError
.
The idea was to have ParseResult
and ParseError
exposed by the BaseAdapter
so that every adapter could use them without importing anything else and without making @sub37/server
export other material.
However, the current situation reveals itself to be somehow inconsistent:
BaseAdapter/index.ts
exports BaseAdapter
(default), ParseResult
, and ParseError
classes. We should technically move Parse*
to a different file;
BaseAdapter
exposes ParseResult
as an instance property, but not ParseError
;
Related to the points above, tests and WebVTT adapter use ParseResult
from there, while BaseAdapter
should have been the entry point to access them.
This happens because BaseAdapter.ParseResult
is actually a method that returns a new ParseResult
and, therefore it cannot be used as right-hand in an instanceof
comparison: we must use instanceof ParseResult
, but we should use instanceof BaseAdapter.ParseResult
;
BaseAdapter
exposes both ParseResult
and ParseError
types by using a namespace;
ParseError
class is exposed from BaseAdapter/index.ts
module, but it is not used as a class (this happens due to the point 3, so it is an useless class.
Fixing the whole situation is quite breaking, so should be fixed in the next major.
According to the image below, the Youtube-like rendering mode (which consists of shifting down the first line, when the mode is enabled), shifts down the content but maintains an empty line (which is utterly ugly and wrong).
This probably happens because translateY
is applied to the wrong element.
To enable the Youtube-like mode, sample's script.mjs
should be edited to add the following lines on the bottom of the file.
/**
* @type {CaptionsRenderer}
*/
const rendererElement = document.getElementsByTagName("captions-renderer")[0];
rendererElement.setRegionProperties({
shiftDownFirstLine: true,
});
With the integration of the TTML adapter, we are evaluating if we should support SMPTE ttp:timeBase
.
There's also an extension of TTML which supports some features.
The article on Speechpad (linked below) describes them nicely.
Should we support SMPTE-TT inside TTML Adapter or in a different adapter?
If choosing a different adapter, we put ourselves in front of a choice: should we duplicate the adapter code or make adapters extendable?
When choosing the extendability, we are talking about changing the architecture to let adapters be hookable
or something like that. Extenders should be able to intercept the parsing phase. Intercepting points must be defined.
captions-renderer
Of course, this change will go to impact also the captions-renderer
. This is due to the support of bitmap images by SMPTE-TT, along with rendering style.
When a document is well-composed (no unordered cues, correct timings, etc.) or perhaps streamed (during live content), the Interval Binary Tree beneath the Track
s, will be an all-right tree. This might lead, potentially, to higher retrieving time to reach the last element to show as the current time increases (as we have to navigate the whole tree on the right - e.g. when the content is in the middle of its duration).
The solution is to let it auto-balance, which is an AVL-Tree feature (it can be seen here) when new cues are available. This would let us cut in half the time to retrieve the time of retrieving.
Of course, the discussion is open (if anyone will ever read this ๐) and to implement an auto-balancing, we should decide first a limit after which the balancing is needed.
A little concern that comes to my mind is that auto-balancing could be decided to be performed in as small as possible chunks, as it might require some time (depending on the number of cues in the tree). Even if it is decided to be performed as a micro-queue activity, it should be performed as fast as possible to not block the retrieving of the current cues, which happens through a setTimeout
(hence "macro-queue" or whatever is it called).
With the current implementation, subtitles serving might not honor the media's playback rate and therefore go out of sync.
A solution should be evaluated so that every adapter can benefit from this change.
Interval Binary Tree should probably lookup like with a multiplier to get the right amount of cues on the same interval of time or whatever
TTML adapter is still in the working, but I can surely create the issues for the features that won't be shipped with the first version of it.
tts:fontSize
supports providing two <length>
. According to the conversion table on the standard page, TTML
adds optional second length (or percentage) for specifying separate horizontal and vertical scaling of glyph's EM square;
However, to achieve such vertical scaling (perhaps horizontal? The opposite dimention of font-size
), we need to change how the renderer works.
Specifically, along with font-size
, we need to emit a transform: scaleX()/scaleY()
, with the appropriate scaling factor, and apply it on an element that owns display: inline-block
(hence, three properties to emit).
This translates, for the renderer, that it should create a new depth-level, a new span or whatever, that will let it specify such styles without impacting the previous or past sections.
When using a non-targeting (valid but no matching element) CSS selector, all the elements get that style mistakenly applied on.
In the repro case below, the selector used in the STYLE is wrong as it should be v[voice="Fred"]
.
When using this selector, also cues containing <v Bill>
become blue. None should become blue.
WEBVTT
REGION
id:fred
width:40%
lines:3
regionanchor:0%,100%
viewportanchor:10%,90%
scroll:up
REGION
id:bill
width:40%
lines:3
regionanchor:100%,100%
viewportanchor:90%,90%
scroll:up
STYLE
::cue([v="Fred"]) {
color: blue;
}
00:00:00.000 --> 00:00:20.000 region:fred align:left
<v Fred>Hi, my name is Fred
00:00:02.500 --> 00:00:22.500 region:bill align:right
<v Bill>Hi, Iโm Bill
00:00:05.000 --> 00:00:25.000 region:fred align:left
<v Fred>Would
<00:00:05.250>you
<00:00:05.500>like
<00:00:05.750>to
<00:00:06.000>get
<00:00:06.250>a
<00:00:06.500>coffee?
The vertical-writing mode is not supported in the current situation, either in WEBVTT or the upcoming TTML Renderer. In order to support it, we should change the CaptionsRenderer in order to paint them.
Also, we should change the RenderingModifiers
in order to support eventual writing modifiers.
For what concerns WEBVTT, these are for sure the affected rows:
sub37/packages/webvtt-adapter/src/Parser/RenderingModifiers.ts
Lines 83 to 89 in 7974fc9
sub37/packages/webvtt-adapter/src/Parser/RenderingModifiers.ts
Lines 134 to 145 in 7974fc9
sub37/packages/webvtt-adapter/src/Parser/RenderingModifiers.ts
Lines 163 to 167 in 7974fc9
In order to support vertical writing, this resource might be considered valid. It contains W3C References for CSS, to which we might translate RenderingModifiers: https://w3c.github.io/i18n-drafts/articles/vertical-text/index.en
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.