austinleroy / ebml-iterable Goto Github PK
View Code? Open in Web Editor NEWLibrary for iterating over EBML using Rust
License: MIT License
Library for iterating over EBML using Rust
License: MIT License
Hi, I'm trying to use this crate to remux an ffmpeg output to add the Clues elements, my project requires that the output is streamed over HTTP so I need to be able to send stuff out even with un-closed tags, at the moment by reading the code my understanding is that everything is written to the destination only when flush is called or there are no open tags, problem with that is that the public flush() of the TagWriter also closes the tags ending the stream prematurely which make it impossible to send uncompleted data to the browser.
would it be possible to add a dirty flush or something like that?
Lines 67 to 78 in 37e160c
One of the variants of TagIteratorError
uses the ToolError
type, but the latter is not exported, making it "unnamable" from outside. This is the kind of situation described in rust-lang/rust#34537. I'd like to access data stored in this error, but there's no way to extract it.
Can you consider exporting ToolError
(and maybe reexport it in webm-iterable
)?
I'm having trouble reading back some data written with the library (mkv in this case). This seems to happen if one element exceeds the default buffer size of the reader; and this element is not the last element in its parent master element; and that master element is towards the end of the file.
Reproduce with this minimize program:
use ebml_iterable::specs::{ebml_specification, TagDataType, Master};
#[ebml_specification]
#[derive(Clone, Debug, PartialEq)]
pub enum Trial {
#[id(0x18538067)]
#[data_type(TagDataType::Master)]
Segment,
#[id(0x1F43B675)]
#[data_type(TagDataType::Master)]
Cluster,
#[id(0x4100)]
#[data_type(TagDataType::UnsignedInt)]
Count,
#[id(0xa1)]
#[data_type(TagDataType::Binary)]
Block,
}
#[test]
pub fn roundtrip() {
use ebml_iterable::{TagIterator, TagWriter};
use std::io::Cursor;
let mut dest = Cursor::new(Vec::new());
let mut writer = TagWriter::new(&mut dest);
writer.write(&Trial::Segment(Master::Start)).expect("Error writing tag");
writer.write(&Trial::Cluster(Master::Start)).expect("Error writing tag");
// Why 0x10001 specifically? This exceeds the default buffer length (0x10000)?!
writer.write_raw(0xa1, &[0x00; 0x10001]).expect("Error writing tag");
writer.write(&Trial::Count(0x00)).expect("Error writing tag");
writer.write(&Trial::Cluster(Master::End)).expect("Error writing tag");
writer.write(&Trial::Segment(Master::End)).expect("Error writing tag");
drop(writer);
dest.set_position(0);
let iter = TagIterator::<_, Trial>::new(dest, &[]);
let tags: Vec<_> = iter.into_iter().collect();
assert_eq!(tags.len(), 4+2, "Reading every tag that was written");
}
While scouting code I found one suspicious line:
ebml-iterable/src/tag_iterator.rs
Line 251 in 4247ca7
If I understand correctly this tries to read at least self.internal_buffer_position
more bytes of data. It's also called after that 0xa1
element and the internal buffer position is 65555
at that point. This will surely result in fully reading the file, then getting an empty read, and then returning Ok(false)
—which in turn is interpreted as being done even though the tag was not finished.
To trigger this failure condition the large element must end at a full buffer, exactly, and at the same time less than a full buffer worth of data must be remaining in the file.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.