austinleroy / ebml-iterable Goto Github PK

Hi, I'm trying to use this crate to remux an ffmpeg output to add the Clues elements, my project requires that the output is streamed over HTTP so I need to be able to send stuff out even with un-closed tags, at the moment by reading the code my understanding is that everything is written to the destination only when flush is called or there are no open tags, problem with that is that the public flush() of the TagWriter also closes the tags ending the stream prematurely which make it impossible to send uncompleted data to the browser.

would it be possible to add a dirty flush or something like that?

One of the variants of TagIteratorError uses the ToolError type, but the latter is not exported, making it "unnamable" from outside. This is the kind of situation described in rust-lang/rust#34537. I'd like to access data stored in this error, but there's no way to extract it.

Can you consider exporting ToolError (and maybe reexport it in webm-iterable)?

I'm having trouble reading back some data written with the library (mkv in this case). This seems to happen if one element exceeds the default buffer size of the reader; and this element is not the last element in its parent master element; and that master element is towards the end of the file.

Reproduce with this minimize program:

    use ebml_iterable::specs::{ebml_specification, TagDataType, Master};
    
    #[ebml_specification]
    #[derive(Clone, Debug, PartialEq)]
    pub enum Trial {
        #[id(0x18538067)]
        #[data_type(TagDataType::Master)]
        Segment,

        #[id(0x1F43B675)]
        #[data_type(TagDataType::Master)]
        Cluster,

        #[id(0x4100)]
        #[data_type(TagDataType::UnsignedInt)]
        Count,

        #[id(0xa1)]
        #[data_type(TagDataType::Binary)]
        Block,
    }

    #[test]
    pub fn roundtrip() {
        use ebml_iterable::{TagIterator, TagWriter};
        use std::io::Cursor;
        let mut dest = Cursor::new(Vec::new());
        let mut writer = TagWriter::new(&mut dest);

        writer.write(&Trial::Segment(Master::Start)).expect("Error writing tag");
        writer.write(&Trial::Cluster(Master::Start)).expect("Error writing tag");
        // Why 0x10001 specifically? This exceeds the default buffer length (0x10000)?!
        writer.write_raw(0xa1, &[0x00; 0x10001]).expect("Error writing tag");
        writer.write(&Trial::Count(0x00)).expect("Error writing tag");
        writer.write(&Trial::Cluster(Master::End)).expect("Error writing tag");
        writer.write(&Trial::Segment(Master::End)).expect("Error writing tag");
        drop(writer);

        dest.set_position(0);
        let iter = TagIterator::<_, Trial>::new(dest, &[]);

        let tags: Vec<_> = iter.into_iter().collect();
        assert_eq!(tags.len(), 4+2, "Reading every tag that was written");
   }

While scouting code I found one suspicious line:

If I understand correctly this tries to read at least self.internal_buffer_position more bytes of data. It's also called after that 0xa1 element and the internal buffer position is 65555 at that point. This will surely result in fully reading the file, then getting an empty read, and then returning Ok(false)—which in turn is interpreted as being done even though the tag was not finished.

To trigger this failure condition the large element must end at a full buffer, exactly, and at the same time less than a full buffer worth of data must be remaining in the file.