Git Product home page Git Product logo

bendy's People

Contributors

0ndorio avatar casey avatar davids91 avatar delet0r avatar euclio avatar hdznrrd avatar oliveruv avatar thequux avatar trevyn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bendy's Issues

Implement `Clone` for `Object` and `DictDecoder`

Currently I have the following code in which I have two possible enumerations for an object, first I have to parse the bytes to figure out what object to initialize for the data and after that I have to parse the fields. I need the ability to clone either Object or DictDecoder for this:

impl FromBencode for Info {
    const EXPECTED_RECURSION_DEPTH: usize = 999;

    fn decode_bencode_object(
        object: bendy::decoding::Object,
    ) -> Result<Self, bendy::decoding::Error>
    where
        Self: Sized,
    {
        let dict = object.try_into_dictionary()?;
        let mut info = Info::default();

        // figure out what mode the info type is but gets consumed so borrow checker complains after this point
        info.mode = info.try_mode(dict).unwrap();

        match info.mode {
            // ...
        }
}

Switch to another error handling library

Hello,
I discovered bendy a few days ago, and noticed that it uses failure for error handling. Unfortunately, since commit
135e2a3
, failure is deprecated. The notice says that users may like to use either anyhow or thiserror.

Is there any plan in switching to another error handling library?

NestingTooDeep with any torrent file

I'm trying to parse torrent files, I've tried a dozen and with any of them i get the NestingTooDeep error, here's my code:

use bendy::decoding::FromBencode;
use bendy::encoding::AsString;
use std::fs::File;
use std::io::prelude::Read;

fn main() {
    let mut fc = Vec::new(); // file content
    let mut file = File::open("test.torrent").expect("Unable to read file.");
    file.read_to_end(&mut fc).expect("Unable to read");
    let example = AsString::from_bencode(&fc).expect("Unable to bendecode");
    println!("{:#?}", example.0);
}

New release

Would it be possible to push a new release of bendy on crates.io?

I'm preparing to release a crate that depends on serde support. Currently I'm depending on bendy via github, but released crates may only have crates.io dependencies.

`decode::Error::unexpected_field` should take `&[u8]`

The name of an unexpected field is generally available to client code as a &[u8]; it would be much more convenient to be able to report that directly than to have to convert it to a string (for example).

This would likely be a breaking change, thus necessitating a v0.3.0

Relax the enforced correctness rules when decoding?

Hi folks,

I use Bendy in my project for decoding a bencode and it worked very well for me so far. But data on the Internet is not well-formed in all cases.
So, I have some problems parsing bencode data which I'm getting from a server because this data is not well-formed (keys are not sorted). I can't tell the server to give me the response I want, because the server doesn't belong to me.

My question: Is there any way to relax the enforced correctness rules to be able to parse the response?

Thanks,
Eugene

How to handle unused tokens while decoding?

The port of additional test cases (#20) to fix issue #7 revealed a new question. The original test suite contained some cases marked as "illegal" which contain more tokens than actually required to restore the expected data type. Example: Decoding i12345e5:hellointo an integer.

How should FromBencode handle the case? Potential solutions I can think of:

  • silently drop them (as we do right now)
  • return an error variant which contains the unused tokens
  • add an additional default implementation next to from_bencode with stricter handling

Serde rename not working

Trying to use serde rename in a filed causes an UnexpectedToken error

#[derive(Debug, Serialize, Deserialize)]
pub struct MetaInfo {
    #[serde(rename = "creation date")]
    creation_date: Option<u64>,
}

let t = fs::read("temp/manjaro.torrent").unwrap();
let torrent: MetaInfo = bendy::serde::from_bytes(&t).unwrap();
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Decode(Error { context: None, error: UnexpectedToken("List", "Num") })', src/main.rs:70:58
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This makes it impossible to deserialize certain fields

`StructureError(UnsortedKeys)` on decoding?

I have the following FromBencode implementation which very surprisingly fails with an error that should be only relevant to encoding.

impl FromBencode for ScrapeResponse {
    const EXPECTED_RECURSION_DEPTH: usize = 5;
    fn decode_bencode_object(
        object: bendy::decoding::Object,
    ) -> Result<Self, bendy::decoding::Error>
    where
        Self: Sized,
    {
        // &bytes = b"d5:filesd20:\x1b\x84\xcb\xd2TZf\x8a\xd85\x04!\xd4\x1b\x88\x0f?d\xcc\xf4d8:completei17e10:incompletei2e10:downloadedi1539eeee"

        let mut dict = object.try_into_dictionary()?;
        let mut result = Vec::new();

        while let Some(files) = dict.next_pair()? {
            let mut files = files.1.try_into_dictionary()?;

            while let Some(file) = files.next_pair()? {
                let mut decoder = file.1.try_into_dictionary()?;
                let mut status: Status = Default::default();

                while let Some(pair) = decoder.next_pair()? {
                    match pair {
                        (b"complete", _) => {
                            status.seeders = u32::decode_bencode_object(pair.1)?;
                        }
                        (b"incomplete", _) => {
                            status.leechers = u32::decode_bencode_object(pair.1)?;
                        }
                        (b"downloaded", _) => {
                            let err = u32::decode_bencode_object(pair.1);
                            dbg!(err.unwrap());
                        }
                        _ => {}
                    }
                }
                result.push((file.0.to_vec(), status));
            }
        }
        Ok(Self { files: result })
    }
}
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { context: None, error: StructureError(UnsortedKeys) }

Change MAX_DEPTH of value::Value to 0

Per the documentation in encoding.rs, MAX_DEPTH of a variable-sized object should be set to 0 and then explicitly padded when creating the decoder.

This will fix #37 properly, but is a breaking change.

Actually allow FromBencode to return references to the input stream

Input streams with large blocks of binary data should be decodable without copying their contents.

This is probably best done by making FromBencode take 'ser (the lifetime of the data being deserialized) as a parameter, thus we could have impls for &'ser [u8], Cow<'ser, [u8]>, and (possibly) &'ser str and Cow<'ser, str>

Serde Derive Support

I've been using serde-bencode, but I ran into some issues, so I'm reconsidering my choice of library.

Bendy looks great, but I'm making heavy use of serde derived serialization and deserialization, which I think Bendy doesn't do.

Is this a feature which bendy might eventually support?

Support decoding an initial bencode structure and retrieving the remaining bytes

BEP 9 (the protocol for resolving magnet URIs) defines a data packet that consists of a bencoded dictionary followed by arbitrary data (actually a slice of a torrent's info dict), and I would like to parse such a packet using bendy. The best way I've figured out to do this so far is to create a Decoder, call next_object(), retrieve the DictDecoder, iterate through & massage the dict's fields, call DictDecoder::into_raw() after handling all fields, get the length of the raw dict, and use that length as an offset to slice the original payload. This is rather unwieldy, and I believe this could be simplified by just giving Decoder a remaining(self) -> &'ser [u8] method for retrieving all remaining bytes of the payload after doing next_object() + FromBencode::decode_bencode_object().

Failure in display message of `decoding::Error::UnexpectedToken`

The current display message is defined as

#[fail(display = "discovered {} but expected {}", _0, _1)]
but based on the usage in
pub fn unexpected_token(expected: impl Display, discovered: impl Display) -> Error {
Self::from(ErrorKind::UnexpectedToken(
expected.to_string(),
discovered.to_string(),
))
}
it should be the other way around.

`AsString` missing basic traits

The AsString wrapper should derive all of the basic traits (Default, Eq, Hash, ...) to prevent any kind of artificial restriction. E.g. without these derives it is not possible to parse a bencode dictionary into sth. like a HashMap<AsString<Vec<u8>>, _>.

Unable to encode Value type in a container

Easiest to illustrate with an example, attempting to encode an empty Vec<Value>

use bendy::{value::Value,encoding::ToBencode};

fn main() {
    let _ = Vec::<Value>::new().to_bencode();
}

Results in a compile error:

error[E0080]: could not evaluate constant
   --> $path/bendy-0.3.0/src/encoding/to_bencode.rs:114:38
    |
114 |             const MAX_DEPTH: usize = ContentT::MAX_DEPTH + 1;
    |                                      ^^^^^^^^^^^^^^^^^^^^^^^ attempt to add with overflow

Which I suspect is due to Value::MAX_DEPTH being the max_usize. A saturating add would be ideal in this case, but using a slightly lower MAX_DEPTH with room for a bunch parent containers may work, too.

Trying to deserialize bytes as SocketAddr

Hi there,

Thanks for the handy library!

I'm trying to deserialize an ip block from the Bittorrent DHT according to BEP 5.

I can use the following to deserialize the ip block into a byte vector:

#[derive(Debug, Deserialize)]
struct KRPCMessage {
    #[serde(rename = "t")]
    transaction_id: String,
    #[serde(rename = "y")]
    type_code: String,
    #[serde(rename = "v")]
    version: Option<String>,
    #[serde(rename = "r")]
    response: HashMap<String, ByteBuf>,
    #[serde(with = "serde_bytes")]
    ip: Vec<u8>,
}
from_bytes::<KRPCMessage>(&data).unwrap();

This works and gives me:

KRPCMessage { transaction_id: "aa", type_code: "r", version: None, response: {"id": [50, 245, 78, 105, 115, 81, 255, 74, 236, 41, 205, 186, 171, 242, 251, 227, 70, 124, 194, 103]}, ip: [70, 66, 178, 80, 26, 225] }

But what I would like to do is deserialize the ip block into a std::net::SocketAddr struct. I've tried various approaches including using the "new type" pattern over SocketAddr but I can't seem to figure out how to get it to work:

fn deserialize<'de, D>(deserializer: D) -> Result<SocketAddr, D::Error>
where
    D: Deserializer<'de>,
{
    Ok(SocketAddr::new(
        IpAddr::V4(Ipv4Addr::new(67, 215, 246, 10)),
        6881,
    ))
}

#[derive(Debug, Deserialize)]
struct KRPCMessage {
    #[serde(rename = "t")]
    transaction_id: String,
    #[serde(rename = "y")]
    type_code: String,
    #[serde(rename = "v")]
    version: Option<String>,
    #[serde(rename = "r")]
    response: HashMap<String, ByteBuf>,
    #[serde(with = "self")]
    ip: SocketAddr,
}
from_bytes::<KRPCMessage>(&data).unwrap();

I always get the error:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Decode(Error { context: None, error: MalformedContent(Utf8Error { valid_up_to: 2, error_len: Some(1) }) })'

It seems that it always try to decode the block as a utf-8 string.

Can anyone guide me on how to accomplish this? Please let me know if there any obvious errors—I'm new to Rust.

How to implement FromBencode for enum

I'm trying to implement FromBencode for an enum but I'm at a loose, decode_bencode_object doesn't take &self as a parameter so I don't have anything to match against to choose an enum variant

Relevant code
pub enum Info {
    SingleFile {
        length: u64,
        md5sum: Option<String>,
        name: String,
        piece_length: u64,
        pieces: Vec<u8>,
        private: Option<u8>,
    },
    MultiFile {
        files: Vec<File>,
        name: String,
        pieces: Vec<u8>,
        piece_length: u64,
        private: Option<u8>,
    },
}

impl FromBencode for MetaInfo {
   const EXPECTED_RECURSION_DEPTH: usize = 3;

   fn decode_bencode_object(object: Object) -> Result<Self, DecodingError>
       where
           Self: Sized,
       {
           // I can't match self because the trait signature doesn't include it
       }
}

I was able to easily implement ToBencode by matching self

Relevant code
impl ToBencode for Info {
    const MAX_DEPTH: usize = 1;

    fn encode(&self, encoder: SingleItemEncoder) -> Result<(), EncodingError> {
        match self {
            Info::SingleFile {
                name,
                length,
                md5sum,
                piece_length,
                pieces,
                private,
            } => encoder.emit_unsorted_dict(|e| {
                e.emit_pair(b"name", name)?;
                e.emit_pair(b"length", length)?;
                if let Some(sum) = md5sum {
                    e.emit_pair(b"md5sum", sum)?;
                }
                e.emit_pair(b"piece length", piece_length)?;
                e.emit_pair(b"pieces", AsString(pieces))?;
                if let Some(p) = private {
                    e.emit_pair(b"private", p)?;
                }
                Ok(())
            })?,
            Info::MultiFile {
                name,
                files,
                piece_length,
                pieces,
                private,
            } => encoder.emit_unsorted_dict(|e| {
                e.emit_pair(b"name", name)?;
                e.emit_pair(b"files", files)?;
                e.emit_pair(b"piece length", piece_length)?;
                e.emit_pair(b"pieces", AsString(pieces))?;
                if let Some(p) = private {
                    e.emit_pair(b"private", p)?;
                }
                Ok(())
            })?,
        }
        Ok(())
    }
}

Hopefully, I'm missing something obvious but I haven't found any example of this

Add methods to sort lists on output

Many times Vec<T> is used as a set datastructure, where nothing cares about the order of elements within the list. To allow such lists to easily be emitted canonically, we should have methods on the various Enocder types (including SingleItemEncoder) that buffer the encoded objects and output them sorted by either Bencode order (i.e, ordered ASCIIbetically by their encoded form) or by any existing Ord impl (if one exists)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.