Git Product home page Git Product logo

bytes's Introduction

Bytes

A utility library for working with bytes.

Crates.io Build Status

Documentation

Usage

To use bytes, first add this to your Cargo.toml:

[dependencies]
bytes = "1"

Next, add this to your crate:

use bytes::{Bytes, BytesMut, Buf, BufMut};

Serde support

Serde support is optional and disabled by default. To enable use the feature serde.

[dependencies]
bytes = { version = "1", features = ["serde"] }

Building documentation

When building the bytes documentation the docsrs option should be used, otherwise feature gates will not be shown. This requires a nightly toolchain:

RUSTDOCFLAGS="--cfg docsrs" cargo +nightly doc

License

This project is licensed under the MIT license.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in bytes by you, shall be licensed as MIT, without any additional terms or conditions.

bytes's People

Contributors

alexcrichton avatar asomers avatar braddunbar avatar carllerche avatar cyborus04 avatar danburkert avatar darksonn avatar doumanash avatar espindola avatar fhartwig avatar jq-rs avatar kohensu avatar kpp avatar mina86 avatar nanpuyue avatar nobodyxu avatar nox avatar nzentzis avatar paolobarbolini avatar phantomical avatar ralfjung avatar robjtede avatar seanmonstar avatar sfackler avatar stepancheg avatar taiki-e avatar timhambourger avatar tmiasko avatar vorner avatar zettroke avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bytes's Issues

Define API behavior

  • What does bytes_vec do when remaining == 0

Probably guaranteed to return 0.

  • Can the argument to bytes_vec be an empty slice?

Yes

  • Buf::advance past remaining

Maybe panic?

`io::Cursor` is a bit out of place

I think this may not wish to use io::Cursor pervasively. This use case doesn't have much to do with std::io. The std version also stores a u64 which isn't needed for bytes's use case I believe.

Bytes[Mut]::into_vec()

It would be convenient to have Bytes::into_vec() and BytesMut::into_vec() operations.

These operations should return owned Vec if KIND_VEC or KIND_ARC and refcount is 1.

Ability to opt out of thread safety

Apparently right now you need to use the internal AtomicPtr even though the implementation might never see multiple threads. You should be able to statically opt out of thread safety somehow.

RingBuf::reset sets full buffers to empty

Calling RingBuf::reset on a full buffer causes the buffer to become empty, due to the recalculation of len from pos and mark, as demonstrated by this test case:

let mut buf = RingBuf::new(8);
buf.write(&[1, 2, 3, 4, 5, 6, 7, 8]).unwrap();
assert_eq!(MutBuf::remaining(&buf), 0);
buf.mark();
buf.reset();
assert_eq!(MutBuf::remaining(&buf), 0);

Bytes::concat(Bytes, Bytes)

Implement Bytes::concat(Bytes, Bytes).

It can do two optimizations beyond just allocating a new buffer and copying data there:

  • if either parameter is empty, return the other one
  • if either parameter is "mut safe" and have enough capacity, copy the other one to it (this one is not possible to do from the outside of bytes crate)
  • otherwise allocate a new buf and copy both parameters to it

BytesMut::reserve not working for most values.

It's possible I'm missunderstanding something, but this seems to be a bug: most calls to bytes.reserve(a) on a buffer with capacity b result in the buffer having a capacity smaller than a + b.

extern crate bytes;
use bytes::*;

#[test]
fn bytes_reserve() {
   let mut bytes = BytesMut::with_capacity(20);
   bytes.reserve(12);
   bytes.put(vec![0; 32]);

   let mut bytes = BytesMut::with_capacity(20);
   bytes.reserve(44);
   bytes.put(vec![0; 64]);
}

Both calls to bytes.put panic:

thread 'bytes_reserve' panicked at 'assertion failed: self.remaining_mut() >= src.remaining()', /home/xal/.cargo/git/checkouts/bytes-72cf269819849e12/37f6cab/src/buf/buf_mut.rs:229```

Drop original capacity and implement truncate_front

Drop original_capacity

Currently bytes crate have a logic of preserving "original capacity" during split.

So

drop(bytes.take());
// `bytes` here will have original capacity preserved if original capacity was <= 16K

I'm not sure if this logic is actually useful, and it may be confusing (e. g. you got a 100 bytes buffer, you called reserve(1) and got 64K allocated).

I'd simply remove it (even without implementing truncate_front).

Implement truncate_front

On the other hand, without "original capacity" field it would be possible to implement cheap truncate_front operation.

truncate_front is the same as truncate, but truncates from front, not from back.

truncate_front is implemented trivally for KIND_ARC, KIND_STATIC and KIND_INLINE.

And for KIND_VEC truncate_front can be implementing by changing arc field of Inner struct to store a pointer to original Vec.

So for KIND_VEC

struct Inner {
    arc: AtomicPtr<Shared>, // vec.as_mut()
    ptr: *mut u8, // position to current view
    len: usize, // len of current view
    cap: usize, // capacity to current view
}

So to reconstruct Vec from Inner (e. g. to convert to Arc) you could write something like:

fn reconstruct_vec(Inner inner) {
    debug_assert!(inner.kind() == KIND_VEC);
    let offset = inner.ptr - inner.arc;
    Vec::from_raw_parts(inner.arc, inner.len + offset, inner.cap + offset)
}

Why truncate front

I believe there are lot of sitations when you need to take a Bytes and simply truncate front and back. For example, if you parse HTTP/2 frame, you can take Bytes by value, and truncate headers and padding to retrieve payload as Bytes. If kind == KIND_VEC this operation will be cheaper than currently with split_off, because Bytes won't need to be converted to KIND_ARC.

Consider parameterization of Bytes with lifetime

Bytes can be transformed into Bytes<'a>.

Bytes<'a> is identical to current Bytes, except when KIND_STATIC, ptr lifetime is 'a, but not 'static'. So Bytes<'static> is identical to current Bytes.

Why

Suppose you have Bytes object. And you want, for example, split that Bytes into several slices and pass these objects somewhere else, and you keep original Bytes object. For example, you want to create a sequence of

struct MyProtocolFrame {
   header: Header,
   payload: Bytes,
}

and pass these objects down to serialization. You cannot change payload type to &'a [u8], because sometimes MyProtocolFrame must own data (in different part of project), and sometimes it could simply hold borrowed data.

Currently you can "borrow" with Bytes::slice operation, which is expensive, because it:

  • upgrades Bytes to KIND_ARC when Bytes is KIND_VEC
  • does expensive atomic increment and decrement

If Bytes is parameterized with lifetime, Bytes<'a> could have another cheap operation like:

impl Bytes {
    fn slice_ref<'a>(&'a self, from: usize, to: usize) -> Bytes<'a> {
        Bytes::from_borrowed(self.as_ref()[from..to])
    }
}

A little bonus

And as a little bonus, you can now create a borrowed Bytes object from &[u8] which is obtained from some library which doesn't used Bytes. Sometimes that's convenient.

Drawback

The major problem with this design in ergonomics: users would have to write lifetimes a lot, and get lots of compiler errors, which is annoying.

I can think of a couple of ways to avoid this problem.

First, declare a type alias:

// Flexible type for advanced use cases
pub struct BytesMaybeBorrowed<'a> { ... }

// Identical to current Bytes
pub type Bytes = BytesMaybeBorrowed<'static>;

or

pub struct BytesMaybeBorrowed<'a> { ... }

pub struct Bytes(pub BytesMaybeBorrowed<'static>);

Second version seems to be easier to use, but it requires duplication of lots of functions. Which is fine by me.

What do you think?

Need method to mutably access already written parts of MutByteBuf

It would be convenient to have something like MutByteBuf::mut_bytes() that gives access to the already written part of the byte buffer instead of the remaining memory area. This would allow to modify something that was received from the network, and potentially forwarding it to some other socket, without having to copy everything.

tests cannot be compiled with rust beta or nightly

% cargo test
   Compiling bytes v0.4.1 (file:///Users/yozh/devel/left/bytes)
error[E0502]: cannot borrow `buf` as mutable because it is also borrowed as immutable
  --> tests/test_chain.rs:64:5
   |
58 |     assert_eq!(2, buf.bytes_vec(&mut iovecs));
   |                   --- immutable borrow occurs here
...
64 |     buf.advance(2);
   |     ^^^ mutable borrow occurs here
...
93 | }
   | - immutable borrow ends here

error[E0502]: cannot borrow `buf` as mutable because it is also borrowed as immutable
  --> tests/test_chain.rs:74:5
   |
58 |     assert_eq!(2, buf.bytes_vec(&mut iovecs));
   |                   --- immutable borrow occurs here
...
74 |     buf.advance(3);
   |     ^^^ mutable borrow occurs here
...
93 | }
   | - immutable borrow ends here

error[E0502]: cannot borrow `buf` as mutable because it is also borrowed as immutable
  --> tests/test_chain.rs:84:5
   |
58 |     assert_eq!(2, buf.bytes_vec(&mut iovecs));
   |                   --- immutable borrow occurs here
...
84 |     buf.advance(3);
   |     ^^^ mutable borrow occurs here
...
93 | }
   | - immutable borrow ends here

error: aborting due to 3 previous errors

error: Could not compile `bytes`.

To learn more, run the command again with --verbose.
% rustc --version
rustc 1.17.0-beta.1 (408d49e60 2017-03-14)

Probably, it's compiler bug.

BytesMut lacks an implementation of std::io::Write

Copied from tokio-rs/tokio-io#28

I'm currently using the codec module to implement a framed tansport.

Before the refactoring, the encode-method of the old Codec trait received a Vec as parameter to serialize the message into. Vec directly implements std::io::Write which makes it very easy to put any kind of data into it, since most implementations have some way of writing into an instance of Write.

Now the Vec has been replaced by BytesMut, which lacks a direct implementation of that trait. Write-compatibility is only archievable through into_writer, which consumes the BytesMut. Consuming is not possible, though, because the method only receives the BytesMut by &mut.

Is this desired behavior?

Consider reducing the struct size

Great work on the library, super useful. I was wondering if you would consider reducing the Buffer struct to 24bytes (64bits) (by using 32bit len/cap or something else)?

This will reduce the inline capacity but 23 is still plenty in practice. Even if moving single Buffers around is essentially the same speed (2x128bit moves vs 1x128 and 1x64bit) it really helps when you have more than a single buffer in a struct, enum, array, etc..

ReadExt may not be safe

The current implementation exposes uninitialized memory via mut_bytes to arbitrary implementors of the Read trait. Unfortunately though arbitrary implementors don't guarantee that they won't read the uninitialized data.

Provide support for a &'static [u8] in Bytes

It'd be nice if Bytes::from(b"foo bar") would just grab the static pointer, and not copy the bytes.

This would probably requiring changing the representation slightly of Bytes since it couldn't be a BytesMut of a static memory pointer. Additionally, the try_mut and into_mut methods would not be allowed to return a BytesMut in that case. Would be very useful for hyperium/hyper#953

Consider getting rid of BytesMut in favor of only Bytes

Currently, Bytes represents an immutable, potentially shared, view into a memory region. BytesMut represents a unique view into a memory region, which means that the memory can be mutated.

Perhaps there is a way to collapse these two types into a single type.

Pros

  • Less types

Cons

  • Mutation may have to implicitly copy the memory into a new buffer... this expense may not be obvious. It may also be that extra runtime state may be needed to track if a handle has a unique view or not.

Build fails on latest nightly

The build currently fails on the latest nightly because of rust-lang/rust@d131f33

The full compiler error:

src/alloc/mod.rs:67:18: 67:32 error: mutating transmuted &mut T from &T may cause undefined behavior,consider instead using an UnsafeCell, #[deny(mutable_transmutes)] on by default
src/alloc/mod.rs:67         unsafe { mem::transmute(self.bytes()) }
                                     ^~~~~~~~~~~~~~
src/str/bytes.rs:117:18: 117:32 error: mutating transmuted &mut T from &T may cause undefined behavior,consider instead using an UnsafeCell, #[deny(mutable_transmutes)] on by default
src/str/bytes.rs:117         unsafe { mem::transmute(self.obj()) }
                                      ^~~~~~~~~~~~~~
error: aborting due to 2 previous errors

I'd fix it myself, but I probably shouldn't try to fix unsafe code that I don't fully understand.

Docs don't include public structures

For instance, ByteBuf certainly looks like it's trying to be publicly documented, but it's not showing up in the rustdoc. All you see is a list of re-exports from private modules.

Debug assertion fails when splitting a Bytes created with from_static

extern crate bytes;

use bytes::Bytes;

fn main() {
    let mut a = Bytes::from_static(b"ab");
    let b = a.split_off(1);
    println!("{:?}, {:?}", a, b);
}

In a debug build, this code results in the following:

thread 'main' panicked at 'assertion failed: self.is_shared()', /Users/foo/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.4.1/src/bytes.rs:1510
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: bytes::bytes::Inner::set_start
   1: bytes::bytes::Inner::split_off
   2: bytes::bytes::Bytes::split_off
   3: bytes_test::main

A release build works fine:

[97], [98]

panic after few incoming messages

thread 'main' panicked at 'assertion failed: end <= self.cap', /Users/raviteja/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.4.2/src/bytes.rs:1607
stack backtrace:
   1:        0x107c204fc - std::sys::imp::backtrace::tracing::imp::write::h21ca2762819c7ae8
   2:        0x107c225ae - std::panicking::default_hook::{{closure}}::h38f99a37d00bb19b
   3:        0x107c22250 - std::panicking::default_hook::ha2186ee24b50729c
   4:        0x107c22a67 - std::panicking::rust_panic_with_hook::h979db19ee91d2a53
   5:        0x107be73d3 - std::panicking::begin_panic::h55f38a455bc31745
   6:        0x107bea488 - bytes::bytes::Inner::set_end::h82befb8e78b27d8f
   7:        0x107bea0c7 - bytes::bytes::Inner::split_to::hdf30edda54ab41e6
   8:        0x107be9c3b - bytes::bytes::BytesMut::split_to::h58c00b1381d01291
   9:        0x107af9bf1 - <rumqttd::codec::MqttCodec as tokio_io::framed_read::Decoder>::decode::hc5c6fc4b82691933
  10:        0x107af45e2 - <tokio_io::framed::Fuse<T, U> as tokio_io::framed_read::Decoder>::decode::hf27d43a9a8b23015
  11:        0x107af6c8b - <tokio_io::framed_write::FramedWrite2<T> as tokio_io::framed_read::Decoder>::decode::h8e63524e3ad209a5
  12:        0x107af1543 - <tokio_io::framed_read::FramedRead2<T> as futures::stream::Stream>::poll::had597800fa87b314
  13:        0x107af055b - <tokio_io::framed::Framed<T, U> as futures::stream::Stream>::poll::h0add0986ba64e75f
  14:        0x107af1fb2 - <futures::stream::split::SplitStream<S> as futures::stream::Stream>::poll::h3fd48e3dce02280d
  15:        0x107a6b8f7 - <futures::stream::for_each::ForEach<S, F, U> as futures::future::Future>::poll::hfe201db90d0693d7
  16:        0x107ad61ed - <futures::future::chain::Chain<A, B, C>>::poll::hb70001641cce7c49
  17:        0x107af813c - <futures::future::then::Then<A, B, F> as futures::future::Future>::poll::h96d61bd1db456977
  18:        0x107b2d3c1 - <alloc::boxed::Box<F> as futures::future::Future>::poll::ha15f7165eb975205
  19:        0x107b17f8c - <futures::task_impl::Spawn<F>>::poll_future::{{closure}}::h7fa62e61d191c22c
  20:        0x107b1847e - <futures::task_impl::Spawn<T>>::enter::{{closure}}::he00427d7d77b3a1b
  21:        0x107b2f77a - futures::task_impl::set::{{closure}}::h9e1b772b2c99b467
  22:        0x107b1a211 - <std::thread::local::LocalKey<T>>::with::hf1203554760ba1fd
  23:        0x107b2f633 - futures::task_impl::set::h261628f58d78254f
  24:        0x107b18332 - <futures::task_impl::Spawn<T>>::enter::hf7dd52e83ef51739
  25:        0x107b17f0f - <futures::task_impl::Spawn<F>>::poll_future::h0f85f03fa28897d6
  26:        0x107b3cab6 - tokio_core::reactor::Core::dispatch_task::{{closure}}::h0acc6ca4c4f4174b
  27:        0x107b141b4 - <scoped_tls::ScopedKey<T>>::set::h7f15a903853429bc
  28:        0x107b3c4c5 - tokio_core::reactor::Core::dispatch_task::hc3fd8d6531a32c3c
  29:        0x107b3b765 - tokio_core::reactor::Core::dispatch::ha135ccae5690bfb4
  30:        0x107b3b2d9 - tokio_core::reactor::Core::poll::h76e14e964798dba8
  31:        0x107a6c69e - tokio_core::reactor::Core::run::hdb4568c67ea0558d
  32:        0x107aff980 - rumqttd::main::h380ac17aef49e4e9
  33:        0x107c239da - __rust_maybe_catch_panic
  34:        0x107c22e36 - std::rt::lang_start::hfc9882558f9403bf
  35:        0x107b02609 - main

here is my decode for reference

fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<Packet>> {
        // NOTE: `decode` might be called with `buf.len == 0` when prevous
        // decode call read all the bytes in the stream. We should return
        // Ok(None) in those cases or else the `read` call will return
        // Ok(0) => translated to UnexpectedEOF by `byteorder` crate.
        // `read` call Ok(0) happens when buffer specified was 0 bytes in len
        // https://doc.rust-lang.org/std/io/trait.Read.html#tymethod.read
        if buf.len() == 0 {
            return Ok(None);
        }

        let (packet, len) = {
            let mut buf_ref = buf.as_ref();
            match buf_ref.read_packet_with_len() {
                Err(e) => {
                    if let mqtt3::Error::Io(e) = e {
                        match e.kind() {
                            ErrorKind::TimedOut | ErrorKind::WouldBlock => return Ok(None),
                            _ => return Err(io::Error::new(e.kind(), e.description())),
                        }
                    } else {
                        return Err(io::Error::new(ErrorKind::Other, e.description()));
                    }
                }
                Ok(v) => v,
            }
        };

        buf.split_to(len);

        // println!("{:?}, {:?}, {:?}", len, packet, buf.len());
        Ok(Some(packet))
}

Publish recent updates to crates.io

The latest version of bytes on crates.io (v0.2.8) does not appear to include this commit which looks like it fixes the build error I'm seeing from Cargo:

/home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.2.8/src/alloc/mod.rs:67:18: 67:32 error: mutating transmuted &mut T from &T may cause undefined behavior,consider instead using an UnsafeCell, #[deny(mutable_transmutes)] on by default
/home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.2.8/src/alloc/mod.rs:67         unsafe { mem::transmute(self.bytes()) }
                                                                                                              ^~~~~~~~~~~~~~
/home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.2.8/src/str/bytes.rs:117:18: 117:32 error: mutating transmuted &mut T from &T may cause undefined behavior,consider instead using an UnsafeCell, #[deny(mutable_transmutes)] on by default
/home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.2.8/src/str/bytes.rs:117         unsafe { mem::transmute(self.obj()) }

Index/indexMut instead of slice function

It would be really cool if you could just use slicing syntax on a Buf and get a Buf again, like let b: Bytes = Bytes::from(&b"12323456"[..])[0..2];. That can be done via implementing the Index/IndexMut trait for ranges.

Right now you have the slice function, that's quite ugly.

Chars wrapper around Bytes

Would be convenient if bytes trait had a thin Chars wrapper around Bytes.

Chars should be identical to Bytes except it should guarantee that it always contains valid UTF-8 sequence.

Chars to Bytes is the same as String to Vec<u8>.

Tighten BytesMut guarantees

The docs for BufMut::bytes_mut specifically allow for the method to return an empty slice . This should be changed so that the method may only return a 0-length slice if the buffer is exhausted. Otherwise, it's extremely easy to trigger infinite loops (for instance, the current implementation of BufMut::put_slice). I believe the stdlib Write trait puts similar restrictions on 0-length writes for similar reasons.

Likewise, BufMut::bytes_vec_mut should not return a 0-length vector unless the buffer is exhausted, and probably shouldn't return 0-length IoVecs ever.

Consider returning a slice from bytes_vec in 0.5

From a caller perspective I think this is more ergonomic, but from an implementation perspective it's a bit more of a PITA because of more lifetimes. We should see what usage arises during 0.4 and see if we'd benefit much from this change.

cc #60

Removing the RingBufReader and RingBufWriter types

This is really a question more than an issue:
Could we remove the RingBuf Reader and Writer newtypes (and implement Buf and MutBuf directly on RingBuf)? I don't actually understand what purpose they have, apart from disambiguating method names from Buf and MutBuf (but that could be done in other ways).
If we could get rid of them, that would massively improve the ergonomics of RingBuf, at least for my use case. There are two problems that I have with them:

  • There are about a dozen instances in my code where I have to write something along the lines of let r = buf.reader(); let s = r.bytes(); where I would like to write let s = buf.bytes() instead. (Writing let s = buf.reader().bytes() doesn't work because the returned reader would not live long enough). This is admittedly only a minor annoyance.
  • More importantly, I have a struct containing a RingBuf and I would like to implement a method returning a slice into that RingBuf. I can't do that, though, since the slice I want to return cannot outlive the RingBufReader that I have to create in that method. This is a bigger problem than the first one, since I don't think I can work around it.

I'd be interested in an explanation of the necessity of the newtypes or any thoughts you have on the subject. If you think we could get rid of them, I'd be happy to do the implementation work.

Do not atomic decrement in drop when refcount == 1

There's PR #88 which implements it.

With patch applied, benchmark becomes measurably (about 5%) faster.

Same optimization is implemented in libc++:

void
__shared_weak_count::__release_weak() _NOEXCEPT
{
    // NOTE: The acquire load here is an optimization of the very
    // common case where a shared pointer is being destructed while
    // having no other contended references.
    //
    // BENEFIT: We avoid expensive atomic stores like XADD and STREX
    // in a common case.  Those instructions are slow and do nasty
    // things to caches.
    //
    // IS THIS SAFE?  Yes.  During weak destruction, if we see that we
    // are the last reference, we know that no-one else is accessing
    // us. If someone were accessing us, then they would be doing so
    // while the last shared / weak_ptr was being destructed, and
    // that's undefined anyway.
    //
    // If we see anything other than a 0, then we have possible
    // contention, and need to use an atomicrmw primitive.
    // The same arguments don't apply for increment, where it is legal
    // (though inadvisable) to share shared_ptr references between
    // threads, and have them all get copied at once.  The argument
    // also doesn't apply for __release_shared, because an outstanding
    // weak_ptr::lock() could read / modify the shared count.
    if (__libcpp_atomic_load(&__shared_weak_owners_, _AO_Acquire) == 0)
    {
        // no need to do this store, because we are about
        // to destroy everything.
        //__libcpp_atomic_store(&__shared_weak_owners_, -1, _AO_Release);
        __on_zero_shared_weak();
    }
    else if (__libcpp_atomic_refcount_decrement(__shared_weak_owners_) == -1)
        __on_zero_shared_weak();
}

Is Buf::bytes allowed to return an incomplete slice from the underlying buffer?

I see that Chain<T, U> implements Buf, and that its bytes method calls the bytes method of the first input if it isn't fully consumed, and otherwise of the second input. That means that bytes never return a slice to the complete input when the first one isn't empty yet.

Is that a bug? Is that intended? The documentation of the bytes method mislead me thinking this was illegal.

Implement quickcheck::Arbitrary for Bytes and BytesMut

See http://burntsushi.net/rustdoc/quickcheck/trait.Arbitrary.html for a description.

Quickcheck is a randomized testing tool that lets you check general properties. To make your type part of the Quickcheck ecosystem, you need to implement Arbitrary, which involves two methods:

(1) generate a random input
(2) shrink a failing input

This should be relatively easy -- probably just using the Vec code underneath.

This can be behind a feature gate to make sure that if you aren't using quickcheck otherwise, you don't have to pull it in.

error: unused import: `ByteStr`

[stewart@rivergod:~/dev/fractalide/nixcrates/nixcrates]$ nix-build -A allCrates.bytes
these derivations will be built:
  /nix/store/knncqp6lagbsw1ilbrvj163big9v8j2q-bytes.drv
building path(s) ‘/nix/store/m4wc5x700z5raw723w4h42v9k8lwsskm-bytes’
unpacking sources
bytes-0.3.0/Cargo.toml
bytes-0.3.0/LICENSE
bytes-0.3.0/README.md
bytes-0.3.0/src/alloc/heap.rs
bytes-0.3.0/src/alloc/mod.rs
bytes-0.3.0/src/buf/byte.rs
bytes-0.3.0/src/buf/mod.rs
bytes-0.3.0/src/buf/ring.rs
bytes-0.3.0/src/buf/sink.rs
bytes-0.3.0/src/buf/slice.rs
bytes-0.3.0/src/buf/source.rs
bytes-0.3.0/src/buf/take.rs
bytes-0.3.0/src/lib.rs
bytes-0.3.0/src/str/bytes.rs
bytes-0.3.0/src/str/mod.rs
bytes-0.3.0/src/str/rope.rs
bytes-0.3.0/src/str/seq.rs
bytes-0.3.0/src/str/small.rs
bytes-0.3.0/TODO.md
patching sources
configuring
no configure script, doing nothing
building
bytes - 
namefix bytes
name bytes
About to use rustc to compile some lib - bytes
error: unused import: `ByteStr`
 --> src/buf/byte.rs:2:38
  |
2 | use traits::{Buf, MutBuf, MutBufExt, ByteStr};
  |                                      ^^^^^^^
  |
note: lint level defined here
 --> src/lib.rs:2:9
  |
2 | #![deny(warnings)]
  |         ^^^^^^^^

error: unused import: `Any`
 --> src/str/bytes.rs:4:16
  |
4 | use std::any::{Any, TypeId};
  |                ^^^

error: unused import: `MutBuf`
 --> src/str/small.rs:2:19
  |
2 | use traits::{Buf, MutBuf, ByteStr, ToBytes};
  |                   ^^^^^^

error: aborting due to 3 previous errors

builder for ‘/nix/store/knncqp6lagbsw1ilbrvj163big9v8j2q-bytes.drv’ failed with exit code 101
error: build of ‘/nix/store/knncqp6lagbsw1ilbrvj163big9v8j2q-bytes.drv’ failed

Implement BufMut for std::collections::VecDeque?

In some networking scenarios, a growable ring buffer is more appropriate than a growable vector. In particular, it can do drain without a memmove.

The standard library has an implementation, VecDeque. I wonder if it would be possible (or makes sense at all) to implement BufMut for it, similarly to how it's implemented for Vec?

Doesn’t compile on 32 bit platforms

It doesn’t compile on 32 bit platforms.

src/alloc.rs:5:32: 5:39 error: bitshift exceeds the type's number of bits, #[deny(exceeding_bitshifts)] on by default
src/alloc.rs:5 const MAX_ALLOC_SIZE: usize = (1 << 32) - 1;
                                              ^~~~~~~

Buf::copy_to_bytes

Buf could have copy_to_bytes operation:

fn copy_to_bytes(&mut self, len: usize) -> Bytes;

Default implementation should simply allocate new Bytes, while implementation for Bytes could call slice.

Buffer overflow

Buffer overflow in impl<'a> Sink for &'a mut Vec<u8>

    let mut sink: Vec<u8> = Vec::new();
    sink.reserve(16);
    println!("Capacity: {}", sink.capacity());
    let mut source = Cursor::new(b"0123456789abcdef0123456789abcdef");
    sink.copy_from(&mut source);
    println!("Length: {}, Capacity: {}", sink.len(), sink.capacity());

prints:

Capacity: 16
Length: 32, Capacity: 16

The error is in the call to reserve: you don't reserve "more capacity", instead you are only guaranteed there is enough space to insert the requested number of elements, i.e. you need to base it on length instead of capacity.

Is there a really good reason why you need unsafe here?

    self.resize(buf.remaining(), 0);
    buf.read_slice(&mut self[..]);

Is not zeroing the buffer really that performance critical to risk buffer overflows like this one?

Inconsistency between BytesMut and Vec impl of BufMut

This code works:

    let mut buf = Vec::new();

    for _ in 0..1000 {
        buf.put_u8(1);
    }

This code crashes:

    let mut buf = BytesMut::new();

    for _ in 0..1000 {
        buf.put_u8(1);
    }

I think behavior should be similar because of least surprise principle.

(Probably, BytesMut should implicitly reserve).

Consider making bytes_vec a required method for 0.5

It's arguably more low-level than the bytes method, and the bytes method could be implemented in terms of bytes_vec.

The downside, however, is that bytes_vec is more complicated to implement and may be more intimidating. It also requires importing IoVec which can be unfortunate.

Character escape function

I would love to have a character escape function for byte strings, in the manner of escape_default for Unicode strings.

RingBuf marks aren't cleared correctly on write

When writing to a RingBuf with a mark set, the conditions for clearing the mark are not correct. For example, the following test case should succeed but it panics due to the mark being cleared when it shouldn't be:

let mut buf = RingBuf::new(8);
buf.write(&[1, 2, 3, 4, 5, 6, 7]).unwrap();
buf.mark();
buf.write(&[8]).unwrap();
buf.reset();

let mut buf2 = [0; 8];
buf.read(&mut buf2).unwrap();
assert_eq!(buf2, [1, 2, 3, 4, 5, 6, 7, 8]);

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.