tokio-rs / bytes Goto Github PK
View Code? Open in Web Editor NEWUtilities for working with bytes
License: MIT License
Utilities for working with bytes
License: MIT License
It would be convenient to have Bytes::into_vec()
and BytesMut::into_vec()
operations.
These operations should return owned Vec
if KIND_VEC
or KIND_ARC
and refcount is 1.
It would be convenient to have something like MutByteBuf::mut_bytes() that gives access to the already written part of the byte buffer instead of the remaining memory area. This would allow to modify something that was received from the network, and potentially forwarding it to some other socket, without having to copy everything.
There's PR #88 which implements it.
With patch applied, benchmark becomes measurably (about 5%) faster.
Same optimization is implemented in libc++:
void
__shared_weak_count::__release_weak() _NOEXCEPT
{
// NOTE: The acquire load here is an optimization of the very
// common case where a shared pointer is being destructed while
// having no other contended references.
//
// BENEFIT: We avoid expensive atomic stores like XADD and STREX
// in a common case. Those instructions are slow and do nasty
// things to caches.
//
// IS THIS SAFE? Yes. During weak destruction, if we see that we
// are the last reference, we know that no-one else is accessing
// us. If someone were accessing us, then they would be doing so
// while the last shared / weak_ptr was being destructed, and
// that's undefined anyway.
//
// If we see anything other than a 0, then we have possible
// contention, and need to use an atomicrmw primitive.
// The same arguments don't apply for increment, where it is legal
// (though inadvisable) to share shared_ptr references between
// threads, and have them all get copied at once. The argument
// also doesn't apply for __release_shared, because an outstanding
// weak_ptr::lock() could read / modify the shared count.
if (__libcpp_atomic_load(&__shared_weak_owners_, _AO_Acquire) == 0)
{
// no need to do this store, because we are about
// to destroy everything.
//__libcpp_atomic_store(&__shared_weak_owners_, -1, _AO_Release);
__on_zero_shared_weak();
}
else if (__libcpp_atomic_refcount_decrement(__shared_weak_owners_) == -1)
__on_zero_shared_weak();
}
The build currently fails on the latest nightly because of rust-lang/rust@d131f33
The full compiler error:
src/alloc/mod.rs:67:18: 67:32 error: mutating transmuted &mut T from &T may cause undefined behavior,consider instead using an UnsafeCell, #[deny(mutable_transmutes)] on by default
src/alloc/mod.rs:67 unsafe { mem::transmute(self.bytes()) }
^~~~~~~~~~~~~~
src/str/bytes.rs:117:18: 117:32 error: mutating transmuted &mut T from &T may cause undefined behavior,consider instead using an UnsafeCell, #[deny(mutable_transmutes)] on by default
src/str/bytes.rs:117 unsafe { mem::transmute(self.obj()) }
^~~~~~~~~~~~~~
error: aborting due to 2 previous errors
I'd fix it myself, but I probably shouldn't try to fix unsafe code that I don't fully understand.
Could be useful?
Buffer overflow in impl<'a> Sink for &'a mut Vec<u8>
let mut sink: Vec<u8> = Vec::new();
sink.reserve(16);
println!("Capacity: {}", sink.capacity());
let mut source = Cursor::new(b"0123456789abcdef0123456789abcdef");
sink.copy_from(&mut source);
println!("Length: {}, Capacity: {}", sink.len(), sink.capacity());
prints:
Capacity: 16
Length: 32, Capacity: 16
The error is in the call to reserve: you don't reserve "more capacity", instead you are only guaranteed there is enough space to insert the requested number of elements, i.e. you need to base it on length instead of capacity.
Is there a really good reason why you need unsafe
here?
self.resize(buf.remaining(), 0);
buf.read_slice(&mut self[..]);
Is not zeroing the buffer really that performance critical to risk buffer overflows like this one?
Currently, it is ~2x slower. A cursory investigation reveals that there are way too many is_inline
checks. Removing them brings the impls back on par.
I see that Chain<T, U>
implements Buf
, and that its bytes
method calls the bytes
method of the first input if it isn't fully consumed, and otherwise of the second input. That means that bytes
never return a slice to the complete input when the first one isn't empty yet.
Is that a bug? Is that intended? The documentation of the bytes
method mislead me thinking this was illegal.
Buf
could have copy_to_bytes
operation:
fn copy_to_bytes(&mut self, len: usize) -> Bytes;
Default implementation should simply allocate new Bytes
, while implementation for Bytes
could call slice
.
Great work on the library, super useful. I was wondering if you would consider reducing the Buffer struct to 24bytes (64bits) (by using 32bit len/cap or something else)?
This will reduce the inline capacity but 23 is still plenty in practice. Even if moving single Buffers around is essentially the same speed (2x128bit moves vs 1x128 and 1x64bit) it really helps when you have more than a single buffer in a struct, enum, array, etc..
Apparently right now you need to use the internal AtomicPtr
even though the implementation might never see multiple threads. You should be able to statically opt out of thread safety somehow.
See http://burntsushi.net/rustdoc/quickcheck/trait.Arbitrary.html for a description.
Quickcheck is a randomized testing tool that lets you check general properties. To make your type part of the Quickcheck ecosystem, you need to implement Arbitrary, which involves two methods:
(1) generate a random input
(2) shrink a failing input
This should be relatively easy -- probably just using the Vec
code underneath.
This can be behind a feature gate to make sure that if you aren't using quickcheck otherwise, you don't have to pull it in.
Implement Bytes::concat(Bytes, Bytes)
.
It can do two optimizations beyond just allocating a new buffer and copying data there:
I think this may not wish to use io::Cursor
pervasively. This use case doesn't have much to do with std::io
. The std version also stores a u64
which isn't needed for bytes
's use case I believe.
I would love to have a character escape function for byte strings, in the manner of escape_default
for Unicode strings.
When writing to a RingBuf
with a mark set, the conditions for clearing the mark are not correct. For example, the following test case should succeed but it panics due to the mark being cleared when it shouldn't be:
let mut buf = RingBuf::new(8);
buf.write(&[1, 2, 3, 4, 5, 6, 7]).unwrap();
buf.mark();
buf.write(&[8]).unwrap();
buf.reset();
let mut buf2 = [0; 8];
buf.read(&mut buf2).unwrap();
assert_eq!(buf2, [1, 2, 3, 4, 5, 6, 7, 8]);
It would be nice to be able to do a realloc on the RingBuf to grow it in size. A deep copy is necessary today to accomplish this.
p.s. I might just submit a PR :)
It would be really cool if you could just use slicing syntax on a Buf and get a Buf again, like let b: Bytes = Bytes::from(&b"12323456"[..])[0..2];
. That can be done via implementing the Index/IndexMut trait for ranges.
Right now you have the slice function, that's quite ugly.
extern crate bytes;
use bytes::Bytes;
fn main() {
let mut a = Bytes::from_static(b"ab");
let b = a.split_off(1);
println!("{:?}, {:?}", a, b);
}
In a debug build, this code results in the following:
thread 'main' panicked at 'assertion failed: self.is_shared()', /Users/foo/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.4.1/src/bytes.rs:1510
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
0: bytes::bytes::Inner::set_start
1: bytes::bytes::Inner::split_off
2: bytes::bytes::Bytes::split_off
3: bytes_test::main
A release build works fine:
[97], [98]
Would be convenient if bytes
trait had a thin Chars
wrapper around Bytes
.
Chars
should be identical to Bytes
except it should guarantee that it always contains valid UTF-8 sequence.
Chars
to Bytes
is the same as String
to Vec<u8>
.
It doesn’t compile on 32 bit platforms.
src/alloc.rs:5:32: 5:39 error: bitshift exceeds the type's number of bits, #[deny(exceeding_bitshifts)] on by default
src/alloc.rs:5 const MAX_ALLOC_SIZE: usize = (1 << 32) - 1;
^~~~~~~
The docs for BufMut::bytes_mut
specifically allow for the method to return an empty slice . This should be changed so that the method may only return a 0-length slice if the buffer is exhausted. Otherwise, it's extremely easy to trigger infinite loops (for instance, the current implementation of BufMut::put_slice
). I believe the stdlib Write
trait puts similar restrictions on 0-length writes for similar reasons.
Likewise, BufMut::bytes_vec_mut
should not return a 0-length vector unless the buffer is exhausted, and probably shouldn't return 0-length IoVec
s ever.
% cargo test
Compiling bytes v0.4.1 (file:///Users/yozh/devel/left/bytes)
error[E0502]: cannot borrow `buf` as mutable because it is also borrowed as immutable
--> tests/test_chain.rs:64:5
|
58 | assert_eq!(2, buf.bytes_vec(&mut iovecs));
| --- immutable borrow occurs here
...
64 | buf.advance(2);
| ^^^ mutable borrow occurs here
...
93 | }
| - immutable borrow ends here
error[E0502]: cannot borrow `buf` as mutable because it is also borrowed as immutable
--> tests/test_chain.rs:74:5
|
58 | assert_eq!(2, buf.bytes_vec(&mut iovecs));
| --- immutable borrow occurs here
...
74 | buf.advance(3);
| ^^^ mutable borrow occurs here
...
93 | }
| - immutable borrow ends here
error[E0502]: cannot borrow `buf` as mutable because it is also borrowed as immutable
--> tests/test_chain.rs:84:5
|
58 | assert_eq!(2, buf.bytes_vec(&mut iovecs));
| --- immutable borrow occurs here
...
84 | buf.advance(3);
| ^^^ mutable borrow occurs here
...
93 | }
| - immutable borrow ends here
error: aborting due to 3 previous errors
error: Could not compile `bytes`.
To learn more, run the command again with --verbose.
% rustc --version
rustc 1.17.0-beta.1 (408d49e60 2017-03-14)
Probably, it's compiler bug.
thread 'main' panicked at 'assertion failed: end <= self.cap', /Users/raviteja/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.4.2/src/bytes.rs:1607
stack backtrace:
1: 0x107c204fc - std::sys::imp::backtrace::tracing::imp::write::h21ca2762819c7ae8
2: 0x107c225ae - std::panicking::default_hook::{{closure}}::h38f99a37d00bb19b
3: 0x107c22250 - std::panicking::default_hook::ha2186ee24b50729c
4: 0x107c22a67 - std::panicking::rust_panic_with_hook::h979db19ee91d2a53
5: 0x107be73d3 - std::panicking::begin_panic::h55f38a455bc31745
6: 0x107bea488 - bytes::bytes::Inner::set_end::h82befb8e78b27d8f
7: 0x107bea0c7 - bytes::bytes::Inner::split_to::hdf30edda54ab41e6
8: 0x107be9c3b - bytes::bytes::BytesMut::split_to::h58c00b1381d01291
9: 0x107af9bf1 - <rumqttd::codec::MqttCodec as tokio_io::framed_read::Decoder>::decode::hc5c6fc4b82691933
10: 0x107af45e2 - <tokio_io::framed::Fuse<T, U> as tokio_io::framed_read::Decoder>::decode::hf27d43a9a8b23015
11: 0x107af6c8b - <tokio_io::framed_write::FramedWrite2<T> as tokio_io::framed_read::Decoder>::decode::h8e63524e3ad209a5
12: 0x107af1543 - <tokio_io::framed_read::FramedRead2<T> as futures::stream::Stream>::poll::had597800fa87b314
13: 0x107af055b - <tokio_io::framed::Framed<T, U> as futures::stream::Stream>::poll::h0add0986ba64e75f
14: 0x107af1fb2 - <futures::stream::split::SplitStream<S> as futures::stream::Stream>::poll::h3fd48e3dce02280d
15: 0x107a6b8f7 - <futures::stream::for_each::ForEach<S, F, U> as futures::future::Future>::poll::hfe201db90d0693d7
16: 0x107ad61ed - <futures::future::chain::Chain<A, B, C>>::poll::hb70001641cce7c49
17: 0x107af813c - <futures::future::then::Then<A, B, F> as futures::future::Future>::poll::h96d61bd1db456977
18: 0x107b2d3c1 - <alloc::boxed::Box<F> as futures::future::Future>::poll::ha15f7165eb975205
19: 0x107b17f8c - <futures::task_impl::Spawn<F>>::poll_future::{{closure}}::h7fa62e61d191c22c
20: 0x107b1847e - <futures::task_impl::Spawn<T>>::enter::{{closure}}::he00427d7d77b3a1b
21: 0x107b2f77a - futures::task_impl::set::{{closure}}::h9e1b772b2c99b467
22: 0x107b1a211 - <std::thread::local::LocalKey<T>>::with::hf1203554760ba1fd
23: 0x107b2f633 - futures::task_impl::set::h261628f58d78254f
24: 0x107b18332 - <futures::task_impl::Spawn<T>>::enter::hf7dd52e83ef51739
25: 0x107b17f0f - <futures::task_impl::Spawn<F>>::poll_future::h0f85f03fa28897d6
26: 0x107b3cab6 - tokio_core::reactor::Core::dispatch_task::{{closure}}::h0acc6ca4c4f4174b
27: 0x107b141b4 - <scoped_tls::ScopedKey<T>>::set::h7f15a903853429bc
28: 0x107b3c4c5 - tokio_core::reactor::Core::dispatch_task::hc3fd8d6531a32c3c
29: 0x107b3b765 - tokio_core::reactor::Core::dispatch::ha135ccae5690bfb4
30: 0x107b3b2d9 - tokio_core::reactor::Core::poll::h76e14e964798dba8
31: 0x107a6c69e - tokio_core::reactor::Core::run::hdb4568c67ea0558d
32: 0x107aff980 - rumqttd::main::h380ac17aef49e4e9
33: 0x107c239da - __rust_maybe_catch_panic
34: 0x107c22e36 - std::rt::lang_start::hfc9882558f9403bf
35: 0x107b02609 - main
here is my decode for reference
fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<Packet>> {
// NOTE: `decode` might be called with `buf.len == 0` when prevous
// decode call read all the bytes in the stream. We should return
// Ok(None) in those cases or else the `read` call will return
// Ok(0) => translated to UnexpectedEOF by `byteorder` crate.
// `read` call Ok(0) happens when buffer specified was 0 bytes in len
// https://doc.rust-lang.org/std/io/trait.Read.html#tymethod.read
if buf.len() == 0 {
return Ok(None);
}
let (packet, len) = {
let mut buf_ref = buf.as_ref();
match buf_ref.read_packet_with_len() {
Err(e) => {
if let mqtt3::Error::Io(e) = e {
match e.kind() {
ErrorKind::TimedOut | ErrorKind::WouldBlock => return Ok(None),
_ => return Err(io::Error::new(e.kind(), e.description())),
}
} else {
return Err(io::Error::new(ErrorKind::Other, e.description()));
}
}
Ok(v) => v,
}
};
buf.split_to(len);
// println!("{:?}, {:?}, {:?}", len, packet, buf.len());
Ok(Some(packet))
}
In some networking scenarios, a growable ring buffer is more appropriate than a growable vector. In particular, it can do drain
without a memmove
.
The standard library has an implementation, VecDeque. I wonder if it would be possible (or makes sense at all) to implement BufMut
for it, similarly to how it's implemented for Vec
?
Copied from tokio-rs/tokio-io#28
I'm currently using the codec module to implement a framed tansport.
Before the refactoring, the encode-method of the old Codec trait received a Vec as parameter to serialize the message into. Vec directly implements std::io::Write which makes it very easy to put any kind of data into it, since most implementations have some way of writing into an instance of Write.
Now the Vec has been replaced by BytesMut, which lacks a direct implementation of that trait. Write-compatibility is only archievable through into_writer, which consumes the BytesMut. Consuming is not possible, though, because the method only receives the BytesMut by &mut.
Is this desired behavior?
From a caller perspective I think this is more ergonomic, but from an implementation perspective it's a bit more of a PITA because of more lifetimes. We should see what usage arises during 0.4 and see if we'd benefit much from this change.
cc #60
Currently, Bytes
represents an immutable, potentially shared, view into a memory region. BytesMut
represents a unique view into a memory region, which means that the memory can be mutated.
Perhaps there is a way to collapse these two types into a single type.
Pros
Cons
An std::iter::ExactSizeIterator is "an iterator that knows its exact length".
Implementing ExactSizeIterator for Iter<T: Buf> is trivial because it already returns the exact size in its implementation of size_hint()--it just needs an empty trait impl.
In both cases Ok(0) is returned, making it impossible to distinguish between a closed connection and a full buffer ( !buf.has_remaining() )
https://github.com/carllerche/bytes/blob/f693e038d9ad3042c0a68bb3ef609117b1d7a86a/src/buf/mod.rs#L439
This is really a question more than an issue:
Could we remove the RingBuf Reader and Writer newtypes (and implement Buf
and MutBuf
directly on RingBuf
)? I don't actually understand what purpose they have, apart from disambiguating method names from Buf
and MutBuf
(but that could be done in other ways).
If we could get rid of them, that would massively improve the ergonomics of RingBuf
, at least for my use case. There are two problems that I have with them:
let r = buf.reader(); let s = r.bytes();
where I would like to write let s = buf.bytes()
instead. (Writing let s = buf.reader().bytes()
doesn't work because the returned reader would not live long enough). This is admittedly only a minor annoyance.RingBufReader
that I have to create in that method. This is a bigger problem than the first one, since I don't think I can work around it.I'd be interested in an explanation of the necessity of the newtypes or any thoughts you have on the subject. If you think we could get rid of them, I'd be happy to do the implementation work.
I'd expect Buf for Bytes
and BufMut for BytesMut
in terms of what the name implies, but unfortunately there's no Buf for Bytes
Maybe it is better to have BufExt/MutBufExt trait instead of cluttering Buf/MutBuf trait but it will be good to support these for performance reasons.
It'd be nice if Bytes::from(b"foo bar")
would just grab the static pointer, and not copy the bytes.
This would probably requiring changing the representation slightly of Bytes
since it couldn't be a BytesMut
of a static memory pointer. Additionally, the try_mut
and into_mut
methods would not be allowed to return a BytesMut
in that case. Would be very useful for hyperium/hyper#953
It's arguably more low-level than the bytes
method, and the bytes method could be implemented in terms of bytes_vec.
The downside, however, is that bytes_vec is more complicated to implement and may be more intimidating. It also requires importing IoVec
which can be unfortunate.
Bytes
can be transformed into Bytes<'a>
.
Bytes<'a>
is identical to current Bytes
, except when KIND_STATIC
, ptr
lifetime is 'a
, but not 'static
'. So Bytes<'static>
is identical to current Bytes
.
Suppose you have Bytes
object. And you want, for example, split that Bytes
into several slices and pass these objects somewhere else, and you keep original Bytes
object. For example, you want to create a sequence of
struct MyProtocolFrame {
header: Header,
payload: Bytes,
}
and pass these objects down to serialization. You cannot change payload
type to &'a [u8]
, because sometimes MyProtocolFrame
must own data (in different part of project), and sometimes it could simply hold borrowed data.
Currently you can "borrow" with Bytes::slice
operation, which is expensive, because it:
Bytes
to KIND_ARC
when Bytes
is KIND_VEC
If Bytes
is parameterized with lifetime, Bytes<'a>
could have another cheap operation like:
impl Bytes {
fn slice_ref<'a>(&'a self, from: usize, to: usize) -> Bytes<'a> {
Bytes::from_borrowed(self.as_ref()[from..to])
}
}
And as a little bonus, you can now create a borrowed Bytes
object from &[u8]
which is obtained from some library which doesn't used Bytes
. Sometimes that's convenient.
The major problem with this design in ergonomics: users would have to write lifetimes a lot, and get lots of compiler errors, which is annoying.
I can think of a couple of ways to avoid this problem.
First, declare a type alias:
// Flexible type for advanced use cases
pub struct BytesMaybeBorrowed<'a> { ... }
// Identical to current Bytes
pub type Bytes = BytesMaybeBorrowed<'static>;
or
pub struct BytesMaybeBorrowed<'a> { ... }
pub struct Bytes(pub BytesMaybeBorrowed<'static>);
Second version seems to be easier to use, but it requires duplication of lots of functions. Which is fine by me.
What do you think?
Using the advance fn could result in reading uninitialized bytes.
For instance, ByteBuf
certainly looks like it's trying to be publicly documented, but it's not showing up in the rustdoc. All you see is a list of re-exports from private modules.
The latest version of bytes
on crates.io (v0.2.8) does not appear to include this commit which looks like it fixes the build error I'm seeing from Cargo:
/home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.2.8/src/alloc/mod.rs:67:18: 67:32 error: mutating transmuted &mut T from &T may cause undefined behavior,consider instead using an UnsafeCell, #[deny(mutable_transmutes)] on by default
/home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.2.8/src/alloc/mod.rs:67 unsafe { mem::transmute(self.bytes()) }
^~~~~~~~~~~~~~
/home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.2.8/src/str/bytes.rs:117:18: 117:32 error: mutating transmuted &mut T from &T may cause undefined behavior,consider instead using an UnsafeCell, #[deny(mutable_transmutes)] on by default
/home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/bytes-0.2.8/src/str/bytes.rs:117 unsafe { mem::transmute(self.obj()) }
Currently bytes crate have a logic of preserving "original capacity" during split.
So
drop(bytes.take());
// `bytes` here will have original capacity preserved if original capacity was <= 16K
I'm not sure if this logic is actually useful, and it may be confusing (e. g. you got a 100 bytes buffer, you called reserve(1)
and got 64K allocated).
I'd simply remove it (even without implementing truncate_front
).
On the other hand, without "original capacity" field it would be possible to implement cheap truncate_front
operation.
truncate_front
is the same as truncate
, but truncates from front, not from back.
truncate_front
is implemented trivally for KIND_ARC
, KIND_STATIC
and KIND_INLINE
.
And for KIND_VEC
truncate_front
can be implementing by changing arc
field of Inner
struct to store a pointer to original Vec
.
So for KIND_VEC
struct Inner {
arc: AtomicPtr<Shared>, // vec.as_mut()
ptr: *mut u8, // position to current view
len: usize, // len of current view
cap: usize, // capacity to current view
}
So to reconstruct Vec
from Inner
(e. g. to convert to Arc
) you could write something like:
fn reconstruct_vec(Inner inner) {
debug_assert!(inner.kind() == KIND_VEC);
let offset = inner.ptr - inner.arc;
Vec::from_raw_parts(inner.arc, inner.len + offset, inner.cap + offset)
}
I believe there are lot of sitations when you need to take a Bytes
and simply truncate front and back. For example, if you parse HTTP/2 frame, you can take Bytes
by value, and truncate headers and padding to retrieve payload as Bytes
. If kind == KIND_VEC
this operation will be cheaper than currently with split_off
, because Bytes
won't need to be converted to KIND_ARC
.
[stewart@rivergod:~/dev/fractalide/nixcrates/nixcrates]$ nix-build -A allCrates.bytes
these derivations will be built:
/nix/store/knncqp6lagbsw1ilbrvj163big9v8j2q-bytes.drv
building path(s) ‘/nix/store/m4wc5x700z5raw723w4h42v9k8lwsskm-bytes’
unpacking sources
bytes-0.3.0/Cargo.toml
bytes-0.3.0/LICENSE
bytes-0.3.0/README.md
bytes-0.3.0/src/alloc/heap.rs
bytes-0.3.0/src/alloc/mod.rs
bytes-0.3.0/src/buf/byte.rs
bytes-0.3.0/src/buf/mod.rs
bytes-0.3.0/src/buf/ring.rs
bytes-0.3.0/src/buf/sink.rs
bytes-0.3.0/src/buf/slice.rs
bytes-0.3.0/src/buf/source.rs
bytes-0.3.0/src/buf/take.rs
bytes-0.3.0/src/lib.rs
bytes-0.3.0/src/str/bytes.rs
bytes-0.3.0/src/str/mod.rs
bytes-0.3.0/src/str/rope.rs
bytes-0.3.0/src/str/seq.rs
bytes-0.3.0/src/str/small.rs
bytes-0.3.0/TODO.md
patching sources
configuring
no configure script, doing nothing
building
bytes -
namefix bytes
name bytes
About to use rustc to compile some lib - bytes
error: unused import: `ByteStr`
--> src/buf/byte.rs:2:38
|
2 | use traits::{Buf, MutBuf, MutBufExt, ByteStr};
| ^^^^^^^
|
note: lint level defined here
--> src/lib.rs:2:9
|
2 | #![deny(warnings)]
| ^^^^^^^^
error: unused import: `Any`
--> src/str/bytes.rs:4:16
|
4 | use std::any::{Any, TypeId};
| ^^^
error: unused import: `MutBuf`
--> src/str/small.rs:2:19
|
2 | use traits::{Buf, MutBuf, ByteStr, ToBytes};
| ^^^^^^
error: aborting due to 3 previous errors
builder for ‘/nix/store/knncqp6lagbsw1ilbrvj163big9v8j2q-bytes.drv’ failed with exit code 101
error: build of ‘/nix/store/knncqp6lagbsw1ilbrvj163big9v8j2q-bytes.drv’ failed
bytes_vec
do when remaining == 0Probably guaranteed to return 0
.
bytes_vec
be an empty slice?Yes
remaining
Maybe panic?
It's possible I'm missunderstanding something, but this seems to be a bug: most calls to bytes.reserve(a)
on a buffer with capacity b
result in the buffer having a capacity smaller than a + b
.
extern crate bytes;
use bytes::*;
#[test]
fn bytes_reserve() {
let mut bytes = BytesMut::with_capacity(20);
bytes.reserve(12);
bytes.put(vec![0; 32]);
let mut bytes = BytesMut::with_capacity(20);
bytes.reserve(44);
bytes.put(vec![0; 64]);
}
Both calls to bytes.put
panic:
thread 'bytes_reserve' panicked at 'assertion failed: self.remaining_mut() >= src.remaining()', /home/xal/.cargo/git/checkouts/bytes-72cf269819849e12/37f6cab/src/buf/buf_mut.rs:229```
The current implementation exposes uninitialized memory via mut_bytes
to arbitrary implementors of the Read
trait. Unfortunately though arbitrary implementors don't guarantee that they won't read the uninitialized data.
Since it is possible to access uninitialized memory using these functions, they should be unsafe.
This code works:
let mut buf = Vec::new();
for _ in 0..1000 {
buf.put_u8(1);
}
This code crashes:
let mut buf = BytesMut::new();
for _ in 0..1000 {
buf.put_u8(1);
}
I think behavior should be similar because of least surprise principle.
(Probably, BytesMut
should implicitly reserve).
Could you make a new bytes release and bump mio's bytes dependency? The buffers that mio re-exports still exhibit the bugs fixed in c454649
Calling RingBuf::reset
on a full buffer causes the buffer to become empty, due to the recalculation of len
from pos
and mark
, as demonstrated by this test case:
let mut buf = RingBuf::new(8);
buf.write(&[1, 2, 3, 4, 5, 6, 7, 8]).unwrap();
assert_eq!(MutBuf::remaining(&buf), 0);
buf.mark();
buf.reset();
assert_eq!(MutBuf::remaining(&buf), 0);
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.