Bytes[Mut]::into_vec() about bytes HOT 16 CLOSED

The only issue I can think of is what happens when ref count is 1 but the Bytes handle doesn't point to the beginning of the vec? There is no way to represent that w/ Vec, so should the bytes be moved?

/cc @alexcrichton

from bytes.

@carllerche yes, I think it should move bytes. It will still be cheaper than copying to new Vec because of saved allocation.

from bytes.

Seems reasonable to me, but it should be clearly documented as "this is likely to allocate a new vector and copy bytes, it's not guaranteed to be a cheap operation.

from bytes.

from bytes.

try_into_vec is OK with me too. I can update patch if you like.

from bytes.

On a second though, try_into_vec is not good, because if successful it may do O(N) memmove which could be unexpected to users.

from bytes.

I'm not sure why you think it would do a memmove if successful? I was thinking that it would not...

So, try_into_vec would not succeed if the buf is backed by an Arc w/ only one reference BUT there is an offset...

from bytes.

In that case it is not possible to recover underlying vec when rc=1 and offset!=0, and I think it is suboptimal (unnecessary malloc/free).

BTW is there a use case for try_into_vec where Bytes from Err is used other way than to copy bytes into a new vec and discard Bytes?

from bytes.

One case that I can think of is "try to get the vector as is w/o any additional work, but if it is not possible, allocate a new vector 2x the size and copy the data into it".

from bytes.

Probably both into_vec and non-copying try_into_vec functions should be added.

from bytes.

It seems quite useful as a way to deal with the fact Bytes could be used with arbitrary allocators.

Alternatively I would send a patch to provide another kind of backing buffer and deal with it inside Bytes, @carllerche would that be acceptable for you?

from bytes.

@lu-zero I am not following how this related to arbitrary allocators.

Bytes doesn't really have the goal of being generic or using trait objects of any sort internally, the Buf / BufMut traits are for that. Maybe you should sketch out your thoughts for alternate backing buffers first?

from bytes.

Right now I'm using Bytes + a quite relatively bad hack to implement a multimedia FrameBuffer.

A Frame is more or less a bunch of sort of related buffers (different audio channels, video planes), some time information and other optional ancillary information/metadata.

Some hardware acceleration interfaces and simd implementations require to have a single allocation for all the planes AND have the memory aligned to some relatively higher-than-natural value (e.g. 32bytes), so I use the nightly allocator to get the memory aligned properly and feed Bytes with it by wrapping it to a intermediate Vec.

The bad hack relies on the fact right now the nightly allocator doesn't have problems dropping the memory so everything works, but nothing would prevent that in the future the actual dealloc function must be called with the correct layout.

The least work path is just to have a mean to unwrap away the raw memory and dealloc it using a custom Drop .

Since Bytes should be used for networking code you might end up having the same issue once you have to deal with hardware accelerators as well, so adding a way to allocate Bytes/BytesMut directly using the heap api might be useful for more that my use-case.

from bytes.

@lu-zero thanks for providing the feedback. I think I understand the problem.

The proposed api would not solve the problem you are having. It is not always possible to get the underlying memory due to the ref counting.

In general, this crate is targetting stable rust. As such, a built in allocator is assumed. Whenever custom allocators are stable, we can reevaluate the situation.

from bytes.

Right now the problem is purely theoretical so there isn't any hurry and looks like the heap API will drop the Layout arg for dealloc making my life simpler :)

from bytes.

Closing due to inactivity. This probably cannot be done easily anymore due to the vtable strategy.

from bytes.