Why don't slices use dedicated metadata structs? about rust HOT 13 OPEN

@dead-claudia Not with offset nor with &[_], though. As mentioned in the offset docs and places like https://doc.rust-lang.org/nightly/std/ptr/#allocated-object, Rust says isize::MAX bytes at most, period. (No, the kernel isn't special here, nor is any target.)

from rust.

However, that's not possible because T must be sized

It is possible. There isn't really a difference between [T] and WrapsSlice<T>.

If <[T]>::Metadata is SliceMetadata<T>, then given struct WrapsSlice<T>([T]);, the WrapsSlice::<T>::Metadata is also SliceMetadata<T>. This is how metadata currently works for all structures -- metadata of the structure is the metadata of the last field.

Note that SliceMetadata::size_of will not necessarily return the size of the thing a reference with the given metadata points. Metadata only describes the unsized tail for structures:

struct A(u16, [u8]);

let a: &A = ...;
assert_eq!(a.1.len(), 2); // given length of the slice = 2
assert_eq!(metadata(a).size_of(), 2); // the byte size of the slice
assert_eq!(size_of_val(a), 4); // size of the whole structure

(this is the same what currently happens with DynMetadata)

from rust.

Oh, I see. I misunderstood the proposal, I thought T would be the whole type, not just the slice element. Never mind me then!

I think this looks good then. Will you be implementing it?

from rust.

• (The SliceMetadata itself essentially models a std::alloc::Layout with a static alignment, so this is not unlike storing layout info with the data.)

Note that that's only precisely true for references to slices. "Thanks" to https://doc.rust-lang.org/std/ptr/fn.slice_from_raw_parts.html, a *const [i32] can have usize::MAX as its metadata, even though that overflows usize (not to mention overflowing isize, which is what Layout checks).

from rust.

@Rua This issue is focused on slices in particular, and ergonomics surrounding them. That's why everything here is centered around slices.

What you're discussing here, I spelled out a bit in #123353 (closed as I thought I had more collected than I did, and I suspect you're in a similar boat). I suggest filing a new issue, linking to it from the main tracking issue #81513, and seeing where it takes you.

from rust.

Related to #81513

from rust.

I was thinking about this myself, and I agree that the metadata for slices should be a distinct type. Moreover, when there is a separate metadata type for each different kind of DST, then it becomes possible to distinguish DSTs by their type. You could specify that a type must be a slice DST, or a trait object DST, or whatever new DSTs people might add in the future.

from rust.

If "multiply-unsized" types ever become a thing, e.g. [dyn Trait] or [[T]], then there could be a SliceMetadata<T>::inner method returning <T as Pointee>::Metadata.

Also, part of the idea here is to replace slice::from_raw_parts{,_mut} and ptr::slice_from_raw_parts{,_mut} with a less error-prone alternative similarly to how mem::MaybeUninit replaced mem::uninit.

I don't think SliceMetadata and {from,into}_raw_parts would necessarily replace these functions; IMO the "create a slice" and "create a slice pointer" sections don't look any safer/more-"correct" in the "new" version, they just look a bit less concise. The "cast a slice" srctions do look better though, assuming "keep the byte size the same or fail" is the goal (and if the goal is "keep the same slice length", just a normal cast works).

from rust.

@zachs18 To be clear:

• Creation of slices I was just thinking it could help make the pointer construction a little more explicit. (The SliceMetadata itself essentially models a std::alloc::Layout with a static alignment, so this is not unlike storing layout info with the data.)
• The transmuting between slice types is where the safety issues really come into play. You could fix that with a std::mem::transmute_slice(&[T]) -> &[U] method (plus mutable variant), but using metadata would be simpler IMO.

from rust.

@scottmcm There's also 32-bit x86 with PAE, which extends the address space beyond what a simple pointer can address. It is possible in kernel land to access those beyond-max-pointer offsets.

from rust.

Should the impl<T> not be impl<T: ?Sized> at least for some of the methods? Certainly new and len work as-is with unsized types. Then again, your current implementation of size_of assumes that T is the slice element and not the slice itself.

So what about slice-based custom DSTs? It would definitely be valuable to have a size_of method for those too, but I don't know how it would be implemented. In parallel with DynMetadata maybe there should also be align_of and layout.

EDIT: Another thought: since we already have #69835, it may even be useful for the size_of, align_of and layout methods to be placed on a new trait. This trait, in turn, can then be used by other future APIs. It could also be desirable for the metadata of sized types to be a zero-sized newtype (SizedMetadata? EmptyMetadata? NoneMetadata?) instead of simply (), so that methods and traits can be added to it as well.

from rust.

Slices and slice-based DSTs use the same metadata. That means that if you call ptr::metadata on either of them, they are both going to return a SliceMetadata<T> under this proposal. However, that's not possible because T must be sized. The method implementations can be left to another discussion, but the fundamental metadata type must be compatible with both.

from rust.

A method that could be useful to add for SliceMetadata is this one:

pub fn is_valid(&self) -> bool

It would check if the metadata follows the requirements for functions like mem::size_of_val_raw, Layout::for_value_raw, Layout::array, and indeed SliceMetadata::size_of (which can overflow as currently implemented).

from rust.