Git Product home page Git Product logo

presser's Introduction

๐Ÿ—œ presser

Utilities to help make copying data around into raw, possibly-uninitialized buffers easier and safer.

Embark Embark Crates.io Published Docs Git Docs dependency status

presser can help you when copying data into raw buffers. One primary use-case is copying data into graphics-api-allocated buffers which will then be accessed by the GPU. Common methods for doing this right now in Rust can often invoke UB in subtle and hard-to-see ways. For example, viewing an allocated but uninitialized buffer as an &mut [u8] is instantly undefined behavior*, and transmuteing even a T: Copy type which has any padding bytes in its layout as a &[u8] to be the source of a copy is also instantly undefined behavior, in both cases because it is invalid to create a reference to an invalid value (and uninitialized memory is an invalid u8), even if your code never actually accesses that memory. This immediately makes what seems like the most straightforward way to copy data into buffers unsound ๐Ÿ˜ฌ

* If you're currently thinking to yourself "bah! what's the issue? surely an uninit u8 is just any random bit pattern and that's fine we don't care," check out this blog post by @RalfJung, one of the people leading the effort to better define Rust's memory and execution model. As is explored in that blog post, anuninitpiece of memory is not simplyan arbitrary bit pattern*, it is a wholly separate state about a piece of memory, outside of its value, which lets the compiler perform optimizations that reorder, delete, and otherwise change the actual execution flow of your program in ways that cannot be described simply by "the value could havesomepossible bit pattern". LLVM and Clang are changing themselves to require special noundef attribute to perform many important optimizations that are otherwise unsound. For a concrete example of the sorts of problems this can cause, see this issue @scottmcm hit.*

A bad example

#[derive(Clone, Copy)]
#[repr(C)]
struct MyDataStruct {
    a: u8,
    b: u32,
}

let my_data = MyDataStruct { a: 0, b: 42 };

// ๐Ÿšจ MyDataStruct contains 3 padding bytes after `a`, which are
// uninit, therefore getting a slice that includes them is UB!
let my_data_bytes: &[u8] = transmute(&my_data);

// allocate an uninit buffer of some size
let my_buffer: MyBufferType = some_api.alloc_buffer_size(2048);

// ๐Ÿšจ this is UB for the same reason, these bytes are uninit!
let buffer_as_bytes: &mut [u8] =
    slice::from_raw_parts(my_buffer.ptr(), my_buffer.size());

// ๐Ÿšจ this is UB because not only are both slices invalid,
// this is not ensuring proper alignment!
buffer_as_bytes.copy_from_slice(my_data_bytes);

presser helps with this by allowing you to view raw allocated memory of some size as a "Slab" of memory and then provides safe, valid ways to copy data into that memory:

A good example

#[derive(Clone, Copy)]
#[repr(C)]
struct MyDataStruct {
    a: u8,
    b: u32,
}

let my_data = MyDataStruct { a: 0, b: 42 };

// allocate an uninit buffer of some size
let my_buffer: MyBufferType = some_api.alloc_buffer_size(2048);

// use `RawAllocation` helper to allow access to a presser `Slab`.
// alternatively, you could implement the `Slab` on your buffer type directly if that
// type is owned by your code!
let raw_allocation = presser::RawAllocation::from_raw_parts(my_buffer.ptr(), my_buffer.size());

// here we assert that we have exclusive access to the data in the buffer, and get the actual
// `Slab` to use to copy into.
let slab = unsafe { raw_allocation.borrow_as_slab(); }

// now we may safely copy `my_data` into `my_buffer`, starting at a minimum offset of 0 into the buffer
let copy_record = presser::copy_to_offset(&my_data, &mut slab, 0)?;

// note that due to alignment requirements of `my_data`, the *actual* start of the bytes of
// `my_data` may be placed at a different offset than requested. so, we check the returned
// `CopyRecord` to check the actual start offset of the copied data.
let actual_start_offset = copy_record.copy_start_offset;

Note that actually accessing the copied data is a completely separate issue which presser does not (as of now) concern itself with. BE CAREFUL!

See more in the git main docs or the released version docs.

Contribution

Contributor Covenant

We welcome community contributions to this project.

Please read our Contributor Guide for more information on how to get started. Please also read our Contributor Terms before you make any contributions.

Any contribution intentionally submitted for inclusion in an Embark Studios project, shall comply with the Rust standard licensing model (MIT OR Apache 2.0) and therefore be dual licensed as described below, without any additional terms or conditions:

License

This contribution is dual licensed under EITHER OF

at your option.

For clarity, "your" refers to Embark or any other licensee/user of the contribution.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.