Git Product home page Git Product logo

anymap's Introduction

AnyMap, a safe and convenient store for one value of each type

AnyMap is a type-safe wrapper around HashMap<TypeId, Box<dyn Any>> that lets you not worry about TypeId or downcasting, but just get on with storing one each of a bag of diverse types, which is really useful for extensibility in some sorts of libraries.

Background

If you’re familiar with Go and Go web frameworks, you may have come across the common “environment” pattern for storing data related to the request. It’s typically something like map[string]interface{} and is accessed with arbitrary strings which may clash and type assertions which are a little unwieldy and must be used very carefully. (Personally I would consider that it is just asking for things to blow up in your face.) In a language like Go, lacking in generics, this is the best that can be done; such a thing cannot possibly be made safe without generics.

As another example of such an interface, JavaScript objects are exactly the same—a mapping of string keys to arbitrary values. (There it is actually more dangerous, because methods and fields/attributes/properties are on the same plane—though it’s possible to use Map these days.)

Fortunately, we can do better than these things in Rust. Our type system is quite equal to easy, robust expression of such problems.

Example

let mut data = anymap::AnyMap::new();
assert_eq!(data.get(), None::<&i32>);
data.insert(42i32);
assert_eq!(data.get(), Some(&42i32));
data.remove::<i32>();
assert_eq!(data.get::<i32>(), None);

#[derive(Clone, PartialEq, Debug)]
struct Foo {
    str: String,
}

assert_eq!(data.get::<Foo>(), None);
data.insert(Foo { str: format!("foo") });
assert_eq!(data.get(), Some(&Foo { str: format!("foo") }));
data.get_mut::<Foo>().map(|foo| foo.str.push('t'));
assert_eq!(&*data.get::<Foo>().unwrap().str, "foot");

Features

  • Store up to one value for each type in a bag.
  • Add Send or Send + Sync bounds.
  • You can opt into making the map Clone. (In theory you could add all kinds of other functionality, but you can’t readily make this work generically, and the bones of it are simple enough that it becomes better to make your own extension of Any and reimplement AnyMap.)
  • no_std if you like.

Cargo features/dependencies/usage

Typical Cargo.toml usage, providing anymap::AnyMap et al. backed by std::collections::HashMap:

[dependencies]
anymap = "1.0.0-beta.2"

No-std usage, providing anymap::hashbrown::AnyMap et al. (note the different path, required because Cargo features are additive) backed by alloc and the hashbrown crate:

[dependencies]
anymap = { version = "1.0.0-beta.2", default-features = false, features = ["hashbrown"] }

On stability: hashbrown is still pre-1.0.0 and experiencing breaking changes. Because it’s useful for a small fraction of users, I am retaining it, but with different compatibility guarantees to the typical SemVer ones. Where possible, I will just widen the range for new releases of hashbrown, but if an incompatible change occurs, I may drop support for older versions of hashbrown with a bump to the minor part of the anymap version number (e.g. 1.1.0, 1.2.0). Iff you’re using this feature, this is cause to consider using a tilde requirement like "~1.0" (or spell it out as >=1, <1.1).

Unsafe code in this library

This library uses a fair bit of unsafe code for several reasons:

  • To support CloneAny, unsafe code is currently required (because the downcast methods are defined on dyn Any rather than being trait methods, and upcasting is an incomplete feature in rustc); if you wanted to ditch Clone support this unsafety could be removed.

  • For dyn CloneAny + Send and dyn CloneAny + Send + Sync’s Clone implementation, an unsafe block is used to attach the auto traits where safe code used to be used, in order to avoid a spurious future-compatibility lint.

  • In the interests of performance, type ID checks are skipped as unnecessary because of the invariants of the data structure (though this does come at the cost of Map::{as_raw_mut, into_raw} being marked unsafe).

It is possible to remove all unsafe code at the cost of only CloneAny functionality and a little performance. The safe branch in the Git repository contains a couple of commits demonstrating the concept. It’s quite straightforward; the core of this library is very simple and perfectly safe.

Colophon

Authorship: Chris Morgan is the author and maintainer of this library.

Licensing: this library is distributed under the terms of the Blue Oak Model License 1.0.0, the MIT License and the Apache License, Version 2.0, at your choice. See COPYING for details.

anymap's People

Contributors

chris-morgan avatar drbawb avatar hellow554 avatar reem avatar tivek avatar tomassedovic avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

anymap's Issues

Rustc warning for future incompatibility due to where_clauses_object_safety

Rustc throws the following error:

warning: the trait `anymap::any::CloneToAny` cannot be made into an object
  |
  = note: `#[warn(where_clauses_object_safety)]` on by default
  = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
  = note: for more information, see issue #51443 <https://github.com/rust-lang/rust/issues/51443>
  = note: method `clone_to_any_send` references the `Self` type in where clauses

warning: the trait `anymap::any::CloneToAny` cannot be made into an object
  |                                                                                                                  
  = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!                                                                                                       
  = note: for more information, see issue #51443 <https://github.com/rust-lang/rust/issues/51443>                    
  = note: method `clone_to_any_sync` references the `Self` type in where clauses                                     
                                                                                                                     
warning: the trait `anymap::any::CloneToAny` cannot be made into an object
  |                                                                                                                  
  = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!                                                                                                       
  = note: for more information, see issue #51443 <https://github.com/rust-lang/rust/issues/51443>                    
  = note: method `clone_to_any_send_sync` references the `Self` type in where clauses

Any way to create a singleton of this?

for example lazy_static? or using ONCE?

Best I could do is

pub fn model_store() -> Arc<Mutex<AnyMap>> {
    static mut STORE: *const Arc<Mutex<anymap::AnyMap>> = 0 as *const Arc<Mutex<anymap::AnyMap>>;
    static ONCE: Once = Once::new();
    ONCE.call_once(|| {
        let singleton = AnyMap::new();
        unsafe { STORE = std::mem::transmute(Arc::new(Mutex::new(singleton)))};
    });
    unsafe { (*STORE).clone() } // Is this safe?
}

which is definitely unsafe?

Split off TypeIdHasher into a separate crate

TypeIdHasher is useful for other abstractions dealing with dynamic typing.

I have copy-pasted that code into a private project of mine, because I wanted to use it to implement another collection type that is similar to AnyMap.

The collection I implemented seems generally-useful. It might be a good idea for me to publish it as a crate of my own (I don't think it belongs in anymap, as it is different enough from the purpose of this crate). However, to do that, I would need to duplicate the TypeIdHasher code from anymap. If we split that off into a separate crate, we could both depend on it instead of duplicating code.

Why does `anymap::Entry` require a Clone bound?

anymap::Entry::or_insert and anymap::Entry::or_insert_with require a Clone bound on the element to be inserted.

It is unclear from both the docs and the source why that is the case, or if its intentional.

Unsound usages of unsafe implementation from `Self` to `T`

Hi, I am scanning the anymap in the latest version with my own static analyzer tool.

Unsafe conversion found at: src/any.rs

#[inline]
unsafe fn downcast_ref_unchecked<T: 'static>(&self) -> &T {
    &*(self as *const Self as *const T)
}

#[inline]
unsafe fn downcast_mut_unchecked<T: 'static>(&mut self) -> &mut T {
    &mut *(self as *mut Self as *mut T)
}

#[inline]
unsafe fn downcast_unchecked<T: 'static>(self: Box<Self>) -> Box<T> {
    Box::from_raw(Box::into_raw(self) as *mut T)
}

This unsound implementation would create a misalignment issues, if the generics type T is not properly handled like it's other random types.

This would potentially cause undefined behaviors in Rust. If we further manipulate the problematic converted types, it would potentially lead to different consequences. I am reporting this issue for your attention.

build fails under rust 1.0

jhaddad@jhaddad-rmbp15 ~/dev/anymap$ cargo test                                                                                                                           master
   Compiling anymap v0.10.3 (file:///Users/jhaddad/dev/anymap)
src/lib.rs:4:19: 4:33 error: unstable feature
src/lib.rs:4 #![cfg_attr(test, feature(test))]
                               ^~~~~~~~~~~~~~
note: this feature may not be used in the stable release channel
error: aborting due to previous error
Could not compile `anymap`.

Publish a new release

Hi, @chris-morgan could you or somebody else involved in this project please publish a new release on crates.io, or possibly hand over maintainership of this project to somebody that can publish new releases?

How to actually use this

I want to crate a Map<Component>, but whatever I do, Rust always says error[E0277]: the trait bound `Component + 'static: anymap::any::UncheckedAnyExt` is not satisfied.

I have tried so much now:

pub trait Component {
pub trait Component: std::any::Any {
pub trait Component: anymap::any::Any {
pub trait Component: anymap::any::Any + anymap::any::UncheckedAnyExt {
pub trait Component: 'static + anymap::any::Any + anymap::any::UncheckedAnyExt {
pub trait Component: 'static + std::any::Any + anymap::any::Any + anymap::any::UncheckedAnyExt {

I am lost on what to do...

Add ability to check for and retrieve a value by TypeId

I'm trying to write an entity component system for game development, and I've gotten to the point where I would like to start caching lists of components for a system, so my first thought was to have a component trait that returns a list of TypeIds which describes which component types are required. Then the idea is to iterate over them when it is time to update the list of components and retrieve the value of the AnyMap by the TypeId. Currently there doesn't seem to be a way to do that exposed in your library, but it doesn't look like it would be too hard, but I'm not confident in my Rust skills yet to make a pull request myself.

My request is for a set of methods like contains_typeid, get_by_typeid, etc...

Recent change in core::any::TypeId breaks the `type_id_hasher` unittest

Howdy,

The size of TypeId changed in rust-lang/rust@9e5573a, and the unittest type_id_hasher no longer builds:

cannot transmute between types of different sizes, or dependently-sized types
    |
638 |         assert_eq!(hasher.finish(), unsafe { core::mem::transmute::<TypeId, u64>(type_id) });
    |                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: source type: `TypeId` (128 bits)
    = note: target type: `u64` (64 bits)

It seems to me that the commit doesn't affect the behavior of the anymap crate, it only affects the unittest. Could you please confirm that that is the case?

Internal storage as Box<Any> prevents passing references between tasks

This is not a bug report, but a request for comments on a possibly new feature.

AnyMap internally stores data as boxed references to Any. Since Any does not carry Share and Send traits, the compiler does not allow access to the map from multiple tasks even though the original types of stored data should in principle allow it.

I have modified AnyMap by adding Send and Share bounds to Any and to type parameters of generic functions. The result is ConcurrentAnyMap, tivek/anymap@63f992f. It is safely accessible from more than just the creating task through immutable references, which makes it useful as a centralized, immutable data store. std::sync::Arc so far seems to be happy.

At this point I do not like the code duplication between AnyMap and ConcurrentAnyMap. Share+Send need to be added at several places, which is pretty intrusive. To me it seems deriving would not work (but, I have just started with Rust and maybe I'm missing something obvious).

Hope someone finds it useful, and I'd appreciate your comments.

Future compatibility warning ("CloneToAny" cannot be made into an object)

Compiling anymap on the current nightly results in this warning: rust-lang/rust#51443

warning: the trait `any::CloneToAny` cannot be made into an object                                                                                                                
  --> src/any.rs:15:5                                                                                                                                                             
   |                                                                                                                                                                              
15 |     fn clone_to_any_send(&self) -> Box<CloneAny + Send> where Self: Send;                                                                                                    
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                    
   |                                                                                                                                                                              
   = note: #[warn(where_clauses_object_safety)] on by default                                                                                                                     
   = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!                                              
   = note: for more information, see issue #51443 <https://github.com/rust-lang/rust/issues/51443>                                                                                
   = note: method `clone_to_any_send` references the `Self` type in where clauses                                                                                                 

(same warning for all clone_to_any_X methods)

`any` module is no longer public

In 1.0.0-beta.2, the any module is now private and only CloneAny is exported, meaning the Downcast trait is no longer visible.

Release anymap 0.13?

Looks like anymap could use a release so people can pick up the fix for #26, which is not in 0.12.

`anymap` stopping being compilable in future?

rust-lang/rust#51443

warning: the trait `any::CloneToAny` cannot be made into an object
  --> /mnt/src/git/anymap/src/any.rs:15:8
   |
15 |     fn clone_to_any_send(&self) -> Box<CloneAny + Send> where Self: Send;
   |        ^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(where_clauses_object_safety)]` on by default
   = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
   = note: for more information, see issue #51443 <https://github.com/rust-lang/rust/issues/51443>
   = note: method `clone_to_any_send` references the `Self` type in where clauses

Does anymap rely on a trick deemed unsound?

Can't make all nickel.rs

Could not compile anymap.

--- stderr
src/lib.rs:56:41: 56:44 error: explicit lifetime bound required
src/lib.rs:56 impl<'a> UncheckedAnyRefExt<'a> for &'a Any {
^~~
src/lib.rs:74:48: 74:51 error: explicit lifetime bound required
src/lib.rs:74 impl<'a> UncheckedAnyMutRefExt<'a> for &'a mut Any {
^~~
src/lib.rs:111:31: 111:34 error: explicit lifetime bound required
src/lib.rs:111 data: HashMap<TypeId, Box, TypeIdHasher>,
^~~
error: aborting due to 3 previous errors

[root@localhost nickel]# rustc -v
rustc 0.12.0-pre-nightly (5419b2ca2 2014-08-29 22:16:20 +0000)

concurrent feature

Just for the record, since we already discussed this a bit.

It would be nice if we created a concurrent feature that would basically add the Sync + Send bounds where necessary. I already do something similar in my minor fork and found that it's a very straightforward change compared to making it clonable.

Personally I really need this so that I could put stuff in an Arc and send it to a thread pool, but I understand if it's more of a hassle than it's worth here.

Specifically I need it to be both clonable and concurrent. What I was discussing on IRC was that I don't know if it'd be possible to have to independent yet composable features for each thing: clone and concurrent, given the fact (?) that the with_clone stuff would need to have the Sync + Send bounds added.

Future incompatability warnings

building anymap on nightly rust now triggers a future incompatibility warning, here's a copypasted terminal output

The package `anymap v1.0.0-beta.2` currently triggers the following future incompatibility lints:
> warning: adding an auto trait `Send` to a trait object in a pointer cast may cause UB later on
>    --> /home/draconium/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anymap-1.0.0-beta.2/src/any.rs:37:40
>     |
> 37  |                 unsafe { Box::from_raw(raw as *mut $t) }
>     |                                        ^^^^^^^^^^^^^^
> ...
> 144 | impl_clone!(dyn CloneAny + Send);
>     | -------------------------------- in this macro invocation
>     |
>     = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
>     = note: for more information, see issue #127323 <https://github.com/rust-lang/rust/issues/127323>
>     = note: this warning originates in the macro `impl_clone` (in Nightly builds, run with -Z macro-backtrace for more info)
>
> warning: adding auto traits `Send` and `Sync` to a trait object in a pointer cast may cause UB later on
>    --> /home/draconium/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anymap-1.0.0-beta.2/src/any.rs:37:40
>     |
> 37  |                 unsafe { Box::from_raw(raw as *mut $t) }
>     |                                        ^^^^^^^^^^^^^^
> ...
> 145 | impl_clone!(dyn CloneAny + Send + Sync);
>     | --------------------------------------- in this macro invocation
>     |
>     = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
>     = note: for more information, see issue #127323 <https://github.com/rust-lang/rust/issues/127323>
>     = note: this warning originates in the macro `impl_clone` (in Nightly builds, run with -Z macro-backtrace for more info)

use stable api

We should work towards getting this to build on the "stable" releases, e.g. beta. This means we have to remove our dependence on the std_misc and core features.

  • core seems to be needed for std::raw::TraitObject which is needed for downcasting functionality.
  • std_misc seems to be needed for std::collections::hash_state::HashState and Drain

Perhaps it might be worth a look at @reem's typemap which doesn't require any features. In the worst case, we can probably salvage those features behind some sort of "nightly" feature, or better yet each one of those features under a separate feature, like "downcast" and "drain".

Maintainership

Dear @chris-morgan,

I don't wish to be rude, but from the issue tracker e.g. #45 I wonder if you have had enough of maintaining this crate / real life has gotten in the way or whatever.

I have released a new version under the crate name anymap3 but I wonder whether you'd prefer for someone to look after the original crate / 'hand over' maintainership?

I noticed some users of the crate have moved to vendoring their own copy with the soundness issues fixed (and other crates have rolled their own internal ones), which overall seems bad for 'the community' as I can't just pull in their copy into my project ;-).

It's already a fairly 'complete' and maintenance-mode crate so if the idea interested you, I'd be happy to take over its keepership.

Thanks for writing this crate for us and saving us having to do it ourselves.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.