hankjordan / bevy_save Goto Github PK

It took me a bit of digging to find the default save location on my machine was .local/share. It would be nice if the README listed the different default locations for each platform.

Previously, I had this code:

if let Ok(applier) = world.deserialize_applier(&mut serde) {
	return applier
		// Prefer these values because they're the Bevy default (rather than bevy_save),
		// and because they're necessary to be able to call `clear_empty()` on the snapshot serializer.
		// `clear_empty()` is useful because view-related entities don't need to have empty entries persisted, but
		// with the default AppDespawnMode setting - it would result in Window getting despawned.
		.despawn(DespawnMode::None)
		.mapping(MappingMode::Strict)
		.apply()
		.map_err(|e| {
			error!("Deserialization error: {:?}", e);
		});
}

It wasn't immediately clear to me if these settings are no longer supported and/or needed in the latest version.

My assumption is that they are no longer needed because it seems like bevy_save is working more closely in unison with core bevy logic, but I wanted to confirm.

Thanks

Would save a significant amount of space for rollback-heavy games

When saving and loading a large world, users may choose to do so in parts.
Like Applier, there should be a Builder that allows users to customize how a snapshot is taken from the World.

bevy_scene has a lot of other bevy dependencies, such as bevy_render. I want bevy_save to work in games that only use the minimal plugin set.

thanks.

I found a bug with mapping references to entity ids. If you delete the entities in your world before loading, then re-load, the world will contain invalid entity IDs where there was an entity pointer. It happens with Parent components and custom components with an Entity property implementing MapEntities.

After loading the new Parent component points to a non-existent entity.

I've included an example to reproduce below.

What else I have tried

When it first happend with Parent/Children, I tried implementing MapEntities in a component to hold the relationship instead. The same issue occurred, after loading it held an invalid ID.

I noticed that my component's map_entities function was never called. I put a println in it's map_entities and never saw it in the console.

EDIT: with MappingMode::Strict, the below example crashes after load from a heirarchy assertion in bevy.

What I want to try next, but haven't tried yet

Next, I'll try to debug print the EntityMap so I can see what entities it contains after loading, as well as the code that calls the ReflectMapEntities functions to see what's happening.

What else I considered

moonshine_save uses a custom resource to expose the entity id mappings after load, and components can implement a FromLoaded to set their ID's after load.

I'm not too sure about how MapEntities works, but if I couldn't get it working I was considering implementing a similar thing here.

Example

Modified examples/main.rs, where I spawn a child entity with a Head component on the player.

To reproduce: Hit ENTER to save, then R to delete all players, then BACKSPACE to load. Hit P to print debug info about all heads, and it'll show that the head's Parent entity does not exist. It will still point to the original parent ID from before you deleted them.

use bevy::prelude::*;
use bevy_inspector_egui::quick::WorldInspectorPlugin;
use bevy_save::prelude::*;

#[derive(Component, Reflect, Default)]
#[reflect(Component)]
pub struct Player;

#[derive(Component, Reflect, Default)]
#[reflect(Component)]
pub struct Head;

fn setup(mut commands: Commands) {
    commands
        .spawn((SpatialBundle::default(), Player))
        .with_children(|p| {
            p.spawn(Head);
        });

    // to reproduce the error, hit the following keys:
    // P         - Print debug info about heads and check their parent exists
    // ENTER     - Save
    // R         - Reset, delete all player entities
    // BACKSPACE - Load
    // P         - Print head info <== head will have an invalid parent
    println!("Controls:");
    println!("P: to print debug info on `Head` entities and to validate their parent exists");
    println!("R: to recursively delete all `Player` entities");
    println!("ENTER: Save");
    println!("BACKSPACE: Load");
}

fn interact(world: &mut World) {
    let keys = world.resource::<Input<KeyCode>>();

    if keys.just_released(KeyCode::Return) {
        info!("Save");
        world.save("example").expect("Failed to save");
    } else if keys.just_released(KeyCode::Back) {
        info!("Load");
        world.load("example").expect("Failed to load");
    } else if keys.just_pressed(KeyCode::E) {
        info!("Info");
        for entity in world.iter_entities() {
            info!("Entity: {:?}", entity.id());
            for component_id in entity.archetype().components() {
                if let Some(component) = world.components().get_info(component_id) {
                    info!("  {:?}: {:?}", entity.id(), component.name());
                }
            }
        }
    }
}

fn handle_keys(
    keys: Res<Input<KeyCode>>,
    head_query: Query<(Entity, &Parent)>,
    despawn_query: Query<Entity, With<Player>>,
    mut commands: Commands,
) {
    // Print head debug info, check that all heads have a valid parent
    if keys.just_released(KeyCode::P) {
        println!("{} Heads", head_query.iter().len());
        for (entity, parent) in &head_query {
            println!("  Head {:?} has parent: {:?}", entity, parent.get());
            if commands.get_entity(parent.get()).is_none() {
                println!("    X - Head parent does not exist!");
            } else {
                println!("    Ok - Head parent exists, all good")
            }
        }
    }

    // Reset, delete all entities
    if keys.just_released(KeyCode::R) {
        for entity in &despawn_query {
            commands.entity(entity).despawn_recursive();
        }
    }
}

fn main() {
    App::new()
        .add_plugins(DefaultPlugins.build().set(AssetPlugin {
            asset_folder: "examples/assets".to_owned(),
            ..default()
        }))
        // Inspector
        .add_plugin(WorldInspectorPlugin::new())
        // Bevy Save
        .add_plugins(SavePlugins)
        // Uncomment this and the app will crash after load, due to a `bevy` assertion on the parent/child heirarchy
        // .insert_resource(AppMappingMode::new(MappingMode::Strict))
        .register_saveable::<Player>()
        .register_saveable::<Parent>()
        .register_saveable::<Head>()
        .register_saveable::<Children>()
        .register_type::<Head>()
        .register_type::<Player>()
        .add_startup_system(setup)
        .add_system(interact)
        .add_system(handle_keys)
        .run();
}

How I've been using bevy_save

Not relevant to the issue, but I've been using bevy_save in a level editor for Undo + Redo behavior -- the rolbacks are a really great feature.

I'm updating to the latest bevy_save and it's going well enough. I'm unable to use the default WebStorage backend implementation because I rely on compressing save state prior to writing to local storage. This is especially important because local storage maxes out at 5mb. I've found it pretty easy to exceed that when creating a 2D tile map where each tile is an entity.

It's fine if you consider it out of scope. I think it's reasonable to see it as such and it's easy enough to implement a custom backend, but I thought I'd highlight my use case just to be sure.

Heyo.

Previously, I had code which looked like this:

fn create_save_snapshot(world: &mut World) -> Option<Vec<u8>> {
    let mut buffer: Vec<u8> = Vec::new();
    let mut serde = Serializer::new(&mut buffer);
    // Persistent entities must have an Id marker because Id is fit for uniquely identifying across sessions.
    let mut id_query = world.query_filtered::<Entity, With<Id>>();

    let snapshot = Snapshot::builder(world)
        .extract_entities(id_query.iter(world))
        .extract_all_resources()
        .build();
        
       ...

I am trying to express this using a custom Pipeline.

I see that SnapshotBuilder exposes only immutable access to World which means I am unable to invoke query_filtered from within capture_seed. I wanted to confirm that the expected way of handling this is something like:

struct SaveLoadPipeline {
    key: String,
    id_entities: Vec<Entity>,
}

impl SaveLoadPipeline {
    pub fn new(id_entities: Vec<Entity>) -> Self {
        Self {
            key: LOCAL_STORAGE_KEY.to_string(),
            id_entities
        }
    }
}

pub fn load(world: &mut World) -> bool {
    let mut id_query = world.query_filtered::<Entity, With<Id>>();
    let id_entities = id_query.iter(world).collect();

    world.load(SaveLoadPipeline::new(id_entities)).is_ok()
}

impl Pipeline for SaveLoadPipeline {
    type Backend = CompressedWebStorageBackend;
    type Format = DefaultDebugFormat;

    type Key<'a> = &'a str;

    fn key(&self) -> Self::Key<'_> {
        &self.key
    }

    fn capture_seed(&self, builder: SnapshotBuilder) -> Snapshot {
        builder
            .extract_entities(self.id_entities.iter().cloned())
            .extract_all_resources()
            .build()
    }

    // TODO: Confirm that lack of DespawnMode::None and MappingMode::Strict is OK
    // https://github.com/hankjordan/bevy_save/issues/25
    fn apply_seed(&self, world: &mut World, snapshot: &Snapshot) -> Result<(), bevy_save::Error> {
        snapshot.applier(world).apply()
    }
}

This approach seems okay. My concern is that, in my scenario, this results in cloning tens of thousands of entities rather than using the iterator directly, but Entity is cheap to clone so I'm not expecting major issues.

An alternative could be maintaining the result of the query in a resource and accessing the resource immutably from within capture_seed. This seems like a lot of overhead, but would make the save operation faster. That overhead didn't seem worth it right now.

I didn't think it was reasonable/possible to preserve the iterator on Pipeline as that would get lifetimes involved significantly.

Thanks

Hi, I'm struggling with an issue I can't quite figure out.

I'm building a snapshot like so:

    for ent in &entities {
        let name = world.get::<Name>(*ent);
        println!("{:?} name {:?}", ent, name);
    }

    Snapshot::builder(world)
        .extract_entities(entities.into_iter())
        .build()

Then applying it like so:

            let filter = <dyn Filter>::new::<With<Selectable>>();
            snapshot
                .applier(world)
                .despawn(bevy_save::DespawnMode::MissingWith(Box::new(filter)))
                .hook(move |entity, cmds| {
                    println!("---------spawning entity {:?}", entity.id());
                    let name = entity.get::<Name>();
                    println!("name {:?}", name);
                })
                .apply()
                .unwrap();

These are the entities that are being saved:

5v1 name Some(Name { hash: 4407845981047435273, name: "PlayerSpawn" })
4v1 name None
2v1 name Some(Name { hash: 7943398159714962848, name: "Directional Light" })
1v1 name None
12v0 name Some(Name { hash: 3891227486673895257, name: "Set" })
13v0 name None
11v0 name Some(Name { hash: 7808137298664908353, name: "Ground" })
14v0 name Some(Name { hash: 15962547213224080694, name: "Wall" })
22v1 name Some(Name { hash: 15962547213224080694, name: "Wall" })

But when applying, even immediately after saving the snapshot, this is what I see:

---------spawning entity 11v0
name Some(Name { hash: 6509597628072585063, name: "Ground" })
---------spawning entity 12v0
name Some(Name { hash: 1190878895532957759, name: "Set" })
---------spawning entity 13v0
name None
---------spawning entity 14v0
name Some(Name { hash: 12572175705635817883, name: "Wall" })
---------spawning entity 1v1
name None
---------spawning entity 19v2
name None
---------spawning entity 4v1
name None
---------spawning entity 2v2
name None

For some reason some entity generations have increased, and those that have no longer have Name components (or any components for that matter). I'm pretty sure the despawn step is deleting those entities, which I take it means they're missing from the snapshot, but I'm passing those entities to extract_entities so I don't understand why that would be the case.

I'm trying to figure out if there's something in my code that's causing this but in the meantime I thought maybe there's something I forgot to configure or am doing wrong? Thanks.

This is on:

bevy_save = "0.8.1"
bevy = "0.10.1"

          It would be nice if the features which did work on WASM were able to be used. My understanding is that this crate cannot be built for WASM targets currently because the `Default` for `AppBackend` hasn't been implemented yet. It's clear that this is the commented-out TODO code. For me, it would be preferable if this code panicked if used rather than preventing compilation entirely.

Originally posted by @MeoMix in #4 (comment)

Setting an upper bound on the number of Rollbacks being stored is important

It may be useful to have systems that are run whenever a save / load is triggered.
These systems could be set up to remap entities or deal with texture handle changes.
Users would then be able to chain these systems together to define save / load pipelines that would be able to replace many of the cumbersome filters.

Heyo.

I would appreciate a minimal example of how to save world state in such a way that it is fit for reuse across sessions:

Current documentation correctly calls out Entity as not being fit for persistence across sessions. I've introduced an Id component to my entities. It's a struct containing a Uuid:

#[derive(Resource, Debug, Default)]
pub struct IdMap(pub HashMap<Id, Entity>);

// Id is needed because Entity isn't fit for use across sessions, i.e. save state, refresh page, load state.
#[derive(Component, Debug, Eq, PartialEq, Hash, Clone, Serialize, Deserialize, Reflect)]
#[reflect(Component)]
pub struct Id(pub Uuid);

impl Default for Id {
    fn default() -> Self {
        Id(Uuid::new_v4())
    }
}

I understand that I am supposed to apply an entity_map to my Snapshot applier, but it's not clear to me what to do afterward.

If I create a mapping between Entity and Uuid, write to storage, and then refresh the page... what is supposed to happen when I load from storage? Am I supposed to have a way of reversing the mapping - potentially by persisting the map, too? That doesn't seem right because if that were the case then persisting Entity itself would work fine.

Presumably I can't just load the state without transforming it in some way because Id is too large of a type to be compatible with Entity.

Thanks

Based off the JSON example using the RON crate should work, but unlike the JSON implementation ron::de::from_reader requires a type that satisfies de::DeserializeOwned:

impl Loader for RonLoader {
    fn deserializer<'r, 'de>(&self, reader: Reader<'r>) -> IntoDeserializer<'r, 'de> {
        let deserializer = ron::de::from_reader(reader).unwrap();
        IntoDeserializer::erase(deserializer)
    }
}

error[E0282]: type annotations needed
   --> src/bin/editor.rs:904:13
    |
904 |         let deserializer = ron::de::from_reader(reader).unwrap();
    |             ^^^^^^^^^^^^
    |
help: consider giving `deserializer` an explicit type
    |
904 |         let deserializer: /* Type */ = ron::de::from_reader(reader).unwrap();
    |                         ++++++++++++

However, when I try to use Snapshot as the type, compilation fails because serde::Deserialize is not implemented for Snapshot.

impl Loader for RonLoader {
    fn deserializer<'r, 'de>(&self, reader: Reader<'r>) -> IntoDeserializer<'r, 'de> {
        let deserializer: Snapshot = ron::de::from_reader(reader).unwrap();
        IntoDeserializer::erase(deserializer)
    }
}

error[E0277]: the trait bound `for<'de> bevy_save::Snapshot: Deserialize<'de>` is not satisfied
   --> src/bin/editor.rs:904:38
    |
904 |         let deserializer: Snapshot = ron::de::from_reader(reader).unwrap();
    |                                      ^^^^^^^^^^^^^^^^^^^^ the trait `for<'de> Deserialize<'de>` is not implemented for `bevy_save::Snapshot`
    |
    = help: the following other types implement trait `Deserialize<'de>`:
              &'a [u8]
              &'a serde_json::raw::RawValue
              &'a std::path::Path
              &'a str
              ()
              (T0, T1)
              (T0, T1, T2)
              (T0, T1, T2, T3)
            and 389 others
    = note: required for `bevy_save::Snapshot` to implement `DeserializeOwned`
note: required by a bound in `from_reader`
   --> /Users/dmlary/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ron-0.8.0/src/de/mod.rs:72:8
    |
72  |     T: de::DeserializeOwned,
    |        ^^^^^^^^^^^^^^^^^^^^ required by this bound in `from_reader`

Is there any way to work around this?

This would be helpful to see in action as it is a very common usecase and I don't think any of the examples here do so.

Hello. I've been experimenting with bevy_save for a bit. I've run into an issue where I'm attempting to use a Snapshot to restore to a previous state of a component with a Vec/Vecdeque.

When I have a component like:

    #[derive(Component, Reflect, Default, Debug, Clone, PartialEq, Eq)]
    #[reflect(Component)]
    struct Collect {
        data: Vec<u32>,
    }

Which is registered as a savable (and the Vec is registered as a type in Bevy):

        app.register_saveable::<Collect>();
        app.register_type::<Vec<u32>>();

Applying a snapshot seems to make no change to the data
I suspect it's more general than just "Vecs". But I couldn't figure it out digging through the code. I guess it has something to with the reflected data though!

I've reproduced this with a test. The first test fails, the second is succeeding as a comparison to the normal behavior. Thanks for your help!

mod tests {
    use super::*;
    use bevy_save::{AppSaveableExt, SavePlugins};

    #[derive(Component, Reflect, Default, Debug, Clone, PartialEq, Eq)]
    #[reflect(Component)]
    struct Collect {
        data: Vec<u32>,
    }

    #[test]
    fn this_does_not_work() {
        let mut app = App::new();
        app.add_plugins(SavePlugins);
        app.register_saveable::<Collect>();
        app.register_type::<Vec<u32>>();
        let world = &mut app.world;

        let entity = world
            .spawn_empty()
            .insert(Collect { data: Vec::new() })
            .id();
        world
            .entity_mut(entity)
            .get_mut::<Collect>()
            .unwrap()
            .data
            .push(1);
        let snapshot = world.snapshot();

        world
            .entity_mut(entity)
            .get_mut::<Collect>()
            .unwrap()
            .data
            .push(2);
        snapshot.applier(world).apply().unwrap();
        assert_eq!(world.entity(entity).get::<Collect>().unwrap().data.len(), 1);
        assert_eq!(
            *world
                .entity(entity)
                .get::<Collect>()
                .unwrap()
                .data
                .first()
                .unwrap(),
            1
        );
    }

    #[derive(Component, Reflect, Default, Debug, Clone, PartialEq, Eq)]
    #[reflect(Component)]
    struct Basic {
        data: u32,
    }

    #[test]
    fn this_works() {
        let mut app = App::new();
        app.add_plugins(SavePlugins);
        app.register_saveable::<Basic>();
        let world = &mut app.world;

        let entity = world.spawn_empty().insert(Basic { data: 0 }).id();
        world.entity_mut(entity).get_mut::<Basic>().unwrap().data = 1;
        let snapshot = world.snapshot();

        world.entity_mut(entity).get_mut::<Basic>().unwrap().data = 2;
        snapshot.applier(world).apply().unwrap();
        assert_eq!(world.entity(entity).get::<Basic>().unwrap().data, 1);
    }
}

Heyo.

I updated to Bevy 0.12.1 today which unblocked my WASM builds and allowed me to test the changes I made to my app to support the new bevy_save.

After making the necessary updates, I encountered an error in deserialization:

where StoryTime looks like:

#[derive(Resource, Clone, Reflect)]
#[reflect(Resource)]
pub struct StoryTime {
    elapsed_ticks: isize,
    pub is_real_time: bool,
    pub is_real_sun: bool,
    pub latitude: f32,
    pub longitude: f32,
    real_time_offset: isize,
    demo_time_offset: isize,
}

After a bit of fussing, I realized that this error went away if I stopped using rmp_serde and instead used serde_json. The only changes I made to my code to swap between the two were:

    let mut serde = rmp_serde::Serializer::new(&mut buffer);
    //let mut serde = serde_json::Serializer::new(&mut buffer);

    ...

    let mut deserializer = rmp_serde::Deserializer::new(&data[..]);
    //let mut deserializer = serde_json::Deserializer::from_reader(&data[..]);

After these changes, serialization/deserialization work on 0.12.1 just as I'd hope.

More confusingly, I've been happily using rmp_serde (w/ Brotli compression) against Bevy 0.11 for the past couple of months (https://github.com/MeoMix/symbiants/blob/48b9120e6ea5877ce06b73be3225330979fb4647/src/save/save_web.rs) without issue.

Is there something I'm not understanding here? Why would updating to latest break my usage of rmp_serde? Did you run into a technical reason for using json_serde for WASM support in bevy_save - or was it merely a stopgap because of slight interface differences between rmp_serde and json_serde ?

Thanks

Should be straightforward

As of v0.7.0, bevy_save simply saves files directly to the disk after serialization.

We should allow users to change how bevy_save saves and loads - perhaps they want to use a database or have other requirements like on WASM.

Trying to compile my bevy wasm game to use webgpu,
I noticed that bevy_save enables bevy/webgl2 feature, which auto-disables webgpu support.

Would it be possible to get rid of this from Cargo.toml ?

Saving and loading should be done asynchronously.

If these methods block, saving and loading even moderately sizes worlds may cause a noticeable impact on framerate.

This effect is especially bad on hard drives or platforms with high IO latency.

Additionally, backends that utilize network IO are effectively unusable without async.

Is there a way to automatically capture the new ID of an entity spawned from the snapshot? Or I need a mechanism to match a new entity ID to the old one manually, like in the pipeline example, where they're matched by the position

upd
What I need to achieve:
I have a tilemap, a tile storage stores IDs of tile entities that belong to it. After I load tiles and the storage from the save file all tiles have new IDs so tile storage now contains invalid entity IDs. I wasn't able to figure out how to do that yet. The only idea is, like in the pipeline example, to manually map entity IDs based on some tile props. Is that the only way?

Most features should already work on WASM, these do not:

• World::save
• World::load

This can be closed by implementing a WebStorage backend.

Currently, clear_empty is used such as:

    let snapshot = Snapshot::builder(world)
        .extract_all()
        // Prevent serialization bloat by removing entities with no saveable components
        .clear_empty()
        .build();

This requires iterating all extracted entities a second time. For large worlds, this can be non-trivial, even if most of the world's entities are not intended to be saved.

It would be preferable to express the desire to filter on empty through configuration prior to callling extract_all. Something similar to how a deserialize applier can have despawn and mapping configured on it prior to calling apply.

For noobies like me it would be far easier to understand what is in the save file if it is a known format like .ron or .json. Adding an example for this would be nice.

I'll put up a PR for it at some point, I doubt there's anything too significant for this plugin, but wanted to highlight that we're behind a version.

I get error when trying to load a save file which includes a resource which hasn't been inserted yet into the world. I assumed that bevy_load would insert it for me?

        info!("Load");
        world.load("example");  //<--- Crash
        dgb!(world.get_resource::<Test>().unwrap());

resource Test was added to the save file but left out after restarting the program

error:

thread 'main' panicked at 'Requested resource main::Test does not exist in the `World`. 
                Did you forget to add it using `app.insert_resource` / `app.init_resource`? 
                Resources are also implicitly added via `app.add_event`,
                and can be added by plugins.', /home/pinkponk/.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_ecs-0.9.1/src/reflect.rs:368:42
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Even when the bevy_save save format is stabilized, users will need a way to migrate save files between different versions of their app.

Ideally, this should be handled transparently.

It may be possible to eliminate the need for reflection when using snapshots.
This would dramatically improve the performance of bevy_save.

See deflect branch