aevyrie / bevy_mod_picking Goto Github PK

Need some way of consistently benchmarking, probably using criterion.
Requires #18 .

The stress_test example provides a challenging picking scenario. Need to find a way to supply user input for an integration test, or alternatively, decouple the benchmark from bevy. The raycast functions have been separated already, but that only represents a small part of the code that needs to be profiled.

With the updated tracing branch stress_test, there appear to be dead zones.

When running cargo build --examples I get the error:

error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `rustc - --crate-name ___ --print=file-names -Clink-arg=-fuse-ld=lld -Zshare-generics=y --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit code: 1)
  --- stderr
  error: the option `Z` is only accepted on the nightly compiler

This is because .cargo/config is in .git and contains options which are only available on nightly.

I think .cargo/config shouldn't be committed. Note that bevy itself has this file as .cargo/config_fast_builds so it won't actually be used until it's been renamed.

It would be awesome to have the ability for a user to be able to define custom rays that is consumed by the picking system.

I am not sure if it would be possible to allow a user to either cast a ray from screen coordinates (in which case the mouse position is just another screen coordinate) or cast from a world coordinate. I don't know if these cases would need to be handled separately (I am new to the world of ray casting).

My initial thought was just to extract the cursor position code into some user-defined position but then I wasn't sure if the user would have to specify the position from their system in one frame and not be able to get the result until the next frame (some sort of queue?).

I would be happy to try and help out if you need an extra pair of hands but would need to ask some questions or be guided on what you are currently thinking when it comes to approaching this.

Great work so far by the way!

P.S I am Joonipah on the Discord channel.

Simplify getting topmost pick by moving this to the pick source. This is more ECS-idiomatic, and will remove the need for "global" state in the plugin's top level. This might require restructuring how groups work.

The most sensible way to do this would store this information in a group, as this can only have one intersection. A pick source, however, can belong to multiple groups and have multiple intersections associated with it. Right now, though, groups don't exist as entities and have no way to store data.

Alternatively, maybe a new constraint could be added, where a Pick Source can only be used in a single group? What use cases does this prevent? Using the mouse as a single source and finding the top pick for multiple groups?

Need a way to debug bounding sphere placement when models are transformed. There still seem to be some bugs with this.

Coordinates are currently reported in ndc space. Use the associated camera transform to convert to world space, evaluating lazily.

I have a strange bug with adding children to a parent entity, despawning them, then adding them in again:

https://github.com/rezural/bevy_mod_picking/commits/3d-scene-heirarchy-bug
the example is in this branch

commit 644e10c works as expected
whereas commit 14df054 creates 2 sets of children, one offset a little.

Reproduce:

git checkout 644e10c
cargo run --release --example 3d_scene_heirarchy

git checkout 14df054
cargo run --release --example 3d_scene_heirarchy

As evidenced by the new profiling feature, the pick_mesh function is the hottest loop in the plugin, and it is not splitting up any of the work across threads.

It takes about 2ms each frame to run this function. On a quad core this means there is opportunity to get this down around 1ms on the same system. I attempted to use par_iter_mut() but ran into a few roadblocks preventing it from being a simple switchover from iter_mut(). Need to take a closer look at how to restructure this function to parallelize bounding sphere checks and raycasting.

on OSX

❯ cargo run --features ex --example 3d_scene

  Downloaded bevy_mod_raycast v0.2.1
  Downloaded 1 crate (30.3 KB) in 1.46s
   Compiling libc v0.2.93
   Compiling proc-macro2 v1.0.26
   Compiling unicode-xid v0.2.1
   Compiling syn v1.0.65
   Compiling cfg-if v1.0.0
   Compiling autocfg v1.0.1
   Compiling serde_derive v1.0.125
   Compiling bitflags v1.2.1
   Compiling serde v1.0.125
   Compiling lazy_static v1.4.0
   Compiling libm v0.2.1
   Compiling byteorder v1.4.3
   Compiling pin-project-lite v0.2.6
   Compiling getrandom v0.2.2
   Compiling memchr v2.3.4
   Compiling scopeguard v1.1.0
   Compiling once_cell v1.7.2
   Compiling log v0.4.14
   Compiling version_check v0.9.3
   Compiling futures-core v0.3.14
   Compiling cache-padded v1.1.1
   Compiling futures-io v0.3.14
   Compiling fastrand v1.4.0
   Compiling parking v2.0.0
   Compiling waker-fn v1.1.0
   Compiling async-task v4.0.3
   Compiling event-listener v2.5.1
   Compiling downcast-rs v1.2.0
   Compiling vec-arena v1.1.0
   Compiling ppv-lite86 v0.2.10
   Compiling ryu v1.0.5
   Compiling fixedbitset v0.4.0
   Compiling Inflector v0.11.4
   Compiling serde_json v1.0.64
   Compiling regex-syntax v0.6.23
   Compiling itoa v0.4.7
   Compiling foreign-types-shared v0.1.1
   Compiling core-foundation-sys v0.8.2
   Compiling ansi_term v0.12.1
   Compiling ahash v0.4.7
   Compiling same-file v1.0.6
   Compiling anyhow v1.0.40
   Compiling base64 v0.13.0
   Compiling anymap v0.12.1
   Compiling bit-vec v0.6.3
   Compiling bevy-glsl-to-spirv v0.2.1
   Compiling fixedbitset v0.2.0
   Compiling block v0.1.6
   Compiling color_quant v1.1.0
   Compiling core-foundation-sys v0.7.0
   Compiling bytemuck v1.5.1
   Compiling hex v0.4.3
   Compiling ttf-parser v0.12.0
   Compiling cfg_aliases v0.1.1
   Compiling arrayvec v0.5.2
   Compiling copyless v0.1.5
   Compiling ab_glyph_rasterizer v0.1.4
   Compiling svg_fmt v0.4.1
   Compiling cfg-if v0.1.10
   Compiling range-alloc v0.1.2
   Compiling libm v0.1.4
   Compiling inflections v1.1.1
   Compiling dispatch v0.2.0
   Compiling rectangle-pack v0.3.0
   Compiling xi-unicode v0.3.0
   Compiling instant v0.1.9
   Compiling tracing-core v0.1.17
   Compiling sharded-slab v0.1.1
   Compiling lock_api v0.4.3
   Compiling concurrent-queue v1.2.2
   Compiling thread_local v1.1.3
   Compiling fxhash v0.2.1
   Compiling num-traits v0.2.14
   Compiling num-integer v0.1.44
   Compiling crossbeam-utils v0.8.3
   Compiling indexmap v1.6.2
   Compiling num-rational v0.3.2
   Compiling num-iter v0.1.42
   Compiling foreign-types v0.3.2
   Compiling walkdir v2.3.2
   Compiling ahash v0.7.2
   Compiling hashbrown v0.9.1
   Compiling bit-set v0.5.2
   Compiling wgpu-core v0.7.1
   Compiling storage-map v0.3.0
   Compiling async-channel v1.6.1
   Compiling owned_ttf_parser v0.12.0
   Compiling regex-automata v0.1.9
   Compiling gpu-alloc-types v0.2.0
   Compiling gpu-descriptor-types v0.1.1
   Compiling wgpu-types v0.7.0
   Compiling futures-lite v1.11.3
   Compiling aho-corasick v0.7.15
   Compiling ab_glyph v0.2.10
   Compiling quote v1.0.9
   Compiling jobserver v0.1.21
   Compiling num_cpus v1.13.0
   Compiling fsevent-sys v3.0.2
   Compiling malloc_buf v0.0.6
   Compiling filetime v0.2.14
   Compiling raw-window-handle v0.3.3
   Compiling tracing-log v0.1.2
   Compiling core-foundation v0.9.1
   Compiling matchers v0.0.1
   Compiling core-foundation v0.7.0
   Compiling regex v1.4.5
   Compiling rand_core v0.6.2
   Compiling async-executor v1.4.0
   Compiling cc v1.0.67
   Compiling fsevent v2.0.2
   Compiling crossbeam-channel v0.5.1
   Compiling core-graphics-types v0.1.1
   Compiling rand_chacha v0.3.0
   Compiling petgraph v0.5.1
   Compiling core-graphics v0.19.2
   Compiling bevy_tasks v0.5.0
   Compiling core-graphics v0.22.2
   Compiling stretch v0.3.2
   Compiling rand v0.8.3
   Compiling spirv_headers v1.5.0
   Compiling euclid v0.22.3
   Compiling approx v0.4.0
   Compiling notify v5.0.0-pre.6
   Compiling chrono v0.4.19
   Compiling glyph_brush_layout v0.2.1
   Compiling image v0.23.14
   Compiling objc_exception v0.1.2
   Compiling spirv-reflect v0.2.3
   Compiling spirv_cross v0.23.1
   Compiling guillotiere v0.6.0
   Compiling objc v0.2.7
   Compiling cocoa-foundation v0.1.0
   Compiling core-video-sys v0.1.4
   Compiling metal v0.21.0
   Compiling cocoa v0.24.0
   Compiling thiserror-impl v1.0.24
   Compiling tracing-attributes v0.1.15
   Compiling gltf-derive v0.15.2
   Compiling winit v0.24.0
   Compiling thiserror v1.0.24
   Compiling tracing v0.1.25
   Compiling naga v0.3.2
   Compiling gpu-descriptor v0.1.1
   Compiling gpu-alloc v0.3.0
   Compiling gfx-hal v0.7.0
   Compiling gfx-auxil v0.8.0
   Compiling gfx-backend-empty v0.7.0
   Compiling smallvec v1.6.1
   Compiling toml v0.5.8
   Compiling uuid v0.8.2
   Compiling glam v0.13.1
   Compiling erased-serde v0.3.13
   Compiling tracing-serde v0.1.2
   Compiling ron v0.6.4
   Compiling parking_lot_core v0.8.3
   Compiling bevy_utils v0.5.0
   Compiling parking_lot v0.11.1
   Compiling gfx-backend-metal v0.7.0
   Compiling find-crate v0.6.3
   Compiling tracing-subscriber v0.2.17
   Compiling gltf-json v0.15.2
   Compiling bevy_reflect_derive v0.5.0
   Compiling bevy_ecs_macros v0.5.0
   Compiling bevy_derive v0.5.0
   Compiling hexasphere v3.2.0
   Compiling bevy_reflect v0.5.0
   Compiling gltf v0.15.2
   Compiling bevy_ecs v0.5.0
   Compiling bevy_math v0.5.0
error: unexpected token: `0.`
   --> /Users/me/.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_ecs-0.5.0/src/system/system_param.rs:154:1
    |
154 | impl_query_set!();
    | ^^^^^^^^^^^^^^^^^^
    | |
    | unexpected token
    | help: try parenthesizing the first index: `(self.0).`
    |
    = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0658]: use of unstable library feature 'bool_to_option'
   --> /Users/me/.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_ecs-0.5.0/src/schedule/stage.rs:252:26
    |
252 |                         .then(|| descriptor.system.name()),
    |                          ^^^^
    |
    = note: see issue #64260 <https://github.com/rust-lang/rust/issues/64260> for more information
    = help: add `#![feature(bool_to_option)]` to the crate attributes to enable

error[E0658]: use of unstable library feature 'bool_to_option'
   --> /Users/me/.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_ecs-0.5.0/src/schedule/stage.rs:256:26
    |
256 |                         .then(|| descriptor.system.name()),
    |                          ^^^^
    |
    = note: see issue #64260 <https://github.com/rust-lang/rust/issues/64260> for more information
    = help: add `#![feature(bool_to_option)]` to the crate attributes to enable

error[E0599]: no method named `get_mut` found for struct `std::cell::UnsafeCell<std::vec::Vec<component::ComponentTicks>>` in the current scope
   --> /Users/me/.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_ecs-0.5.0/src/storage/sparse_set.rs:125:37
    |
125 |         let ticks_list = self.ticks.get_mut();
    |                                     ^^^^^^^ method not found in `std::cell::UnsafeCell<std::vec::Vec<component::ComponentTicks>>`

error[E0599]: no method named `get_mut` found for struct `std::cell::UnsafeCell<std::vec::Vec<component::ComponentTicks>>` in the current scope
   --> /Users/me/.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_ecs-0.5.0/src/storage/sparse_set.rs:201:24
    |
201 |             self.ticks.get_mut().swap_remove(dense_index);
    |                        ^^^^^^^ method not found in `std::cell::UnsafeCell<std::vec::Vec<component::ComponentTicks>>`

error[E0599]: no method named `get_mut` found for struct `std::cell::UnsafeCell<std::vec::Vec<component::ComponentTicks>>` in the current scope
   --> /Users/me/.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_ecs-0.5.0/src/storage/sparse_set.rs:217:32
    |
217 |         let ticks = self.ticks.get_mut().iter_mut();
    |                                ^^^^^^^ method not found in `std::cell::UnsafeCell<std::vec::Vec<component::ComponentTicks>>`

error: aborting due to 6 previous errors

Some errors have detailed explanations: E0599, E0658.
For more information about an error, try `rustc --explain E0599`.
error: could not compile `bevy_ecs`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

The goal I am trying to wrap my head around is this:
Suppose I have selected a unit then I want to click somewhere on the world/map so the unit can have a new destination to go to.

How would I go about it? As far as I can see, the events coming out of the system don't have the intersection vector, where the ray hit the world. How could I go about achieving this?

I'm working my way through this bevy tutorial and have hit what seems to be a bug in bevy_mod_picking.

It didn't work initially with the tutorial's original camera config, which (with updates for bevy 0.6) looks like this:

PerspectiveCameraBundle{
    transform: Transform::from_matrix(Mat4::from_rotation_translation(
        Quat::from_xyzw(-0.3, -0.5, -0.3, 0.5).normalize(),
        Vec3::new(-7.0, 20.0, 4.0),
    )),
    ..Default::default()
}

After messing around with the minimal example from bevy_mod_picking, I finally found a version that works:

PerspectiveCameraBundle {
    transform: Transform::from_xyz(-6.0, 5., 3.5)
        .looking_at(Vec3::new(4.0, 0., 3.499), Vec3::Y),
    ..Default::default()
}

However, if I change the z of the Vec3 to 3.5 (or greater), picking stops working. There's no debug cursor or logged events. Also, in the first camera version above, tweaking the rotation a bit can also make picking work again.

Thanks!

Full example here: https://gist.github.com/tgecho/a88a27825f2cb8b404cc0c4699e5bb5f
rust 1.60.0-nightly (17d29dcdc 2022-01-21)
bevy 0.6.0
bevy_mod_picking 0.5.2

Just a tracking issue so I can follow when the new version gets released.

Is master being worked on it now?

Currently the ray cast just grabs the first vertice's depth. Correct behavior should calculate depth based on cursor position within triangle.

There is a noticeable lag on the first mouse-over a mesh when using HighlightablePickMesh.
For example:

I also have the lag on the example code 3d_scene.

cargo run --example 3d_scene --release --features="example_deps"

Above, I'm talking about the small freeze at the ~2.5s mark - it's of course much more noticeable when camera or objects are moving.

I'm using:

• Windows 10 build 1909
• rust 1.51.0-nightly (also tried using stable1.47.0 with same result)
• intel i7-4790K CPU @ 4.00GHz
• NVIDIA GeForce GTX 970

I have also experienced this on another windows-machine with poorer specs but with the same stutter, so I don't believe it's an issue with my machine.

Since it only happens once, I think it is a pretty minor issue, but I'd love if there was a way to "trigger the lag" perhaps during a loading screen, instead of when the user first interacts with the meshes.

Just got this plugin working - great job 🙂

Seems like PickingSource has been renamed to PickSource, but the docs and error messages all reference PickingSource.

If this is the case, I can make a PR to rename them.

Hello, lovely plugin you have here! I was just implementing some simple movement in my game using this, and I ran into a minor issue. See I have a cube that has to move along a plane when a part of the plane's surface is clicked, and it's supposed to move to the point of the plane. The issue I'm facing is that the camera moves with it, and the intersection does not get updated unless I move the mouse, despite the fact that the mouse obviously intersects with the mesh in a different spot due to the new camera position. See attached gif for an example.

I'm using the InteractableMesh and everything else works swimmingly, I figure this might be an oversight in how the InteractableMesh::mouse_down_event interacts with PickableMesh::intersection.

I'm using version 0.2.0 so if this has already been fixed I apologize

So I was having issues with using this repo with bevy main, and I stumbled into an issue with queries like Query<&PickingCamera> and it led me to realize that while Component derives have been added to components on this repo and on bevy_mod_raycast, there has not been a release of raycast with this update, and thus when this repo pulls bevy_mod_raycast v0.2 you get components that do not derive Component.

The ideal solution would be to make a new release for bevy_mod_raycast, then bump the version number on this repo.

Not sure if you were planning on releasing along side the next bevy release but the changes in master that register systems as PreUpdate instead of PostUpdate seem pretty useful. I was about to file a bug/open a PR for this and noticed it was already fixed.

For performance, I'd like to render the scene to an off-screen buffer that renders each pixel as an entity encoded into RGBA. Picking is then as simple(😆) as querying this buffer and doing a lookup to return the entity. This would only allow picking for the topmost item (no pick list), and would have limitations with transparency.

A hybrid mode that uses both might be a good solution.

Some input from others on Discord for reference: https://discord.com/channels/691052431525675048/743663924229963868/755912884885913751

Adding BoundVol to my components causes my app to freeze on startup (showing only a blank white screen). Seems like it's caught in an infinite loop somewhere? On web (not sure that matters).

I am generated meshes by hand, which seems to be the culprit. They render do correctly, but aren't playing well with BoundVol. Replacing all my meshes with generic Cubes seems to work just fine. Here's my relevant code

Hello,
I noticed an issue that happens quite frequently in my app when an Interactable entity keeps receiving the MouseJustReleased event even when the mouse is at rest.
Sorry i dont have an easy repro... Though I managed to repro it in the events example, by adding more "planes" next to another, and clicking frantically on a plane then another then back etc..
Eventually you'll see the MouseJustReleased start pouring non stop, even if you take your hand off the mouse.

This seems to happen when you click on a plane then move the mouse out of the plane quickly, because it stops pourring events when you bring the cursor back on top of the stuck plane (if that makes sense)

I hope that's enought information, i'll try to find time to investigate this, might be an issue in the event system or in bevy's mouse input.

Here is the code i use to repro, in example/event.rs just replace the plane spawning by:

//plane
.spawn(PbrBundle {
mesh: meshes.add(Mesh::from(shape::Plane { size: 4.9 })),
transform: Transform::from_translation(Vec3::new(-5.0, 0.0, -5.0)),
material: materials.add(Color::rgb(1.0, 1.0, 1.0).into()),
..Default::default()
})
.with(PickableMesh::default())
.with(InteractableMesh::default())
//plane
.spawn(PbrBundle {
mesh: meshes.add(Mesh::from(shape::Plane { size: 4.9 })),
transform: Transform::from_translation(Vec3::new(0.0, 0.0, 0.0)),
material: materials.add(Color::rgb(1.0, 1.0, 1.0).into()),
..Default::default()
})
.with(PickableMesh::default())
.with(InteractableMesh::default())
//plane
.spawn(PbrBundle {
mesh: meshes.add(Mesh::from(shape::Plane { size: 4.9 })),
transform: Transform::from_translation(Vec3::new(0.0, 0.0, -5.0)),
material: materials.add(Color::rgb(1.0, 1.0, 1.0).into()),
..Default::default()
})
.with(PickableMesh::default())
.with(InteractableMesh::default())
//plane
.spawn(PbrBundle {
mesh: meshes.add(Mesh::from(shape::Plane { size: 4.9 })),
transform: Transform::from_translation(Vec3::new(-5.0, 0.0, 0.0)),
material: materials.add(Color::rgb(1.0, 1.0, 1.0).into()),
..Default::default()
})
.with(PickableMesh::default())
.with(InteractableMesh::default())

Then quickly click on 1 plane, then another, then another... etc

Hope that helps!

Need to update the readme to account for changes as a result of #52.

Work:

impl Plugin for Battle {
    fn build(&self, app: &mut App) {
        app.add_plugin(DefaultPickingPlugins)
            .add_startup_system(setup.system());
    }
}

Doesn't work:

impl Plugin for Battle {
    fn build(&self, app: &mut App) {
        app.add_plugin(DefaultPickingPlugins)
            .add_system_set(SystemSet::on_enter(GameState::Battle).with_system(setup.system()));
    }
}

(system setup creates entities and set up PickableBundle::default())

The bevy_mod_picking doesn't work then state is GameState::Battle.

Maybe it's better to use, something like this? :

pub struct DefaultPickingPlugins;

impl PluginGroup for DefaultPickingPlugins {
    fn build(&mut self, group: &mut PluginGroupBuilder) {
        group.add(PickingPlugin);
        group.add(InteractablePickingPlugin);
        group.add(HighlightablePickingPlugin);
    }
}

Instead of:

pub struct DefaultPickingPlugins;
impl Plugin for DefaultPickingPlugins {
    fn build(&self, app: &mut App) {
        app.add_plugin(PickingPlugin)
            .add_plugin(InteractablePickingPlugin)
            .add_plugin(HighlightablePickingPlugin);
    }
}

And in Bevy use:

.add_plugins(DefaultPickingPlugins)

Instead of:

.add_plugin(DefaultPickingPlugins)

This plugin should be agnostic to camera transform. Bevy currently forces the 3d camera bundle to use a perspective projection. May require a PR in bevy to fully button this up.

To make critical functions easier to benchmark and test, they should be decoupled from bevy so they can be run independently from it.

Is there a way not to depend on the cursor_events.latest() when updating the picking on line 263 on lib.rs?

I am thinking of adding a field in PickState the previous_mouse_pos and use it when cursor is not moved, but I would like to know if that's the possible solution, thanks.

Sample code is here: https://github.com/Nickan/bevytest/blob/ad375b83351cd6d698c316ff7d00e82db3f7a824/src/bevy_mod_picking_test.rs

The current ray casting implementation is naive, and queries all meshes in the scene. The first optimization I'd like to apply is checking against bounding spheres before checking each triangle in a mesh. This should greatly improve performance in cases where the cursor is hovering over an area with very few objects.

Notes:

• Bounding sphere radius calculation requires iterating over all vertices of a mesh, and should be done only when needed, or the performance improvement is negated.
  • Bounding sphere radius only needs a re-calc if the mesh changes
• Defining the sphere in mesh space means the radius of the sphere remains valid even if the mesh is transformed.
  • Need to apply max(abs(scaling)) if the mesh is scale is not 1.0 in the transform
• Check bounding sphere by transforming sphere center with the ray's position and direction as a transform, to view the point from the perspective of the ray.
  • From this point, it's a 2D distance test from the origin (ray axis) to the sphere's transformed x/y positions. If the distance is greater than the radius, this mesh can be ignored.

Experiencing a memory leak on all web browsers (master branch). Removing the PickableBundle from my component fixes the issue. Any ideas or suggestions for debugging this?

Update cargo.toml and docs for a Bevy 0.3 release. I'd like to have #52 included in this release.

Hello,

Thank you for this excellent feature!

Curiosity, the mesh in our scene are categorized in two camps: friends and enemies.

What would be the best strategy to dynamically assign red or green hover color depending of the mesh camp?

• is there a way to set it when the mesh is spawn?
• is it better to assign it when the user pass the cursor over the mesh?
• other way?

Thank you!
Sébastien

Currently using the area sum method with a manually specified eplison threshold due to floating point error. Investigate using barycentric coordinates.

I think a good improvement would be to add additional logic to handle
on_click_down, on_click_up, on_hover, on_mouse_in, and on_mouse_out cursor events.

I see two ways to accomplish this. One is to have ClickDown (and the other events) as components attached to each entity ala:

.with(ClickDown::new(|event, entity| {
    println!("I have been clicked most judiciously and with great conviction");
})

We could then create a system that would query all of the entities that possess ClickDown and run the attached closure as part of a system this library provides. I'm not sure this is the correct approach as you would be limited by the closure's scope.

Alternatively we could provide a CursorEvents resource that contains all of the Cursor Event data such as hovering, meshes that have been click-down and click-upped etc.

fn tic_tac_toe(mut commands: Commands, cursor_events: Res<CursorEvents>, mut comp_query: Query<&Tile>){
    //Logic here
    
}

The big draw of the second approach is matching the entities up to a click event in a performant manner. Currently you would have to do something like this:

    if let Some(pick_intersection) = pick_state.top(){
        //Handle Pick Data To Flip The Mesh
        for (tile, mut mesh) in tile_query.iter().iter() {
            
            if *tile == *tile_query2.get::<Tile>(pick_intersection.entity()).unwrap() {
                let cube = meshes.sphere.unwrap();
                *mesh = cube;
            }

        }
    }

At the least I think we should make that a utility method provided by the library, which would then make the resource-system oriented approach feasible and approachable to new developers.

    pick_state.find_matches(Query<Passed, To, The, Enclosing, System>) //This query must be a duplicate as internally they mutably borrow bevy resources. 

Would love to get someone's thoughts on this as I think it would be a big improvement to APi usability.

Use the CursorEntered and CursorLeft events to enable and disable mouse picking.

Hi,

First thank you a lot for this great package!

I want to be able to retrieve the world coordinates on the plane y=0 when the user clicks on the screen. I have created a (big) plane on y=0, made it invisible and pickable. With this setup I can get the coordinates I am looking for when the user selects the plane. However, because the same object cannot be selected twice, I always need to deselect the plane manually. Like this:

fn mouse_event_debug(
    mut events: EventReader<PickingEvent>,
    mut query: Query<&mut Selection>,
) {
    for event in events.iter() {
        println!("This event happened! {:?}", event);
        if let PickingEvent::Selection(selection) = event {
            if let SelectionEvent::JustSelected(entity) = selection {
                let mut selection = query.get_mut(*entity).unwrap();
                println!("Deselect {:?}!", entity);
                selection.set_selected(false);
            }
        }
    }
}

Wouldn't it make more sense to have a ClickEvent instead for this kind of situation? and to be able to have objects that can be clicked but not necessarily selected?

Should ignore triangles if the entire triangle is outside of ndc space. Will need to handle cases where all three vertices are outside of ndc, but the body of the triangle is still visible. Can probably handle this best once bounding spheres (#2 ) are implemented. e.g. cull spheres if sphere distance to origin is > (sqrt(2) + sphere.radius)

With updated versions of Bevy / Cargo I get this error:

error[E0596]: cannot borrow `query_selectables` as mutable, as it is not declared as mutable
   --> lib.rs:316:39
    query_selectables: Query<&SelectablePickMesh>,
  ----------------- help: consider changing this to be mutable: `mut query_selectables`

cargo.toml dependencies:

[dependencies]
bevy = { git = "https://github.com/bevyengine/bevy", rev = "34c6f5f41bd2bff7867e269985a54036b8d96771"}
bevy_mod_picking = { git = "https://github.com/aevyrie/bevy_mod_picking", branch = "master" }

Appears to be a breaking change from Bevy.

Rustup / Cargo Info:

active toolchain
----------------

nightly-x86_64-pc-windows-msvc (default)
rustc 1.48.0-nightly (d006f5734 2020-08-28)

For UX or performance, might want to disable cursor picking during certain times, for example during camera manipulation. Probably want selection highlighting to remain active. Maybe allow granular enable/disable per component?

It would be helpful for the debug cursor, bounding sphere optimization #2, and potential users, if the intersection normal ray was expressed as a transform. This would make it simple to transform meshes or make computations from the intersection normal position.

Right now if the mouse is over a UI element, like a button, the plugin ignores it completely and still returns picks as if the element wasn't there.

I don't know if it's something that should be dealt with in the plugin, because a user might need picking to work under a UI element. What are your thoughts on this?

I think the best solution would be to provide an option to enable/disable this behavior. Maybe we could create a PickingBlocker component that can be added to a UI element to disable picking when on top of it.

Support picking non-meshes. This is probably just adding an extensible API to #3.

Generate Bevy Events from interactions

I'd love to be able to group multiple Meshes into one Pickable entity. i.e., if I spawn a Parent entity with multiple children meshes, selecting any one of the children selects the entire hierarchy (parent and siblings).

My use-case is that I am importing a GLTF Scene which has Nodes which are made of multiple Meshes. I want to treat each GLTF Node as one Pickable entity. I've tried implementing this locally but haven't made progress.

If you (A) know of a workaround or (B) know where in the code I could add this feature I would gladly make this a pull request.

I expect this would be implemented with the following psuedocode:

1. When a mesh is selected, check if the mesh has a parent.
2. Update the state for all of that parent's children to match the selected child's state.
3. Hierarchy's could be implemented with a PickableHierarchyBundle on the parent and/or children -- or just tag the parent (as well as the children) with the PickableBundle.

I'd love to discuss design if I'm [not] on the right track.

P.S., I love the plugin! I'm using it in a game and it's made a point-and-click mechanic very easy!

Something weird happens sometimes, where a pick_state.top() returns None even if there is a Plane there. The issue occurs when the Plane is large, like size 50.0. I don't have the problem with size 10.0.

The Plane is created like this:

    commands.PbrComponents {
            mesh: meshes.add(Mesh::from(shape::Plane { size: 50.0 })),
            material: materials.add(Color::rgb_u8(222, 184, 135).into()),
            ..Default::default()
        })
        .with(PickableMesh::new(camera_entity));

I've recorded a couple gifs of this happening, with the same system used in the example:

fn get_picks(pick_state: ResMut<PickState>) {
    println!("All entities:\n{:?}", pick_state.list());
    println!("Top entity:\n{:?}", pick_state.top());
}

I was playing around with this crate (pretty cool btw!). I'm trying to make the wasd keys move the current selected entity. My approach has been to make a system with mut query: Query<&mut SelectablePickMesh>, and then iterate over the query and check for selected, but it turns out it's a private field and it can't be accessed.

Is there another way of doing what I'm trying to do? Or should selected be marked as public?

Provide a cursor vector in world space, both normal to screen and cursor motion direction, useful for manipulating objects in 3D.

I have a CameraRig Entity and the Camera Entity (the pick source) is a child to it. So when I move my CameraRig the GlobalTransform of the Camera changes but not the Transform. This leads to wrong raycasts.

How to replicate:

1. Clone https://github.com/SarthakSingh31/bevy_4x_camera/tree/follow.
2. Run cargo run --release.
3. Move the camera forwards with W, keep the white moving box in view.
4. Try clicking on the white moving box after the camera has been moved forwards.

I tried changing this Transform to GlobalTransform in a local copy and that fixes the issue.
I don't know if this is the best solution as a non-camera pick source might not have a GlobalTransform component.

More granular picking updates. May require upstream work in bevy_mod_raycast.