Question/Thoughts?: Module Unload - Systems Unregister about flecs HOT 10 CLOSED

As @SanderMertens while ago said that deleting systems is not that simple, but indeed I also wanna this feature. @SanderMertens recommends world recreation(delete/create), but I not fan on this approach.

from flecs.

@teksoc-1 the main problem with a feature such as this is not the unloading/reloading of systems (although it would be a lot of work, and potentially a very expensive operation) but the unloading/reloading of components.

If you imported a module, then unload it, then reimport it with a changed component definition, it could cause serious trouble when there are entities in the world that still use the component. There is no "component migration" feature, and this would be hard to get right.

The second biggest problem is that modules are not the same as shared libraries. Modules are meant as tools to organize your code, and it is up to the developer to decide where the module should live. This can be in the same process, a static library, a dynamic library linked at link time, or a dynamic library linked at runtime. For each of these different scenarios, a module load/unload could mean a different thing.

This request has come up before though, so it's obviously something that people are interested in. If we can agree on clear semantics for a module unload/reload, while addressing the above issues, it is something I can add to Flecs.

from flecs.

@ArnCarveris Yes, it does look like that may require some heavy lifting to properly 'delete' a system ( and leaves perhaps some unresolved questions on what to do with respective tables? ) ... I was however ever to work out a PoC that functionally when a system is registered, doing a string copy vice direct assignment for the function name and signature ( so ecs_lookup() and friends don't blow up post unload of the module ), and disabling the respective systems on unload ... and when loading a new version of the module to honor the new system registrations if and only if the signature matched as the previously registered system by the same name. This approach seems to functionally work, but it obviously requires messing with private data structures that are not really intended to be messed with, and maybe a bad idea all together ...

from flecs.

@SanderMertens yes, recognize the binding contract of one module leveraging the data structures (which in turn become Components) declared in another module's header ... and the revisioning/dependency management that would have to occur to ensure module compatibility may be beyond the scope, and perhaps against the intended design approach and purpose of modules as you shared above.

from flecs.

@SanderMertens so internally, the approach that we are taking/trying out, is that with the understanding that the binding contract is on the data structures (component definitions) in a module header, we are providing a Major.Minor revisioning mechanism for modules ( Major revisions are destructive in that they change existing component definitions, and Minor revisions are non destructive in that they add new component definitions for example ) ... we have preflight to check the current revisioning info on a module ... if the Major revision number is greater than what is currently loaded, then gracefully punt/fail on attempting to load the respective module (simply not attempting to address the component migration issue as you described) ... if just a minor upgrade, unload previously loaded module ( disabling previously registered systems ), then load the new 'upgraded' module which may register new systems, re-enable previously registered systems with new action callback (only maintaining signature consistency with previously registered systems - gracefully punting on dealing with non-matched signatures) ... its in a working PoC state ... but readily open to continue the dialog on this, get your feedback, and quite bluntly see if this approach is destined for failure ( or stupid altogether ) in conjunction with direction flecs is going. My team and I are all quite new to ECS as a whole, and flecs is the first implementation we jumped on to try out for its cross platform compatibility among other enticing features (pure C, pure ECS, zero dependencies, etc). We're still working through some of these experimentations, but we think flecs will serve our goals - at least provide a foundation we can build upon. Goes without saying, but appreciate your responsive and action on everything to date.

from flecs.

@teksoc-1 Yes that seems sensible! If you can guarantee that component definitions don't change, you should be able to load/unload modules. You may even be able to just update the function pointers of systems if the system signature does not change.

One word of caution, if you are using Flecs hierarchies, future versions of Flecs will incur some (in some cases significant) overhead if you add/remove/change system signatures that interact with the hierarchy in any way (through CONTAINER or CASCADE columns).

I like your approach, though I am curious as to how you implemented this. Are you just deleting the system entity before reloading it? I have never tested this in a running application.

from flecs.

@SanderMertens So at a high level we have a wrapper interface to the standard flecs api, and post system registrations we reassign system id's and and signatures on the heap so that we can unload later with ecs_lookup() still surviving lookups on the system id's ... when we unload we are simply disabling the systems, vice deleting them, and when loading a 'compatible' upgrade version of the module, we are honoring 'new' registrations of previously registered system with same previous signature, and as you stated, simply updating the function pointers of systems and re-enabling them. We are punting on even attempting to deal with different signatures b/c of unknown side effects, and just not honoring those system registrations all together - and now thinking maybe this should be caught earlier in a preflight check. Anyhow, this rough model seems to functionally work in initial testing and experimentation.

from flecs.

@SanderMertens On the subject of Flecs hierarchies, is there a public API to get all the children of a given parent? I poked around and wasn't seeing it, but seemed like a natural API to have, unless I'm missing something.

from flecs.

@teksoc-1 I created issue #72 for this request. Will get back on it later

from flecs.

You can now unload modules. Deleting the module object will delete all of its contents (including systems & components, if you allow for it, see https://flecs.docsforge.com/master/relations-manual/#relation-cleanup-properties)

If you don't want to delete the systems but just replace them with updated implementations, you can use the ecs_system_init function on an existing system to overwrite the existing (unloaded) callback with a new (hot reloaded) one.

from flecs.