geekrelief / gdnim Goto Github PK

View Code? Open in Web Editor NEW

55.0 3.0 4.0 546 KB

godot-nim based bootstrapping framework supporting hot reloading

License: Other

Nim 99.89% GDScript 0.11%

godot nim godot-nim

gdnim's People

Contributors

Stargazers

Watchers

Forkers

terkwood rattleycooper jrenner winterpixelgames

gdnim's Issues

Something in `Watcher` is causing instability in ORC. Crash occurs when generating thousands of node instances with hot reloading.

Getting this random crash after hot reloading a bunch of times. Looks like nim ORC is trying either call a bad function in trace or gets bad data.

nim version: 3b963a8150e3c541859931a1ef9ecb338e46907f

ERROR: NativeScriptInstance detected crash on method: register_instance
   at: NativeScriptInstance::notification (modules\gdnative\nativescript\nativescript.cpp:735)
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[0] trace__Pgw8McKUNuhxTelmFLa61gsystem (C:\nim\lib\system\orc.nim:81)
[1] scanBlack__t5JPtD1AT5y7bGV1vv4KJA_3 (C:\nim\lib\system\orc.nim:164)
[2] scan__t5JPtD1AT5y7bGV1vv4KJA_2 (C:\nim\lib\system\orc.nim:240)
[3] collectCyclesBacon__cKo5xgNSZEVlhzdcplUDNA (C:\nim\lib\system\orc.nim:319)
[4] collectCycles__amVlU9ajqZ06ujoesRBHcDg_2 (C:\nim\lib\system\orc.nim:372)
[5] registerCycle__W1emNvKsDKJ8tfxEiEnOvg (C:\nim\lib\system\orc.nim:402)
[6] rememberCycle__LoYD9cYK9aJvrcDizBN64qaQ (C:\nim\lib\system\orc.nim:454)
[7] nimDecRefIsLastCyclicDyn (C:\nim\lib\system\orc.nim:465)
[8] eqdestroy___hwjDpahwYF1ImTW9axXRumw (C:\godot\gdnim\deps\godot\nim\godotmacros.nim:708)
[9] createFunc__dTdpNAKWMlJpoMqkqbdcTg (C:\godot\gdnim\deps\godot\nim\godotmacros.nim:709)
[10] main
[11] main
[12] main
[13] main
[14] call__Fcqhs1hhgvwTXPioAUEPjAgodotinternal (C:\godot\gdnim\deps\godot\godotinternal.nim:25)
[15] newOwnObj__9aMXOwroTmREicc52gFImkQ (C:\godot\gdnim\deps\godot\nim\godotnim.nim:410)
[16] gdnew__saqRYK8rD2rR1fcQIiqmdQ (C:\godot\gdnim\deps\godot\nim\godotnim.nim:437)
[17] register_instance__GU3pQzEYT1yfdTsRwpZS9bw (C:\godot\gdnim\gdnim\watcher.nim:319)

Wasm

Dear developer! Please tell me how to compile the code in wasm. And if it's not difficult to write more detailed instructions for integrating gdnim.

could not load: pcre64.dll

Windows 10, gcc

after compiling nim c build i tried to ./build help, but console says: could not load: pcre64.dll

Better hot reloading workflow?

While working with hot reloading here are some issues I've come across and some ideas on how to improve the workflow.

Serializing - Can we make reloading data easier by serializing all the data in the component?
- Currently, you need to save() before you can load(), and often you don't know what you need to save before you reload.
- Can we uses inst2dict for this?
Handling crashes / nil access - Can prevent a bad component from taking down the whole app?
- When recompiling a component, an error like nil access can occur which takes down the app.
- Ideally, the error would be caught, the component unloaded, and the app paused.
- Watcher would then just wait until you recompile and attempt to reload and continue.
Dependencies - Can we make it easier to clear references to reloading nodes?
- Maybe a macro can be created toconnect to a node see if it's reloading. That would require the reloading node to emit a signal to distinguish it from exiting the tree normally.
- Right now the watcher emits a signal when an instance reloads, but anything that binds to it will be receiving the signal for all instances and have to filter for the node it cares about.
- Unloading is easy, but what about reloading the references? Watcher emits a signal for the dependents to reload their dependency references.

Mac support

Started work on support mac builds, but arun had issues with getting things running.

Can we add a monitor task to build to auto compile on saves?

Try out creating a script to monitor the components folder for changes to auto build.

Nim devel breaks godot-nim

Merge core files so we don't have circular dependencies between modules.

Should we include an option to not set godot's path manually in build.ini, but if there is a GODOT_BIN_PATH use that?
GodotNim uses this environment variable too, so it might lower the entry just a lil' bit.

Null function pointers are returned from GDNativeAPI leading to crash. Object.get / Object.set

pragmagic/godot-nim#86

The workaround is to modify godotapi/objects.nim so that method get on line 79 retrieves get instead of _get and the same goes for method set online 127.

Explore https://github.com/godotengine/godot/pull/44879

Godot editor reports resource leaked if main_screen and/or custom_dock plugins enabled

I am not sure this should report to godot-nim or gdnim.

Godot editor reports resource leaked, if main_screen and/or custom_dock enabled. No matter building with arc/orc/realtime gc.

WARNING: cleanup: ObjectDB instances leaked at exit (run with --verbose for details).
   At: core/object.cpp:2132.
Leaked instance: ImageTexture:3216 - Resource path:
Leaked instance: SceneState:17848
Leaked instance: GDScript:17861 - Resource path: res://gdscripts/main_scene_button.gd
Leaked instance: PackedScene:17863 - Resource path: res://addons/main_screen/main_screen.tscn
Leaked instance: Image:3217 - Resource path:
Leaked instance: SceneState:17864
Leaked instance: GDScriptNativeClass:717
Leaked instance: PackedScene:17847 - Resource path: res://addons/custom_dock/custom_dock.tscn
Leaked instance: Button:17850 - Node name: Custom Dock
Hint: Leaked instances typically happen when nodes are removed from the scene tree (with `remove_child()`) but not freed (with `free()` or `queue_free()`).
ERROR: clear: Resources still in use at exit (run with --verbose for details).
   At: core/resource.cpp:450.
Resource still in use: res://addons/custom_dock/custom_dock.tscn (PackedScene)
Resource still in use: res://addons/main_screen/main_screen.tscn (PackedScene)
Resource still in use: res://gdscripts/main_scene_button.gd (GDScript)
Orphan StringName: res://gdscripts/main_scene_button.gd
Orphan StringName: anchor_right
Orphan StringName: margin_bottom
Orphan StringName: __meta__
Orphan StringName: ImageTexture
Orphan StringName: CenterContainer
Orphan StringName: GDScriptNativeClass
Orphan StringName: size_flags_vertical
Orphan StringName: PackedScene
Orphan StringName: .
Orphan StringName: margin_left
Orphan StringName: Custom Dock
Orphan StringName: text
Orphan StringName: anchor_bottom
Orphan StringName: SceneState
Orphan StringName: Button
Orphan StringName: pressed
Orphan StringName: Image
Orphan StringName: margin_right
Orphan StringName: _on_PrintHello_pressed
Orphan StringName: _init
Orphan StringName: GDScript
Orphan StringName: margin_top
Orphan StringName: script
StringName: 24 unclaimed string names at exit.
ERROR: cleanup: There are still MemoryPool allocs in use at exit!
   At: core/pool_vector.cpp:66.

`./build prereqs` requires a compiled build already?

I was setting up gdnim on a new user on my Ubuntu distro. And to nim c build I have to nimble install the compiler and anycase packages. Prereqs does this already but that's...kind of backwards haha?

Maybe do prereqs as a (Power)shell script for now and leverage Nimble/Nimph eventually(tm)?

issues with final executable

none of the nim code seems to run when exporting the project using the godot editor. the dll files are exported but running the exe just shows a static scene

While exiting from editor (with main_screen and custom_dock enabled), arc and orc segfault

I am not sure if this is caused by godot-nim, gdnim or nim. Here is the point where orc segfault

[Thread 0x7fff88b32700 (LWP 1427759) exited]
--Type <RET> for more, q to quit, c to continue without paging--

Thread 1 "godot.x11.opt.t" received signal SIGSEGV, Segmentation fault.
0x00007fffacf5e48f in unregisterCycle__rR8fldvW9aUfvKydORzL1RA (
    s=s@entry=0x7fff895030d0) at Projects/Nim/lib/system/orc.nim:121
121       roots.d[idx] = roots.d[roots.len-1]
(gdb) list
116       let idx = s.rootIdx
117       when false:
118         if idx >= roots.len or idx < 0:
119           cprintf("[Bug!] %ld\n", idx)
120           quit 1
121       roots.d[idx] = roots.d[roots.len-1]
122       roots.d[idx][0].rootIdx = idx
123       dec roots.len
124
125     proc scanBlack(s: Cell; desc: PNimTypeV2; j: var GcEnv) =
(gdb) bt
#0  0x00007fffacf5e48f in unregisterCycle__rR8fldvW9aUfvKydORzL1RA (s=s@entry=0x7fff895030d0) at Projects/Nim/lib/system/orc.nim:121
#1  0x00007fffacf5cbea in rememberCycle__LoYD9cYK9aJvrcDizBN64qaQ (isDestroyAction=<optimized out>, s=0x7fff895030d0, desc=0x7fff897b6860)
    at Projects/Nim/lib/system/orc.nim:386
#2  0x00007fffad021055 in nimDecRefIsLastCyclicDyn (p=0x7fff895030e0) at Projects/Nim/lib/system/orc.nim:405
#3  eqsink___OnHyekGmA1V9brXIsCyD03A_2 (dest=<optimized out>, src=<optimized out>) at Projects/Nim/lib/system.nim:866
#4  0x00007fffad0fe792 in exit_tree__RzDwTf9aVZcWepHLsVSlUrA_2 (self=self@entry=0x7fff89505050) at Projects/Nim/lib/system.nim:866
#5  0x00007fffad0fe8d0 in methFunc__KncVdzu9bnd2yp8CXfumwCQ (objX60gensym3_=<optimized out>, methodDataX60gensym3_=<optimized out>, userDataX60gensym3_=0x7fff89505050,
    numArgsX60gensym3_=0, argsX60gensym3_=<optimized out>) at gdnim/deps/godot/nim/godotmacros.nim:869
#6  0x0000000002e6c710 in NativeScriptInstance::call_multilevel (this=0xa9a6840, p_method=..., p_args=0x0, p_argcount=0)
    at modules/gdnative/nativescript/nativescript.cpp:871
#7  0x0000000001de4b40 in Node::_propagate_exit_tree (this=0xa9be3a0) at ./scene/scene_string_names.h:52
#8  0x0000000001de4ab3 in Node::_propagate_exit_tree (this=0x643a200) at ./core/cowdata.h:154
#9  0x0000000001de4ab3 in Node::_propagate_exit_tree (this=0x62ca6e0) at ./core/cowdata.h:154
#10 0x0000000001de5a5f in Node::_set_tree (this=0x62ca6e0, p_tree=0x0) at scene/main/node.cpp:2626
#11 0x0000000001de5b45 in SceneTree::finish (this=0x538b5d0) at scene/main/scene_tree.cpp:623
#12 0x00000000009a7afa in OS_X11::run (this=<optimized out>) at platform/x11/os_x11.cpp:3642
#13 OS_X11::run (this=0x7fffffffced0) at platform/x11/os_x11.cpp:3618
#14 main (argc=<optimized out>, argv=<optimized out>) at platform/x11/godot_x11.cpp:56

This one is caused by arc

[Thread 0x7fffa440e700 (LWP 1565407) exited]

Thread 1 "godot.x11.opt.t" received signal SIGSEGV, Segmentation fault.
0x00007fffacf4da4b in nimDestroyAndDispose (p=0x7fff88b390d8) at Projects/Nim/lib/system/arc.nim:168
168       if rti.destructor != nil:
(gdb) list
163     template `=dispose`*[T](x: owned(ref T)) = nimRawDispose(cast[pointer](x), T.alignOf)
164     #proc dispose*(x: pointer) = nimRawDispose(x)
165
166     proc nimDestroyAndDispose(p: pointer) {.compilerRtl, raises: [].} =
167       let rti = cast[ptr PNimTypeV2](p)
168       if rti.destructor != nil:
169         cast[DestructorProc](rti.destructor)(p)
170       when false:
171         cstderr.rawWrite cast[ptr PNimTypeV2](p)[].name
172         cstderr.rawWrite "\n"
(gdb) bt
#0  0x00007fffacf4da4b in nimDestroyAndDispose (p=0x7fff88b390d8) at Projects/Nim/lib/system/arc.nim:168
#1  0x00007fffad003803 in eqsink___OnHyekGmA1V9brXIsCyD03A_2 (dest=0x7fff88b39068, src=0x0) at Projects/Nim/lib/system.nim:868
#2  0x00007fffad0de9db in exit_tree__RzDwTf9aVZcWepHLsVSlUrA_2 (self=self@entry=0x7fff88b39048) at Projects/Nim/lib/system.nim:866
#3  0x00007fffad0deb19 in methFunc__KncVdzu9bnd2yp8CXfumwCQ (objX60gensym3_=<optimized out>, methodDataX60gensym3_=<optimized out>, userDataX60gensym3_=0x7fff88b39048, 
    numArgsX60gensym3_=0, argsX60gensym3_=<optimized out>) at gdnim/deps/godot/nim/godotmacros.nim:869
#4  0x0000000002e6c710 in NativeScriptInstance::call_multilevel (this=0x877b160, p_method=..., p_args=0x0, p_argcount=0)
    at modules/gdnative/nativescript/nativescript.cpp:871
#5  0x0000000001de4b40 in Node::_propagate_exit_tree (this=0xa9be6f0) at ./scene/scene_string_names.h:52
#6  0x0000000001de4ab3 in Node::_propagate_exit_tree (this=0x6439530) at ./core/cowdata.h:154
#7  0x0000000001de4ab3 in Node::_propagate_exit_tree (this=0x62c97a0) at ./core/cowdata.h:154
#8  0x0000000001de5a5f in Node::_set_tree (this=0x62c97a0, p_tree=0x0) at scene/main/node.cpp:2626
#9  0x0000000001de5b45 in SceneTree::finish (this=0x62c58c0) at scene/main/scene_tree.cpp:623
#10 0x00000000009a7afa in OS_X11::run (this=<optimized out>) at platform/x11/os_x11.cpp:3642
#11 OS_X11::run (this=0x7fffffffced0) at platform/x11/os_x11.cpp:3618
#12 main (argc=<optimized out>, argv=<optimized out>) at platform/x11/godot_x11.cpp:56

The realtime gc can exit without segfault.

To reproduce

# on Ubuntu 20.04
cd gdnim
git checkout ff694b612a
./build cleanbuild
godot.x11.opt.tools.64 -e --path app --verbose
# enable main_screen and custom_dock plugin, then
# exit from editor

nim async generates cyclic garbage

https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc-in-nim.html
Unfortunately, enabling orc crashes gdnim when nodes are destroyed.

Might need to implement a custom async solution that doesn't generate garbage.

Crash with gc:orc and useMalloc when reloading several times.

When ORC is enabled and nodes are destroyed the app will crash. Your game should avoid creating cyclic garbage, but if that's not possible just use pragmagic's godot-nim and use gc:refc

Hot reload memory leak

There's a small memory leak that occurs on hot reload. When reloading bullet.nim, memory increases by about 0.2MB when looking at the TaskManager on Windows. I don't know if this is happening inside Godot(GDNative initializing) or gdnim (Watcher). There doesn't seem to be any memory leak without hot reloading.

Error: unhandled exception: invalid integer: [ValueError]

After writting any task in console i see this error:

Error: unhandled exception: invalid integer: [ValueError]

method assumes virtual methods always start with an underscore

Example:
EditorPlugin.has_main_screen is a virtual method, but does not have an underscore.

gdnim/deps/godot/nim/godotmacros.nim

Line 186 in 4d17778

isVirtual: meth.kind == nnkMethodDef,

use of method on a proc assumes it's virtual and

gdnim/deps/godot/nim/godotmacros.nim

Line 782 in 4d17778

let godotMethodName = if meth.isVirtual: "_" & toGodotStyle(meth.name)

prepends an underscore automatically.

As a work around to override the virtual function you declare it as proc with gdExport like:

proc hasMainScreen():bool {.gdExport.} =
  true

gdnim/utils.nim#loadScene should execute at compileTime

loadScene is a proc that uses findScene which calls walkDir. On Android (maybe iOS too) we can't assume access to the filesystem since the exported library is placed in a different file structure compared when we're developing in the editor.

loadScene should be a template and findScene should be a macro to execute at compile time.

Support on nim devel

4ec2f74246158759735149e3dd087f373fd787b2 can crash if hot reload occurs. It can crash on reload or when the program closes.

[0] unregisterCycle_system_2933 (C:\nim\lib\system\orc.nim:146)
[1] rememberCycle_system_3292 (C:\nim\lib\system\orc.nim:467)
[2] nimDecRefIsLastCyclicDyn (C:\nim\lib\system\orc.nim:484)
[3] eqdestroy__OOZdepsZgodotZnimZgodotnim_2019 (C:\godot\gdnim\deps\godot\nim\godotnim.nim:196)
[4] removeGodotObject_OOZdepsZgodotZnimZgodotnim_2271 (C:\godot\gdnim\deps\godot\nim\godotnim.nim:197)
[5] nimDestroyFunc_OOZdepsZgodotZnimZgodotmacros_2512 (C:\godot\gdnim\deps\godot\nim\godotmacros.nim:516)
[6] NativeScriptInstance::`scalar deleting destructor'
[7] Object::~Object (C:\godot\geekrelief_godot\core\object.cpp:1955)
[8] Sprite::`scalar deleting destructor'
[9] Node::_notification (C:\godot\geekrelief_godot\scene\main\node.cpp:168)
[10] predelete_handler (C:\godot\geekrelief_godot\core\object.cpp:2000)
[11] SceneTree::_flush_delete_queue (C:\godot\geekrelief_godot\scene\main\scene_tree.cpp:1086)
[12] SceneTree::idle (C:\godot\geekrelief_godot\scene\main\scene_tree.cpp:545)
[13] Main::iteration (C:\godot\geekrelief_godot\main\main.cpp:2056)
[14] OS_Windows::run (C:\godot\geekrelief_godot\platform\windows\os_windows.cpp:3307)
[15] widechar_main (C:\godot\geekrelief_godot\platform\windows\godot_windows.cpp:161)
[16] _main (C:\godot\geekrelief_godot\platform\windows\godot_windows.cpp:185)
[17] main (C:\godot\geekrelief_godot\platform\windows\godot_windows.cpp:195)
[18] __scrt_common_main_seh (D:\agent\_work\10\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288)
[19] BaseThreadInitThunk
-- END OF BACKTRACE --

commit 3b963a81 is still faster to compile.

Error: type expected, but got symbol 'Object' of kind 'enumField'

OS: Windows 10
Compiler: gcc
nim 2.0.0

error when building project with build cleanbuild

Discord channel

You've read the docs and still find something confusing or broken?
Have questions regarding gdnim, godot-nim, nim, godot?
Have suggestions on how to make the gdnim better?
Want to show off something cool you've built with gdnim?

https://discord.gg/p4mUP9JC6w

nimscript?

https://github.com/dsrw/enu uses the nim interpreter to implement a DSL. It creates an interpreter per script. Not sure how performant this is, but it could be useful for quickly testing out ideas.

how to embed nimscript:
https://peterme.net/how-to-embed-nimscript-into-a-nim-program-embedding-nimscript-pt-2.html

Clean install, `nim c build` needs compiler package

Fix nimble install compiler before nim c build, include compiler into deps, or rewrite build script with nimble, or move task genapi to a different file.

question？

Can this be used to develop a hot update for a game? Check whether the server has the latest version at the beginning of the game. If so, download the latest content and restart the game. Start the program again to see the new content.

Get async working on signals

We want to be able to yield on a signal the way gdscript does it. I have some starter code that implements this so far:

import godot
import godotapi / [timer, scene_tree]
import hot
import asyncdispatch

gdobj TimerComp of Timer:

  method enter_tree() =
    discard register(timer_comp)

  proc reload():seq[byte] {.gdExport.} =
    self.queue_free()

  method ready() =
    self.one_shot = true
    registerFrameCallback(
      proc() =
        if hasPendingOperations():
          poll(0)
    )
    asyncCheck self.fireTimer()

#[
  proc fire_timer() {.async.} =
    for c in "STARWARS: A LONG TIME AGO IN A GALAXY FAR, FAR AWAY":
      self.one_shot = true
      self.start(0.15)
      onSignal(self, "timeout")
      print c
]#

  proc fire_timer() {.async.} =
    await on_signal(self.get_tree().createTimer(1), "timeout")
    print "timeout sceneTree"
    self.start(1)
    await on_signal(self, "timeout")
    print "timeout 1"

But I need to handle various expressions and statements where on_signal could be inside of like a for loop, if / else statements, etc. Also I need to implement support for returning values from the signal.

I decided to have the user use the await keyword to wait on the Future from on_signal to allow for composability. I want to be able to use or, and, all, etc from the asyncfutures module.

I'm still undecided on whether to auto generate the registerFrameCallback call somehow or just have a function for it. You need a call to poll for each dll that has Futures. But if you include multiple modules into a dll, you probably only need one Node to poll.