Comments (12)
ryansturmer
commented on May 27, 2024
This testing was performed on the current edge branch.
from g2.
ryansturmer
commented on May 27, 2024
g2 configuration:
https://gist.github.com/ryansturmer/58c4d2aabfe1fba093ab
from g2.
ryansturmer
commented on May 27, 2024
Running this a second time, with the same G2 configuration failed on a much earlier line, with a repeated status "Failed to get planner buffer" - Feels like it might be a memory issue?
from g2.
ryansturmer
commented on May 27, 2024
Incidentally, I had turned up the logging for that second run. The "failed to get planner" could have been happening on the first run too, and I wouldn't have seen it.
from g2.
ryansturmer
commented on May 27, 2024
So yet another one bites the dust, same circumstances. I am running this one headless (no CNC machine) and running it from my macbook rather than from the SBC. This time, the error message sent back is garbled (but not immediately, the error was reported dozens of times before the first garbage message came in. After the garbage message, the system stopped responding.)
from g2.
ryansturmer
commented on May 27, 2024
Tried one last time with this, using a simple file that just draws a circle over and over again at 2ips. Confirmed the same behavior. File seems to run fine, then grinds to a halt and "failed to get planner buffer" until those messages eventually degrade into garbage.
I did this from a fresh build and load of the G2 firmware, which is a clean copy from here, just to be sure:
Built and tested from that version of the build
PLATFORM=shopbotShield
SETTINGS_FILE=settings_default.h
from g2.
ryansturmer
commented on May 27, 2024
The issue appears to be that a hard alarm is called in the motion planner that provokes an infinite recursion that makes the system asplode.
It appears that the planner runs into a full buffer here:
https://github.com/ShopBotTools/FabMo-G2-Core/blob/edge/TinyG2/planner.cpp#l211
Provoking a shutdown of the spindle here:
https://github.com/ShopBotTools/FabMo-G2-Core/blob/edge/TinyG2/canonical_machine.cpp#l641
Which calls the mp_queue_command function again, here:
https://github.com/ShopBotTools/FabMo-G2-Core/blob/edge/TinyG2/spindle.cpp#L100
Which triggers another alarm, causing the system to go into an infinite recursion.
What I don't understand is how the planner is getting to the full state when it's supposed to be checked ouside the mp_queue_command function.
I am continuing to dig!
from g2.
ryansturmer
commented on May 27, 2024
At the time of the crash, the number of motion buffers as reported by mb.buffers_available is typically nonzero - in the vicinity of 6 or 7, and the buffer referenced by mb.w is pointing to a buffer in the RUNNING (4) state. This comment will be followed by some output from gdb that indicates the state of the ring buffer.
from g2.
ryansturmer
commented on May 27, 2024
This at least gives the state of the buffer at the time of the alarm:
Breakpoint 1, mp_queue_command (cm_exec=0x8da61 <_exec_spindle_control(float*, float*)>, value=0x20087df0, flag=0x20087df0) at planner.cpp:211
211 cm_hard_alarm(STAT_BUFFER_FULL_FATAL);
(gdb) print mb.w
$1 = (mpBuf_t *) 0x200720b0 <mb+3296>
(gdb) print mb.r
$2 = (mpBuf_t *) 0x20072578 <mb+4520>
(gdb) set $i = 0
(gdb) print mb.q
$3 = (mpBuf_t *) 0x200720b0 <mb+3296>
(gdb) print mb.bf[$i++].buffer_state
$4 = 2 '\002'
(gdb)
$5 = 2 '\002'
(gdb)
$6 = 2 '\002'
(gdb)
$7 = 2 '\002'
(gdb)
$8 = 2 '\002'
(gdb)
$9 = 2 '\002'
(gdb)
$10 = 2 '\002'
(gdb)
$11 = 2 '\002'
(gdb)
$12 = 2 '\002'
(gdb)
$13 = 2 '\002'
(gdb)
$14 = 2 '\002'
(gdb)
$15 = 2 '\002'
(gdb)
$16 = 1 '\001'
(gdb)
$17 = 1 '\001'
(gdb)
$18 = 1 '\001'
(gdb)
$19 = 1 '\001'
(gdb)
$20 = 4 '\004'
(gdb)
$21 = 0 '\000'
(gdb)
$22 = 0 '\000'
(gdb)
$23 = 0 '\000'
(gdb)
$24 = 0 '\000'
(gdb)
$25 = 0 '\000'
(gdb)
$26 = 3 '\003'
(gdb)
$27 = 2 '\002'
(gdb)
$28 = 2 '\002'
(gdb)
$29 = 2 '\002'
(gdb)
$30 = 2 '\002'
(gdb)
$31 = 2 '\002'
(gdb) print mb.w.buffer_state
$32 = 4 '\004'
(gdb) print mb.r.buffer_state
$33 = 3 '\003'
(gdb) print mb.q.buffer_state
$34 = 4 '\004'
(gdb)
from g2.
ryansturmer
commented on May 27, 2024
Further developments. I have re-run essentially the same file, with different results (I am running an updated build with fixes.)
First of all, I am running the following build:
FabMo/FabMo-G2-Core@1c9a592
The geometry of the file is the same as above, and the result is thus:
In the file above, the pocket (green) is done first, and more or less looks correct. Then the cutouts for the three holes (blue) - also looks correct. Finally, the profile of the outside edge, which is a cutout, and has four tabs, which you can't really see in the diagram. This starts out pink, in the correct position but then stopped at the red X, moving at a very slow pace along the trajectory of the cyan line, until it picked back up again in speed and in the correct toolpath geometry, although it had lost position at this point. The trajectory of the errant segment appears to be that of the segment it was cutting at the moment of the failure, and it appears to be both in X and Y (although this is difficult to discern. You can see that the cut is very slow, as the material surrounding the profile is burned.
Once the cut resumed the geometry was more or less correct, but with an offset as shown. When the cycle completed, the reported positions appeared to be correct - that is, on a second run of the file, the toolpaths started out in, at least approximately, the correct position. No offset was seen in the initial passes of the file when run a second time. The second run of the file appeared to lose position as well, but not in the same manner as the first.
On the second run of the file, (which I did as an air-cut above the table) There appeared to be some queue starvation, especially in the cutting of the outline, which appeared to be very jittery. I am looking into this, but I'm not sure why this would cause a loss of position, especially with symptoms such as seen with the cyan line above.
File analysis doesn't show anything weird, except perhaps for some exceptionally short moves. There are a number of moves that are less than 1mil, which are detailed below, with the format length, line number
I unfortunately do not have logs for all traffic, but my logs do not show any errors returned from g2, and they don't indicate a starvation condition. (System streams more lines when the number of lines left unprocessed drops below 250, so there's plenty of margin for communication delay)
0.0005000000000007888,2910
0.0006708203932503595,3670
0.0005385164807141745,3740
0.000632455532034168,3742
0.0005830951894839327,3818
0.0005830951894851513,3835
0.0005385164807140096,4106
0.000300000000000189,4110
0.0006324555320340275,4183
0.0005385164807140096,4185
0.0005385164807140096,4271
0.0005830951894843135,4288
0.0004123105625620976,4290
0.00044721359550050445,4896
0.00039999999999995595,5245
0.0004123105625616668,5508
0.000300000000000189,5872
0.0002999999999997449,6134
0.0005000000000007888,9044
0.0006708203932503595,9804
0.0005385164807141745,9874
0.000632455532034168,9876
0.0005830951894839327,9952
0.0005830951894851513,9969
0.0005385164807140096,10240
0.000300000000000189,10244
0.0006324555320340275,10317
0.0005385164807140096,10319
0.0005385164807140096,10405
0.0005830951894843135,10422
0.0004123105625620976,10424
0.00044721359550050445,11030
0.00039999999999995595,11379
0.0004123105625616668,11642
0.000300000000000189,12006
0.0002999999999997449,12268
0.00010000000000065512,17971
0.00028284271247458785,19049
0.0007211102550930879,19050
0.000360555127547283,20237
0.0005656854249487047,20238
0.0006403124237437345,20239
0.0005656854249498038,20240
0.0006403124237423474,20241
0.0006403124237437345,20242
0.0006403124237423474,20243
0.0007810249675910627,20244
0.0006403124237437345,20245
0.0007810249675909206,20246
0.0007211102550930879,20247
0.0007810249675896981,20248
0.0007211102550930879,20249
0.0007810249675909206,20250
0.0007810249675909206,20251
0.0008602325267044776,20252
0.0007810249675895558,20253
0.0008602325267043487,20254
0.0008602325267043487,20255
0.0008602325267044776,20256
0.0008602325267043487,20257
0.0008602325267043487,20258
0.0008602325267043487,20259
0.0008602325267043487,20261
0.0009433981132055564,20262
0.0009999999999998899,20263
0.0009433981132056741,20264
0.0009999999999998899,20265
0.0009433981132055564,20266
0.0009999999999998899,20267
0.0009999999999998899,20269
0.0009999999999998899,20270
0.000854400374531737,20524
0.000700000000000145,20739
0.00010000000000065512,21565
0.00028284271247458785,22645
0.0007211102550930879,22646
0.000360555127547283,23835
0.0005656854249487047,23836
0.0006403124237437345,23837
0.0005656854249498038,23838
0.0006403124237423474,23839
0.0006403124237437345,23840
0.0006403124237423474,23841
0.0007810249675910627,23842
0.0006403124237437345,23843
0.0007810249675909206,23844
0.0007211102550930879,23845
0.0007810249675896981,23846
0.0007211102550930879,23847
0.0007810249675909206,23848
0.0007810249675909206,23849
0.0008602325267044776,23850
0.0007810249675895558,23851
0.0008602325267043487,23852
0.0008602325267043487,23853
0.0008602325267044776,23854
0.0008602325267043487,23855
0.0008602325267043487,23856
0.0008602325267043487,23857
0.0008602325267043487,23859
0.0009433981132055564,23860
0.0009999999999998899,23861
0.0009433981132056741,23862
0.0009999999999998899,23863
0.0009433981132055564,23864
0.0009999999999998899,23865
0.0009999999999998899,23867
0.0009999999999998899,23868
0.000854400374531737,24122
0.000700000000000145,24338
0.00010000000000065512,25165
0.00028284271247458785,26245
0.0007211102550930879,26246
0.000360555127547283,27435
0.0005656854249487047,27436
0.0006403124237437345,27437
0.0005656854249498038,27438
0.0006403124237423474,27439
0.0006403124237437345,27440
0.0006403124237423474,27441
0.0007810249675910627,27442
0.0006403124237437345,27443
0.0007810249675909206,27444
0.0007211102550930879,27445
0.0007810249675896981,27446
0.0007211102550930879,27447
0.0007810249675909206,27448
0.0007810249675909206,27449
0.0008602325267044776,27450
0.0007810249675895558,27451
0.0008602325267043487,27452
0.0008602325267043487,27453
0.0008602325267044776,27454
0.0008602325267043487,27455
0.0008602325267043487,27456
0.0008602325267043487,27457
0.0008602325267043487,27459
0.0009433981132055564,27460
0.0009999999999998899,27461
0.0009433981132056741,27462
0.0009999999999998899,27463
0.0009433981132055564,27464
0.0009999999999998899,27465
0.0009999999999998899,27467
0.0009999999999998899,27468
0.000854400374531737,27722
0.000700000000000145,27938
from g2.
giseburt
commented on May 27, 2024
@ryansturmer Did you determine that the cause of this was static from your dust collection? Or is there still an issue here?
from g2.
giseburt
commented on May 27, 2024
@ryansturmer I marked this as invalid and am closing.
To everyone else out there: make sure you have protection against static build up! In particular, ground the dust collection hose, the limit switches, and the toolhead (where possible and safe).
The results of static discharge can be random and seemingly almost-sane behavior. In this case, it was causing it to seemingly randomly skip executing a few instructions, but then resume where it should.
from g2.
Related Issues (20)
- Spiral roughing hiccup? HOT 4
- Not emitting steps at shallow incline. HOT 7
- Skew correction ideas? HOT 4
- G-codes HOT 1
- Error Compiling: Windows 10 Atmel Studio HOT 1
- Spindle sometimes does not enable HOT 3
- 9 axis external driver board HOT 1
- Soft Limits not setting alarm state. Not stopping job. HOT 1
- Is the g2core project still active? HOT 1
- feedrate interpretation for moves involving both, primary axes (XYZ) and secondary linear axes (UVW) differs from LinuxCNC
- Arduino Due with Edge-preview+gShield not working ///'laser_tool' was not declared in this scope/// HOT 4
- Status report not returning "pos" or "mpo" for A-axis HOT 1
- 101.03 branch HOT 3
- UART on g2core
- Spindle Enable Pin Not Functioning HOT 1
- Compile g2core for gShield on Windows HOT 3
- Endstops behaviour HOT 9
- Which version? HOT 15
- Is G2core a good choice for my app?
- Project status HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from g2.