jplevyak / dparser Goto Github PK

A Scannerless GLR parser/parser generater.

Home Page: https://github.com/jplevyak/dparser

License: BSD 3-Clause "New" or "Revised" License

GAP 11.76% C 80.31% Shell 0.56% Python 2.44% Emacs Lisp 0.42% Makefile 0.81% HTML 1.26% Roff 1.94% SWIG 0.06% CMake 0.44%

dparser's Introduction

dparser's People

Contributors

Stargazers

Watchers

Forkers

zcourts ljyss9 ckaran aa10000 stangelandcl g40 moneytech gonzus hosford42 mattfidler phytolizer pombredanne wiegerw ptizoom

dparser's Issues

Nondeterministic result with "if-then-else" expressions

We have used dparser as the parser in our project for years now, but have recently discovered some non-deterministic behaviour from the parser, see this issue.
This has to do with the combination of an "if then" binary operator and an "if then else" ternary operator in our language. The syntax for these operators used by dparser is defined in this file.
Since we could not find support for ternary operators in dparser, the "if then else" operator was defined (after quite some experimentation, done before I personally joined the project) using two unary_op_right keywords, which has worked as desired for years until we found this specific case. I can understand that this might not be a valid way to define such an operator, but I think that the non-deterministic behaviour is concerning regardless. I would expect that either dparser parses the expression the same always or dparser returns an error saying that it has found an ambiguity.

To reproduce the non-deterministic behaviour, you need a specification file test.mcrl2 with the following contents:

proc P = true -> tau.true -> (tau.P) <> P;
init P;

Also you need the mcrl22lps tool. You can download a recent version here with some additional debug output that shows the parsed specification (which is done by adding std::cout << spec; right after this line).

Then if you run the command mcrl22lps --check-only test.mcrl2 repeatedly, you should see differences in the debug output between runs, namely that parentheses are placed differently in the parsed specification, either

proc P = true -> tau . (true -> tau . P <> P);

proc P = true -> tau . (true -> tau . P) <> P;

I'm not exactly sure which version of the dparser is used in our project by default, but I could reproduce this with the latest release of dparser.

Do you think this non-determinism can be fixed or do you have any advice on how to handle ternary operators in dparser?

Add "-Wextra -pedantic -std=c11" to the list of flags

I've been trying to track down why I keep having issues with code building and passing sometimes, and then failing completely when I haven't modified the code. I finally added -Wextra -pedantic -std=c11 to the list of flags to see what is going on. It turned up a LOT of comparisons between signed and unsigned objects, and it turned up a lot of missing field initializers within the generated code. I suspect that some of the code is accessing uninitialized memory and sometimes works by accident, and sometimes fails, depending on what was in that memory location. Can you please look into this? I don't have time to go through all the warnings right now. I'll push my clang-format branch as it is, which should pass the tests at least most of the time, but cleaning up all the warnings is going to take a LONG time.

Tarball 1.30 is broken

Hi!

The current tarball contains two extraneous entries, which are in addition wrong.

$ wget https://freefr.dl.sourceforge.net/dparser/d-1.30-src.tar.gz
--2019-01-08 06:45:03--  https://freefr.dl.sourceforge.net/dparser/d-1.30-src.tar.gz
Résolution de freefr.dl.sourceforge.net (freefr.dl.sourceforge.net)… 88.191.250.136, 2a01:e0d:1:8:58bf:fa88::1
Connexion à freefr.dl.sourceforge.net (freefr.dl.sourceforge.net)|88.191.250.136|:443… connecté.
requête HTTP transmise, en attente de la réponse… 200 OK
Taille : 239510 (234K) [application/x-gzip]
Sauvegarde en : « d-1.30-src.tar.gz »

d-1.30-src.tar.gz                                    100%[===================================================================================================================>] 233,90K  --.-KB/s    ds 0,05s   

2019-01-08 06:45:04 (4,54 MB/s) — « d-1.30-src.tar.gz » sauvegardé [239510/239510]
$ tar tvf d-1.30-src.tar.gz | grep d/my
-rw-rw-r--  0 jplevyak jplevyak    346 23 jan  2013 d/my.c
-rw-rw-r--  0 jplevyak jplevyak    124 23 jan  2013 d/my.g
hrw-rw-r--  0 jplevyak jplevyak      0 23 jan  2013 d/my.c link to d/my.c
hrw-rw-r--  0 jplevyak jplevyak      0 23 jan  2013 d/my.g link to d/my.g

because of this, it cannot be easily extracted:

$ tar xf d-1.30-src.tar.gz
d/my.c: Can't create 'd/my.c'
d/my.g: Can't create 'd/my.g'
tar: Error exit delayed from previous errors.

no support for stdin?

will there every be support for stdin and piece input (eg via input from external lexer) via dparse() , eg

"int"
"a"
";"
NULL

"int a;"

Feature request: Provide column number for syntax errors, too

In the R port for dparser, I wanted to have syntax errors clarified as I was having trouble tracing an issue. I made the following PR there: nlmixr2/dparser-R#13

But, @mattfidler indicated that code was auto-generated from the dparser source. I think that the correct location to make the changes would be here:

dparser/parse.c

Lines 1636 to 1639 in c74f863

 if (after) 

 fprintf(stderr, "%s:%d: syntax error after '%s'\n", fn, p->user.loc.line, after); 

 else 

 fprintf(stderr, "%s:%d: syntax error\n", fn, p->user.loc.line);

But, I'm not really a C programmer, and I wouldn't know how to test what I'm changing, either. So, would it be possible to make the location indicate the line and column?

Version for CRAN?

Thank you John for all of the efforts.

I have used dparser with Wenping for RxODE, and I'm happy with how efficient it is and how well it works.

I have also ported dparser in general to R:

https://github.com/nlmixrdevelopment/dparser-R

I would like to publish it to CRAN and add you as an author.

I was wondering what version of the tree you would consider the most stable. (I currently use the master branch). I also wanted to match the version of dparser available elsewhere. Is that a "released" version?

Just wondering,

Best Regards,

Matt

Errors reported by valgrind

Some easy to fix memory issues reported by valgrind:

diff --git a/gram.c b/gram.c
index 4ef933b..0e7ecd0 100644
--- a/gram.c
+++ b/gram.c
@@ -1431,7 +1431,9 @@ static void print_term_escaped(Term *t, int double_escaped) {
     if (!t->string || !*t->string)
       printf("<EOF> ");
     else {
-      printf("'%s' ", double_escaped ? escape_string_single_quote(s) : s);
+      char *ds = double_escaped ? escape_string_single_quote(s) : s;
+      printf("'%s' ", ds);
+      if(ds != s) FREE(ds);
       if (t->ignore_case) printf("/i ");
       if (t->term_priority) printf("%sterm %d ", double_escaped ? "#" : "$", t->term_priority);
     }
@@ -1439,7 +1441,9 @@ static void print_term_escaped(Term *t, int double_escaped) {
     char *quote = double_escaped ? "\\\"" : "\"";
     s = t->string ? escape_string(t->string) : NULL;
     /* char *s = t->string; // ? escape_string(t->string) : NULL; */
-    printf("%s%s%s ", quote, double_escaped ? escape_string(s) : s, quote);
+    char *ds = double_escaped ? escape_string(s) : s;
+    printf("%s%s%s ", quote, ds, quote);
+    if(ds != s) FREE(ds);
     if (t->ignore_case) printf("/i ");
     if (t->term_priority) printf("%sterm %d ", double_escaped ? "#" : "$", t->term_priority);
   } else if (t->kind == TERM_CODE) {
@@ -1450,7 +1454,7 @@ static void print_term_escaped(Term *t, int double_escaped) {
     printf("%s ", s);
   } else
     d_fail("unknown token kind");
-  if (s) FREE(s);
+  if (s != t->string) FREE(s);
 }
 
 /* print_elem changed to call print_term_escaped */
@@ -1594,7 +1598,7 @@ static void print_declarations(Grammar *g) {
 }
 
 void print_rdebug_grammar(Grammar *g, char *pathname) {
-  char ver[30];
+  char ver[128]; /* the old size was too short */
   d_version(ver);
 
   printf("/*\n  Generated by Make DParser Version %s\n", ver);

is `buildall` supposed to ssh/scp stuff???

I'm reading through the committed executables, and noticed that buildall tries to scp and then ssh to a server. It looks VERY specific to your machines. Should it be removed?

How to run tests? Does not work with CMake, does not work with test_parser

Trying to run tests from CMake runs nothing. Trying the binary, it also does nothing:

--->  Testing dparser
Executing:  cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_dparser/dparser/work/build" && /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_dparser/dparser/work/build/test_parser -vt 
Test DParser Version 1.30.4ab9d43046e5d22ffb365ec5f5184a7cfee09b30 Copyright (c) 2002-2013 John Plevyak
Usage: /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_dparser/dparser/work/build/test_parser [flags|args]
  -l, --longest_match     toggle   false     Use Longest Match Rule for Tokens
  -T, --tokenizer         toggle   false     Tokenizer for START
  -C, --whitespace_states toggle   true      Compute Whitespace States
  -A, --all_states        toggle   false     Compute States For All NTERMs
  -b, --scanner_blocks    int      4         Scanner Blocks
  -p, --op_pri_from_rule  toggle   false     Set Operator Priority From Rule
  -r, --right_recurse_BNF toggle   false     Use Right Recursion For */+
  -S, --start_state       int      0         Start State
  -s, --save_parse_tree   toggle   true      Save Parse Tree
  -p, --partial_parses    toggle   false     Partial Parses
  -c, --compare_stacks    toggle   true      Compare Stacks
  -i, --commit_interval   int      100       Commit Interval
  -f, --fixup             toggle   true      Fixup Internal Productions
  -e, --fixup_ebnf        toggle   false     Fixup EBNF Productions
  -H, --noheight          toggle   false     Do not use Height Disambiguization
  -G, --nogreedy          toggle   false     Do not use Greedy Disambiguization
  -v, --verbose           incr     1         Verbose
  -t, --test              incr     1         Test
  -d, --debug             incr     0         Debug
  -h, --help                                 Help
Command failed:  cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_dparser/dparser/work/build" && /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_dparser/dparser/work/build/test_parser -vt 
Exit code: 1

Usban checks; "Issues" are they important?

I know that USBAN is new to try to reduce runtime errors. I'm unsure if these so-called errors are important, but I thought I would at least report them to you. They are reported as errors in the CRAN project for both gcc and clang.

gcc:

lr.c:180:5: runtime error: null pointer passed as argument 1, which is declared to never be null
lr.c:266:3: runtime error: null pointer passed as argument 1, which is declared to never be null
lr.c:392:5: runtime error: null pointer passed as argument 1, which is declared to never be null
parse.c:935:3: runtime error: null pointer passed as argument 1, which is declared to never be null

clang:

parse.c:1819:36: runtime error: member access within null pointer of type 'PNode' (aka 'struct PNode')
SUMMARY: AddressSanitizer: undefined-behavior parse.c:1819:36 in 
lr.c:180:11: runtime error: null pointer passed as argument 1, which is declared to never be null
/usr/include/stdlib.h:765:30: note: nonnull attribute specified here
SUMMARY: AddressSanitizer: undefined-behavior lr.c:180:11 in 
lr.c:266:9: runtime error: null pointer passed as argument 1, which is declared to never be null
/usr/include/stdlib.h:765:30: note: nonnull attribute specified here
SUMMARY: AddressSanitizer: undefined-behavior lr.c:266:9 in 
lr.c:392:11: runtime error: null pointer passed as argument 1, which is declared to never be null
/usr/include/stdlib.h:765:30: note: nonnull attribute specified here
SUMMARY: AddressSanitizer: undefined-behavior lr.c:392:11 in 
write_tables.c:828:11: runtime error: member access within null pointer of type 'SB_uint8' (aka 'struct SB_uint8')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:828:11 in 
write_tables.c:841:11: runtime error: member access within null pointer of type 'SB_uint8' (aka 'struct SB_uint8')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:841:11 in 
write_tables.c:824:13: runtime error: member access within null pointer of type 'SB_uint8' (aka 'struct SB_uint8')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:824:13 in 
write_tables.c:874:13: runtime error: member access within null pointer of type 'SB_trans_uint8' (aka 'struct SB_trans_uint8')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:874:13 in 
write_tables.c:826:13: runtime error: member access within null pointer of type 'SB_uint8' (aka 'struct SB_uint8')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:826:13 in 
write_tables.c:986:9: runtime error: member access within null pointer of type 'D_RightEpsilonHint' (aka 'struct D_RightEpsilonHint')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:986:9 in 
write_tables.c:1510:13: runtime error: member access within null pointer of type 'D_ErrorRecoveryHint' (aka 'struct D_ErrorRecoveryHint')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1510:13 in 
write_tables.c:1539:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1539:9 in 
write_tables.c:1546:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1546:9 in 
write_tables.c:1555:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1555:9 in 
write_tables.c:1565:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1565:9 in 
write_tables.c:1585:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1585:9 in 
write_tables.c:1588:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1588:9 in 
write_tables.c:1601:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1601:9 in 
write_tables.c:1607:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1607:9 in 
write_tables.c:1568:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1568:9 in 
write_tables.c:1590:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1590:9 in 
write_tables.c:1603:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1603:9 in 
write_tables.c:1609:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1609:9 in 
write_tables.c:1549:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1549:9 in 
write_tables.c:1558:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1558:9 in 
write_tables.c:1541:9: runtime error: member access within null pointer of type 'D_State' (aka 'struct D_State')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1541:9 in 
write_tables.c:1711:5: runtime error: member access within null pointer of type 'D_Symbol' (aka 'struct D_Symbol')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1711:5 in 
write_tables.c:1724:5: runtime error: member access within null pointer of type 'D_Symbol' (aka 'struct D_Symbol')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1724:5 in 
write_tables.c:1784:3: runtime error: member access within null pointer of type 'D_ParserTables' (aka 'struct D_ParserTables')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1784:3 in 
write_tables.c:1785:3: runtime error: member access within null pointer of type 'D_ParserTables' (aka 'struct D_ParserTables')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1785:3 in 
write_tables.c:1788:3: runtime error: member access within null pointer of type 'D_ParserTables' (aka 'struct D_ParserTables')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1788:3 in 
write_tables.c:1793:5: runtime error: member access within null pointer of type 'D_ParserTables' (aka 'struct D_ParserTables')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1793:5 in 
write_tables.c:1798:5: runtime error: member access within null pointer of type 'D_ParserTables' (aka 'struct D_ParserTables')
SUMMARY: AddressSanitizer: undefined-behavior write_tables.c:1798:5 in 
parse.c:935:9: runtime error: null pointer passed as argument 1, which is declared to never be null
/usr/include/stdlib.h:765:30: note: nonnull attribute specified here
SUMMARY: AddressSanitizer: undefined-behavior parse.c:935:9 in 
parse.c:109:15: runtime error: member access within null pointer of type 'PNode' (aka 'struct PNode')
SUMMARY: AddressSanitizer: undefined-behavior parse.c:109:15 in 
parse.c:101:15: runtime error: member access within null pointer of type 'PNode' (aka 'struct PNode')
SUMMARY: AddressSanitizer: undefined-behavior parse.c:101:15 in 
parse.c:575:36: runtime error: member access within null pointer of type 'PNode' (aka 'struct PNode')
SUMMARY: AddressSanitizer: undefined-behavior parse.c:575:36 in 
parse.c:115:15: runtime error: member access within null pointer of type 'PNode' (aka 'struct PNode')
SUMMARY: AddressSanitizer: undefined-behavior parse.c:115:15 in

https://cran.r-project.org/web/checks/check_results_dparser.html

Microsoft VS support?

Hi, and thanks for an unbelieveably nifty bit of tooling.

I've got a Visual Studio project file that builds dparser as a standalone Windows .exe file, along with a .lib for the runtime system. Required a couple of minor tweaks. Any interest in a PR?

BR,

Jerry

dparser memory leak

There is a small memory leak with bad syntax. Here is a reproducible example.

Replace the sample.g with the grammar here
Make the dparser package
Create the following invalid syntax file:

# comment, just to show error in line 3
d/dt(y) = -ka;
C1 = /y;

I name it bad.txt

From the terminal you can see some of the memory is lost:

matt@localhost ~/src/dparser $ valgrind ./sample_parser bad.txt
==2465== Memcheck, a memory error detector
==2465== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==2465== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==2465== Command: ./sample_parser bad.txt
==2465== 
:3: syntax error after '='
fatal error, '' line 3
==2465== 
==2465== HEAP SUMMARY:
==2465==     in use at exit: 11,312 bytes in 37 blocks
==2465==   total heap usage: 403 allocs, 366 frees, 288,690 bytes allocated
==2465== 
==2465== LEAK SUMMARY:
==2465==    definitely lost: 136 bytes in 1 blocks
==2465==    indirectly lost: 11,176 bytes in 36 blocks
==2465==      possibly lost: 0 bytes in 0 blocks
==2465==    still reachable: 0 bytes in 0 blocks
==2465==         suppressed: 0 bytes in 0 blocks
==2465== Rerun with --leak-check=full to see details of leaked memory
==2465== 
==2465== For lists of detected and suppressed errors, rerun with: -s
==2465== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

How to set a syntax error callback without global variables?

The declaration of D_SyntaxErrorFn is simple:

typedef void (*D_SyntaxErrorFn)(struct D_Parser *);

However, as far as I see, I can't pass in any of my own data (in my case, a logger). Is there a way to do that without globals?

Speed of dparser -- possibly increase with std::sort from C++?

For very long parsing blocks, the speed of dparser is long; See:

nlmixrdevelopment/RxODE#61 (comment)

The slowest function is here:

dparser/parse.c

Lines 854 to 897 in 98ae9cc

 static int cmp_greediness(Parser *p, PNode *x, PNode *y) { 

 uint ix = 0, iy = 0; 

 int ret = 0; 

 VecPNode pvx, pvy; 

 vec_clear(&pvx); 

 vec_clear(&pvy); 

 get_unshared_pnodes(p, x, y, &pvx, &pvy); 

 /* set_to_vec(&pvx); set_to_vec(&pvy); */ 

 qsort(pvx.v, pvx.n, sizeof(PNode *), greedycmp); 

 qsort(pvy.v, pvy.n, sizeof(PNode *), greedycmp); 

 while (1) { 

 if (pvx.n <= ix || pvy.n <= iy) RET(0); 

 x = pvx.v[ix]; 

 y = pvy.v[iy]; 

 if (x == y) { 

 ix++; 

 iy++; 

 } else if (x->parse_node.start_loc.s < y->parse_node.start_loc.s) 

 ix++; 

 else if (x->parse_node.start_loc.s > y->parse_node.start_loc.s) 

 iy++; 

 else if (x->parse_node.symbol < y->parse_node.symbol) 

 ix++; 

 else if (x->parse_node.symbol > y->parse_node.symbol) 

 iy++; 

 else if (x->parse_node.end > y->parse_node.end) 

 RET(-1); 

 else if (x->parse_node.end < y->parse_node.end) 

 RET(1); 

 else if (x->children.n < y->children.n) 

 RET(-1); 

 else if (x->children.n > y->children.n) 

 RET(1); 

 else { 

 ix++; 

 iy++; 

 } 

 } 

 Lreturn: 

 vec_free(&pvx); 

 vec_free(&pvy); 

 return ret; 

 }

Which uses qsort

dparser/parse.c

Line 864 in 98ae9cc

qsort(pvy.v, pvy.n, sizeof(PNode *), greedycmp);

In theory C++'s std::sort is 250% to 1000% faster and since you can refer to c++ from c, do you think a wrapper for the std::sort would greatly increase the parsing time?

If so, do you think it is worthwhile to contaminate the pure C with C++?

valgrind errors

I see errors like these:

==28978== Invalid write of size 4
==28978==    at 0x10B6D9: d_final_reduction_code_4_6_gram (4calc.g:21)
==28978==    by 0x113D45: commit_tree (parse.c:1618)
==28978==    by 0x113AD8: commit_tree (parse.c:1603)
==28978==    by 0x113AD8: commit_tree (parse.c:1603)
==28978==    by 0x113AD8: commit_tree (parse.c:1603)
==28978==    by 0x115FC0: dparse (parse.c:2097)
==28978==    by 0x10B561: main (4calc.c:28)
==28978==  Address 0x4a667b0 is 0 bytes after a block of size 240 alloc'd
==28978==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==28978==    by 0x10F8C7: make_PNode (parse.c:935)
==28978==    by 0x1101C7: add_PNode (parse.c:1035)
==28978==    by 0x112760: reduce_one (parse.c:1384)
==28978==    by 0x1152C8: exhaustive_parse (parse.c:1846)
==28978==    by 0x115F5A: dparse (parse.c:2090)
==28978==    by 0x10B561: main (4calc.c:28)

They disappear with this patch:

diff --git a/parse.c b/parse.c
index 05e0d32..13c6e9e 100644
--- a/parse.c
+++ b/parse.c
@@ -927,6 +927,9 @@ static PNode *make_PNode(Parser *p, uint hash, int symbol, d_loc_t *start_loc, c
                          VecZNode *path, D_Shift *sh, D_Scope *scope) {
   int i;
   uint l = sizeof(PNode) - sizeof(d_voidp) + p->user.sizeof_user_parse_node;
+  l += 4;
   PNode *new_pn = p->free_pnodes;
   if (!new_pn)
     new_pn = MALLOC(l);

Fewer than 4 extra bytes per node are not enough to fix the errors.

I am guessing you need some extra room due to struct padding?

support for aquiring a tokens matched string like $$.buf or yy.text

how can i obtain a tokens matched string as it is difficult to do so, for example

{
#include <stdio.h>

typedef struct My_Sym {
  int value;
} My_Sym;
#define D_UserSym My_Sym
typedef struct My_ParseNode {
  int value;
  struct D_Scope *scope;
} My_ParseNode;
#define D_ParseNode_User My_ParseNode
}

{
#include "libdparse.h"
#define f(x, y) if (y) x(y)
}

/* r: constant* [ puts("const"); ]; */

translation_unit: terminal* s;

terminal : identifier s ':' s rule* s ';' ;
rule : i* s rule* s ;

i : decimalint* s | hexint* s | octalint* s | character* s | float1* s | float2* s | string* s | identifier* s | integer* s | i* s;

s: [
	puts("speculative start");
	char * c = NULL;
	ps($n.start_loc.s);
	ps($n.end);
	ps($n.end_skip);
	for (c = $n.start_loc.s; c < $n.end_skip; c++) pc(*c);
	ps($n.start_loc.s)
	ps($n.end)
	puts("speculative end");
] 
{
	puts("final start");
	char * c = NULL;
	ps($n.start_loc.s);
	ps($n.end);
	ps($n.end_skip);
	for (c = $n.start_loc.s; c < $n.end_skip; c++) pc(*c);
	ps($n.start_loc.s)
	ps($n.end)
	puts("final end");
};


symbols : "[!~`@#$%^&*\-_+=|:;\\<,>.?/]";
character : "'([^'\\]|\\[^])*'";
float1 : "([0-9]+.[0-9]*|[0-9]*.[0-9]+)([eE][\-\+]?[0-9]+)?[fFlL]?" $term -4;
float2 : "[0-9]+[eE][\-\+]?[0-9]+[fFlL]?" $term -5;
string : "'([^'\\]|\\[^])*'";
regex : "\"([^\"\\]|\\[^])*\"";
unicode_char : "[uU]\+[0-9a-fA-F]+";
decimalint : "-?[1-9][0-9]*[uUlL]?";
hexint : "-?(0x|0X)[0-9a-fA-F]+[uUlL]?";
octalint : "-?0[0-7]*[uUlL]?";
 
statement 
  : expression ';'
  { printf("val %d\n", $0.value); }
  | '{' new_scope statement* '}'
  [ ${scope} = enter_D_Scope(${scope}, $n0.scope); ]
  ;

new_scope: [ ${scope} = new_D_Scope(${scope}); ];

expression 
  : identifier ':' expression 
  [ 
    D_Sym *s;
	ps($n.start_loc.s);
	ps($n.end);
    if (find_D_Sym_in_Scope(${scope}, ${scope}, $n0.start_loc.s, $n0.end))
      printf("duplicate identifier line %d\n", $n0.start_loc.line);
    s = NEW_D_SYM(${scope}, $n0.start_loc.s, $n0.end);
    s->user.value = $2.value;
    $$.value = s->user.value;
  ]
  | identifier '=' expression
  [ D_Sym *s = find_D_Sym(${scope}, $n0.start_loc.s, $n0.end);
    s = UPDATE_D_SYM(s, &${scope});
    s->user.value = $2.value;
    $$.value = s->user.value;
  ]
  | integer 
  [ $$.value = atoi($n0.start_loc.s); ]
  | identifier 
  [ D_Sym *s = find_D_Sym(${scope}, $n0.start_loc.s, $n0.end);
    if (s)
      $$.value = s->user.value;
  ]
  | expression '+' expression
  [ $$.value = $0.value + $1.value; ]
  ;

integer: "-?([0-9]|0(x|X))[0-9]*(u|U|b|B|w|W|L|l)*" $term -1;
identifier: "[a-zA-Z_][a-zA-Z_0-9]*";

results in:

mobile_c@Mobile-C:~/git/dparser$ ./make_dparser ./PARSER_GRAMMER.g && gcc PARSER_GRAMMER.c -lpthread && ./a.out PARSER_LEXER.g -v  -v
In file included from ../CCR/Scripts/Shell/shell.h:18:0,
                 from PARSER_GRAMMER.c:4:
../CCR/Scripts/Shell/builtins/env.h: In function ‘env__get_env’:
../CCR/Scripts/Shell/builtins/env.h:355:10: warning: return from incompatible pointer type [-Wincompatible-pointer-types]
   return *b;
          ^~
In file included from ../CCR/Scripts/Shell/shell.h:125:0,
                 from PARSER_GRAMMER.c:4:
../CCR/Scripts/Shell/builtins/whereis.h: In function ‘builtin__whereis’:
../CCR/Scripts/Shell/builtins/whereis.h:51:16: warning: initialization discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
  char * file = argv[skip_arg0];
                ^~~~
../CCR/Scripts/Shell/builtins/whereis.h:35:143: warning: function returns address of local variable [-Wreturn-local-addr]
 #define try DEBUG printf("trying %s\n", ph); if (access(ph, F_OK) == 0) { if (shell.builtin) printf("found '%s' at '%s'\n", file, ph); return ph; }
                                                                                                                                               ^
../CCR/Scripts/Shell/builtins/whereis.h:75:5: note: in expansion of macro ‘try’
     try
     ^~~
../CCR/Scripts/Shell/builtins/whereis.h:35:143: warning: function returns address of local variable [-Wreturn-local-addr]
 #define try DEBUG printf("trying %s\n", ph); if (access(ph, F_OK) == 0) { if (shell.builtin) printf("found '%s' at '%s'\n", file, ph); return ph; }
                                                                                                                                               ^
../CCR/Scripts/Shell/builtins/whereis.h:77:5: note: in expansion of macro ‘try’
     try
     ^~~
../CCR/Scripts/Shell/builtins/whereis.h:35:143: warning: function returns address of local variable [-Wreturn-local-addr]
 #define try DEBUG printf("trying %s\n", ph); if (access(ph, F_OK) == 0) { if (shell.builtin) printf("found '%s' at '%s'\n", file, ph); return ph; }
                                                                                                                                               ^
../CCR/Scripts/Shell/builtins/whereis.h:80:5: note: in expansion of macro ‘try’
     try
     ^~~
../CCR/Scripts/Shell/builtins/whereis.h:35:143: warning: function returns address of local variable [-Wreturn-local-addr]
 #define try DEBUG printf("trying %s\n", ph); if (access(ph, F_OK) == 0) { if (shell.builtin) printf("found '%s' at '%s'\n", file, ph); return ph; }
                                                                                                                                               ^
../CCR/Scripts/Shell/builtins/whereis.h:87:4: note: in expansion of macro ‘try’
    try
    ^~~
../CCR/Scripts/Shell/builtins/whereis.h:35:143: warning: function returns address of local variable [-Wreturn-local-addr]
 #define try DEBUG printf("trying %s\n", ph); if (access(ph, F_OK) == 0) { if (shell.builtin) printf("found '%s' at '%s'\n", file, ph); return ph; }
                                                                                                                                               ^
../CCR/Scripts/Shell/builtins/whereis.h:89:4: note: in expansion of macro ‘try’
    try
    ^~~
../CCR/Scripts/Shell/builtins/whereis.h:35:143: warning: function returns address of local variable [-Wreturn-local-addr]
 #define try DEBUG printf("trying %s\n", ph); if (access(ph, F_OK) == 0) { if (shell.builtin) printf("found '%s' at '%s'\n", file, ph); return ph; }
                                                                                                                                               ^
../CCR/Scripts/Shell/builtins/whereis.h:92:4: note: in expansion of macro ‘try’
    try
    ^~~
../CCR/Scripts/Shell/builtins/whereis.h:35:143: warning: function returns address of local variable [-Wreturn-local-addr]
 #define try DEBUG printf("trying %s\n", ph); if (access(ph, F_OK) == 0) { if (shell.builtin) printf("found '%s' at '%s'\n", file, ph); return ph; }
                                                                                                                                               ^
../CCR/Scripts/Shell/builtins/whereis.h:96:3: note: in expansion of macro ‘try’
   try
   ^~~
../CCR/Scripts/Shell/builtins/whereis.h:35:143: warning: function returns address of local variable [-Wreturn-local-addr]
 #define try DEBUG printf("trying %s\n", ph); if (access(ph, F_OK) == 0) { if (shell.builtin) printf("found '%s' at '%s'\n", file, ph); return ph; }
                                                                                                                                               ^
../CCR/Scripts/Shell/builtins/whereis.h:98:3: note: in expansion of macro ‘try’
   try
   ^~~
In file included from ../CCR/Scripts/Shell/shell.h:135:0,
                 from PARSER_GRAMMER.c:4:
../CCR/Scripts/Shell/builtins/argv.h: In function ‘Xmalloc’:
../CCR/Scripts/Shell/builtins/argv.h:30:25: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘size_t {aka long unsigned int}’ [-Wformat=]
         fprintf(stderr, nomem, sz);
                         ^~~~~
../CCR/Scripts/Shell/builtins/argv.h: In function ‘Xrealloc’:
../CCR/Scripts/Shell/builtins/argv.h:41:25: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘size_t {aka long unsigned int}’ [-Wformat=]
         fprintf(stderr, nomem, sz);
                         ^~~~~
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h: In function ‘builtin’:
../CCR/Scripts/Shell/shell.h:179:43: warning: passing argument 2 of ‘builtin__hex’ from incompatible pointer type [-Wincompatible-pointer-types]
   ifb("hex") iftimeret(builtin__hex(argc, argv))
                                           ^
../CCR/Scripts/Shell/shell.h:161:24: note: in definition of macro ‘iftimeret’
 #define iftimeret(x) { x; }
                        ^
../CCR/Scripts/Shell/shell.h:164:5: note: expected ‘char **’ but argument is of type ‘const char **’
 int builtin__hex(int argc, char * argv[]) { return 0; }
     ^~~~~~~~~~~~
../CCR/Scripts/Shell/shell.h:180:45: warning: passing argument 2 of ‘builtin__cat’ from incompatible pointer type [-Wincompatible-pointer-types]
   elifb("cat") iftimeret(builtin__cat(argc, argv))
                                             ^
../CCR/Scripts/Shell/shell.h:161:24: note: in definition of macro ‘iftimeret’
 #define iftimeret(x) { x; }
                        ^
../CCR/Scripts/Shell/shell.h:165:5: note: expected ‘char **’ but argument is of type ‘const char **’
 int builtin__cat(int argc, char * argv[]) { return 0; }
     ^~~~~~~~~~~~
../CCR/Scripts/Shell/shell.h:181:52: warning: passing argument 2 of ‘builtin__json’ from incompatible pointer type [-Wincompatible-pointer-types]
   elifb("json") iftimeret(puts(builtin__json(argc, argv)))
                                                    ^
../CCR/Scripts/Shell/shell.h:161:24: note: in definition of macro ‘iftimeret’
 #define iftimeret(x) { x; }
                        ^
../CCR/Scripts/Shell/shell.h:166:5: note: expected ‘char **’ but argument is of type ‘const char **’
 int builtin__json(int argc, char * argv[]) { return 0; }
     ^~~~~~~~~~~~~
../CCR/Scripts/Shell/shell.h:181:32: warning: passing argument 1 of ‘puts’ makes pointer from integer without a cast [-Wint-conversion]
   elifb("json") iftimeret(puts(builtin__json(argc, argv)))
                                ^
../CCR/Scripts/Shell/shell.h:161:24: note: in definition of macro ‘iftimeret’
 #define iftimeret(x) { x; }
                        ^
In file included from ../CCR/Scripts/Shell/shell.h:6:0,
                 from PARSER_GRAMMER.c:4:
/usr/include/stdio.h:632:12: note: expected ‘const char *’ but argument is of type ‘int’
 extern int puts (const char *__s);
            ^~~~
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h:182:45: warning: passing argument 2 of ‘builtin__xxd’ from incompatible pointer type [-Wincompatible-pointer-types]
   elifb("xxd") iftimeret(builtin__xxd(argc, argv))
                                             ^
../CCR/Scripts/Shell/shell.h:161:24: note: in definition of macro ‘iftimeret’
 #define iftimeret(x) { x; }
                        ^
../CCR/Scripts/Shell/shell.h:167:5: note: expected ‘char **’ but argument is of type ‘const char **’
 int builtin__xxd(int argc, char * argv[]) { return 0; }
     ^~~~~~~~~~~~
../CCR/Scripts/Shell/shell.h:183:54: warning: passing argument 2 of ‘builtin__CPU_Info’ from incompatible pointer type [-Wincompatible-pointer-types]
   elifb("CPUInfo") iftimeret(builtin__CPU_Info(argc, argv))
                                                      ^
../CCR/Scripts/Shell/shell.h:161:24: note: in definition of macro ‘iftimeret’
 #define iftimeret(x) { x; }
                        ^
../CCR/Scripts/Shell/shell.h:168:5: note: expected ‘char **’ but argument is of type ‘const char **’
 int builtin__CPU_Info(int argc, char * argv[]) { return 0; }
     ^~~~~~~~~~~~~~~~~
../CCR/Scripts/Shell/shell.h:184:43: warning: passing argument 2 of ‘builtin__ls’ from incompatible pointer type [-Wincompatible-pointer-types]
   elifb("ls") iftimeret(builtin__ls(argc, argv))
                                           ^
../CCR/Scripts/Shell/shell.h:161:24: note: in definition of macro ‘iftimeret’
 #define iftimeret(x) { x; }
                        ^
../CCR/Scripts/Shell/shell.h:169:5: note: expected ‘char **’ but argument is of type ‘const char **’
 int builtin__ls(int argc, char * argv[]) { return 0; }
     ^~~~~~~~~~~
../CCR/Scripts/Shell/shell.h:185:45: warning: passing argument 2 of ‘builtin__gcc’ from incompatible pointer type [-Wincompatible-pointer-types]
   elifb("gcc") iftimeret(builtin__gcc(argc, argv))
                                             ^
../CCR/Scripts/Shell/shell.h:161:24: note: in definition of macro ‘iftimeret’
 #define iftimeret(x) { x; }
                        ^
../CCR/Scripts/Shell/shell.h:170:5: note: expected ‘char **’ but argument is of type ‘const char **’
 int builtin__gcc(int argc, char * argv[]) { return 0; }
     ^~~~~~~~~~~~
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h: In function ‘fix_string’:
../CCR/Scripts/Shell/shell.h:274:9: warning: passing argument 1 of ‘quote’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
  mq(qt, fmt);
         ^
../CCR/Scripts/Shell/shell.h:133:35: note: in definition of macro ‘mq’
 #define mq(a, b) char * a = quote(b)
                                   ^
../CCR/Scripts/Shell/shell.h:127:8: note: expected ‘char *’ but argument is of type ‘const char *’
 char * quote (char * str) {
        ^~~~~
../CCR/Scripts/Shell/shell.h: In function ‘replace’:
../CCR/Scripts/Shell/shell.h:323:8: warning: passing argument 1 of ‘quote’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
  mq(qt,str);
        ^
../CCR/Scripts/Shell/shell.h:133:35: note: in definition of macro ‘mq’
 #define mq(a, b) char * a = quote(b)
                                   ^
../CCR/Scripts/Shell/shell.h:127:8: note: expected ‘char *’ but argument is of type ‘const char *’
 char * quote (char * str) {
        ^~~~~
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h: In function ‘execute’:
../CCR/Scripts/Shell/shell.h:430:106: warning: passing argument 3 of ‘builtin’ from incompatible pointer type [-Wincompatible-pointer-types]
  if (!ifbuiltin(cmd->vector_word[0])) return (void *) builtin(cmd->vector_word[0], cmd->vector_word_idx, cmd->vector_word);
                                                                                                          ^~~
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h:177:5: note: expected ‘const char **’ but argument is of type ‘char **’
 int builtin(const char * builtin, const int argc, const char * argv[]) {
     ^~~~~~~
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h:430:46: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  if (!ifbuiltin(cmd->vector_word[0])) return (void *) builtin(cmd->vector_word[0], cmd->vector_word_idx, cmd->vector_word);
                                              ^
../CCR/Scripts/Shell/shell.h:459:85: warning: passing argument 3 of ‘builtin’ from incompatible pointer type [-Wincompatible-pointer-types]
              builtin(cmd->vector_sub[idx].vector_sub_word[1], cmd->vector_word_idx, cmd->vector_word);
                                                                                     ^~~
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h:177:5: note: expected ‘const char **’ but argument is of type ‘char **’
 int builtin(const char * builtin, const int argc, const char * argv[]) {
     ^~~~~~~
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h:478:51: warning: passing argument 2 of ‘builtin__whereis’ from incompatible pointer type [-Wincompatible-pointer-types]
      exe = builtin__whereis(cmd->vector_word_idx, cmd->vector_word, ".c", false, NULL, whereis_mode_return_first);
                                                   ^~~
In file included from ../CCR/Scripts/Shell/shell.h:125:0,
                 from PARSER_GRAMMER.c:4:
../CCR/Scripts/Shell/builtins/whereis.h:39:8: note: expected ‘const char **’ but argument is of type ‘char **’
 char * builtin__whereis(const int argc, const char * argv[], const char * extention, int skip_arg0, char * optional_path, int mode) {
        ^~~~~~~~~~~~~~~~
In file included from ../CCR/Scripts/Shell/shell.h:20:0,
                 from PARSER_GRAMMER.c:4:
../CCR/Scripts/Shell/builtins/printfmacro.h:210:27: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘char *’ [-Wformat=]
 #define pi(x)      printf("%s = %d\n", #x, x);
                           ^
../CCR/Scripts/Shell/shell.h:482:6: note: in expansion of macro ‘pi’
      pi(*(cmd->vector_word))
      ^~
../CCR/Scripts/Shell/builtins/printfmacro.h:210:27: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘char *’ [-Wformat=]
 #define pi(x)      printf("%s = %d\n", #x, x);
                           ^
../CCR/Scripts/Shell/shell.h:484:6: note: in expansion of macro ‘pi’
      pi(*(cmd->vector_word+1))
      ^~
../CCR/Scripts/Shell/builtins/printfmacro.h:210:27: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘char *’ [-Wformat=]
 #define pi(x)      printf("%s = %d\n", #x, x);
                           ^
../CCR/Scripts/Shell/shell.h:486:6: note: in expansion of macro ‘pi’
      pi(*(cmd->vector_word+2))
      ^~
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h:497:13: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
      return (void *) ret;
             ^
../CCR/Scripts/Shell/shell.h: In function ‘execute_thread’:
../CCR/Scripts/Shell/shell.h:534:83: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
   DEBUG printf("Main: completed join with thread %ld having a status of %d\n", t, (int)status);
                                                                                   ^
In file included from PARSER_GRAMMER.c:4:0:
../CCR/Scripts/Shell/shell.h: In function ‘parse’:
../CCR/Scripts/Shell/shell.h:669:8: warning: passing argument 1 of ‘quote’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
  mq(qt,args);
        ^
../CCR/Scripts/Shell/shell.h:133:35: note: in definition of macro ‘mq’
 #define mq(a, b) char * a = quote(b)
                                   ^
../CCR/Scripts/Shell/shell.h:127:8: note: expected ‘char *’ but argument is of type ‘const char *’
 char * quote (char * str) {
        ^~~~~
In file included from libdparse.h:1:0,
                 from ./PARSER_GRAMMER.g:16,
                 from PARSER_GRAMMER.c:5:
arg.c: In function ‘process_arg’:
arg.c:35:12: warning: format ‘%s’ expects argument of type ‘char *’, but argument 2 has type ‘char **’ [-Wformat=]
     printf("invalid argument: %s\n", *argv);
            ^~~~~~~~~~~~~~~~~~~~~~~~  ~~~~~
In file included from PARSER_GRAMMER.c:8:0:
../CCR/Scripts/Shell/builtins/regex_str.h: In function ‘getbitgroupcount’:
../CCR/Scripts/Shell/builtins/regex_str.h:8:18: warning: passing argument 1 of ‘snprintf’ from incompatible pointer type [-Wincompatible-pointer-types]
  return snprintf(stdout, 0, "%d", bit);
                  ^~~~~~
In file included from ../CCR/Scripts/Shell/shell.h:6:0,
                 from PARSER_GRAMMER.c:4:
/usr/include/stdio.h:340:12: note: expected ‘char * restrict’ but argument is of type ‘struct _IO_FILE *’
 extern int snprintf (char *__restrict __s, size_t __maxlen,
            ^~~~~~~~
speculative start
(*(D_PN(_ps, _offset))).start_loc.s = token : rule;

(*(D_PN(_ps, _offset))).end = token : rule;

(*(D_PN(_ps, _offset))).end_skip = token : rule;

(*(D_PN(_ps, _offset))).start_loc.s = token : rule;

(*(D_PN(_ps, _offset))).end = token : rule;

speculative end
speculative start
(*(D_PN(_ps, _offset))).start_loc.s = : rule;

(*(D_PN(_ps, _offset))).end = : rule;

(*(D_PN(_ps, _offset))).end_skip = : rule;

(*(D_PN(_ps, _offset))).start_loc.s = : rule;

(*(D_PN(_ps, _offset))).end = : rule;

speculative end
speculative start
(*(D_PN(_ps, _offset))).start_loc.s = rule;

(*(D_PN(_ps, _offset))).end = rule;

(*(D_PN(_ps, _offset))).end_skip = rule;

(*(D_PN(_ps, _offset))).start_loc.s = rule;

(*(D_PN(_ps, _offset))).end = rule;

speculative end
speculative start
(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

(*(D_PN(_ps, _offset))).end_skip = ;

(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

speculative end
speculative start
(*(D_PN(_ps, _offset))).start_loc.s = 
(*(D_PN(_ps, _offset))).end = 
(*(D_PN(_ps, _offset))).end_skip = 
(*(D_PN(_ps, _offset))).start_loc.s = 
(*(D_PN(_ps, _offset))).end = 
speculative end
final start
(*(D_PN(_ps, _offset))).start_loc.s = : rule;

(*(D_PN(_ps, _offset))).end = : rule;

(*(D_PN(_ps, _offset))).end_skip = : rule;

(*(D_PN(_ps, _offset))).start_loc.s = : rule;

(*(D_PN(_ps, _offset))).end = : rule;

final end
final start
(*(D_PN(_ps, _offset))).start_loc.s = rule;

(*(D_PN(_ps, _offset))).end = rule;

(*(D_PN(_ps, _offset))).end_skip = rule;

(*(D_PN(_ps, _offset))).start_loc.s = rule;

(*(D_PN(_ps, _offset))).end = rule;

final end
final start
(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

(*(D_PN(_ps, _offset))).end_skip = ;

(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

final end
final start
(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

(*(D_PN(_ps, _offset))).end_skip = ;

(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

final end
final start
(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

(*(D_PN(_ps, _offset))).end_skip = ;

(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

final end
final start
(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

(*(D_PN(_ps, _offset))).end_skip = ;

(*(D_PN(_ps, _offset))).start_loc.s = ;

(*(D_PN(_ps, _offset))).end = ;

final end
final start
(*(D_PN(_ps, _offset))).start_loc.s = 
(*(D_PN(_ps, _offset))).end = 
(*(D_PN(_ps, _offset))).end_skip = 
(*(D_PN(_ps, _offset))).start_loc.s = 
(*(D_PN(_ps, _offset))).end = 
final end
76 states 23 scans 4 shifts 58 reductions 34 compares 0 ambiguities
[translation_unit]([translation_unit__2]([terminal]([identifier]([[a-zA-Z_][a-zA-Z_0-9]*] token )[s][:] : [s][terminal__4]([rule]([rule__6]([i]([i__16]([identifier]([[a-zA-Z_][a-zA-Z_0-9]*] rule ))[s]))[s][rule__7][s]))[s][;] ; ))[s])
mobile_c@Mobile-C:~/git/dparser$

Building DParser in Visual Studio

Could you post or provide detailed instructions on how to build DParser in Visual Studio?

Apparent incompatibility with bytes objects in Python 3

I just tried installing dparser 1.31 under Python 3.7.6 and Swig 3.0.8 and ran into this error when running make test in the python directory:

    running test tests/test2.py
Traceback (most recent call last):
  File "tests/test2.py", line 53, in <module>
    if Parser().parse('1  +2* (3+ 4+5)', ambiguity_fn=ambiguity_func,
  File "/home/bkovitz/github/dparser/python/build/lib.linux-x86_64-3.7/dparser.py", line 290, in __init__
    self.tables.load_tables(grammar_str, self.filename, make_grammar_file)
  File "/home/bkovitz/github/dparser/python/build/lib.linux-x86_64-3.7/dparser.py", line 190, in load_tables
    dparser_swigc.make_tables(grammar_str, filename.encode())
TypeError: in method 'make_tables', argument 1 of type 'char *'

The problem appears to be that dparser.py is passing a bytes object but the Swig wrapper expects an ordinary string. Perhaps this is an incompatibility with Python 3 (which made bytes objects distinct from strings)?

I got the tests to pass by editing dparser.py to call .decode('ascii') on various arguments passed through Swig, like this (line 190):

        dparser_swigc.make_tables(grammar_str.decode('ascii'), filename.encode().decode('ascii'))

in a few places. I can submit a pull request with these changes if you like, but I don't know enough to say if this is the correct solution or not (especially for Python 2).

Inconsistent results in tests

I ran the tests with MAKE=make ./parser_tests and got this:

17c17
< :161: syntax error after 'RightButton) ) {
---
> :161: syntax error after 'LeftButton || button == Key.RightButton) ) {
g50.test.g.1 ******** FAILED ********

Ran them again and all tests passed. Then did a git clean -dfx and ran them again, and the test failed again; I also saw this error at the beginning of the run (which was not happening before): cp: cannot stat 'D_BUILD_VERSION': No such file or directory

Are these known issues?

	if (after)
	fprintf(stderr, "%s:%d: syntax error after '%s'\n", fn, p->user.loc.line, after);
	else
	fprintf(stderr, "%s:%d: syntax error\n", fn, p->user.loc.line);

	static int cmp_greediness(Parser p, PNode x, PNode *y) {
	uint ix = 0, iy = 0;
	int ret = 0;

	VecPNode pvx, pvy;
	vec_clear(&pvx);
	vec_clear(&pvy);
	get_unshared_pnodes(p, x, y, &pvx, &pvy);
	/* set_to_vec(&pvx); set_to_vec(&pvy); */
	qsort(pvx.v, pvx.n, sizeof(PNode *), greedycmp);
	qsort(pvy.v, pvy.n, sizeof(PNode *), greedycmp);
	while (1) {
	if (pvx.n <= ix \|\| pvy.n <= iy) RET(0);
	x = pvx.v[ix];
	y = pvy.v[iy];
	if (x == y) {
	ix++;
	iy++;
	} else if (x->parse_node.start_loc.s < y->parse_node.start_loc.s)
	ix++;
	else if (x->parse_node.start_loc.s > y->parse_node.start_loc.s)
	iy++;
	else if (x->parse_node.symbol < y->parse_node.symbol)
	ix++;
	else if (x->parse_node.symbol > y->parse_node.symbol)
	iy++;
	else if (x->parse_node.end > y->parse_node.end)
	RET(-1);
	else if (x->parse_node.end < y->parse_node.end)
	RET(1);
	else if (x->children.n < y->children.n)
	RET(-1);
	else if (x->children.n > y->children.n)
	RET(1);
	else {
	ix++;
	iy++;
	}
	}
	Lreturn:
	vec_free(&pvx);
	vec_free(&pvy);
	return ret;
	}

jplevyak / dparser Goto Github PK

dparser's Introduction

dparser's People

Contributors

Stargazers

Watchers

Forkers

dparser's Issues

Recommend Projects

Recommend Topics

Recommend Org