jddurand / c-marpaeslif Goto Github PK

View Code? Open in Web Editor NEW

6.0 4.0 3.0 72.82 MB

Extended perl's Marpa::R2 SLIF grammar writen in C

License: MIT License

CMake 0.43% C 99.57%

parser marpa

c-marpaeslif's People

Contributors

Stargazers

Watchers

Forkers

abhisheknishantpuresoftware jeffreykegler clayne

c-marpaeslif's Issues

perl empty string import/export

It seems that empty string import/export is done via an opaque value to lua:

lua_proxy received value of type: userdata
lua_proxy value: userdata: 0x2e286bc
ok 21 - import/export of <empty string>

warning in ESLIF.xs

Has to do with the maxDepth option of JSON decoder:

eslif.xs(3607) : warning C4700: uninitialized local variable 'svp' used

"transverse" in the docs and comments

Sometimes the worst person to ask about a language is a native speaker, since they judge things by the speech of whatever bog they grew up in. :-) So when I saw you use the word "transverse" in a sense I did not recognize, I did my homework, looking in dictionaries, etc. You use it (I think) in the sense of "applying across [some domain]", and that sense isn't in any dictionary I consulted.

So I was just about to file an issue saying that "transverse" does not occur in English in this sense, when by pure change I listened to a commentary on "Whiter Shade of Pale", where an LA musicologist used "transverse" as follows: "the past became a fungible toolbox [...] which transversed the classical and popular music worlds". So, here, caught in the wild, was a use of "transverse" by (apparently) a native speaker in the same sense you use it.

The musicologist is a PhD and apparently a postmodernist or influenced by them. So I suspect this sense of "transverse" is from the postmodern lexicon.

All of which is a long preamble to this: I stumbled over the work "transverse" in your docs and it took me a while to puzzle out what you meant by it. I've actually read in philosophy, including a fair amount of Deleuze, so I think if the word was strange to me, it's likely to puzzle a lot of other readers even among those otherwise fluent in English. I'd like to revise the docs to eliminate the use of "transverse" in favor of locutions more like to be familiar to those who read tech docs.

Install failure on OpenBSD

This is linked to C++ flags discovery that failed, c.f. http://www.cpantesters.org/cpan/report/563e4562-90a8-11ed-a657-2f0965a359d2

Undefined symbol "libiconv"

c.f. https://rt.cpan.org/Public/Bug/Display.html?id=120784

Blog / document ESLIF timings

I am opening this issue to ask questions for my blog post on issue #9.

[rt.cpan.org #123444] Modification of a read-only value attempted at Makefile.PL line 144. (2.0.21)

C.f. https://rt.cpan.org/Ticket/Display.html?id=123444

Perl import/export is wrong

Perl bindings uses Types::Standard to determine SV nature and this is wrong.
This is at the origin of https://rt.cpan.org/Ticket/Display.html?id=129152 and has its own ticket https://rt.cpan.org/Ticket/Display.html?id=129164

perl on misconfigured solaris platform

C.f. http://www.cpantesters.org/cpan/report/85af1db6-5e2b-11e9-aeb7-f3056157a1ce :

perl is misconfigured, referencing c++ instead of CC:

==========================================
Original compilers and linker settings as per ExtUtils::CBuilder

CC           (overwrite) /opt/SUNWspro/bin/cc
CFLAGS       (    fixed) -I/usr/gnu/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPERL_USE_SAFE_PUTENV
CFLAGS       (   append) 
CXX          (overwrite) c++
CXXFLAGS     (overwrite) -I/usr/gnu/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPERL_USE_SAFE_PUTENV
LD           (overwrite) /opt/SUNWspro/bin/cc
LDFLAGS      (    fixed)  -L/usr/lib -L/usr/ccs/lib -L/opt/sunstudio12.1/prod/lib -L/lib -L/usr/gnu/lib 
LDFLAGS      (   append) 
==========================================

ESLIF time is no-linear as a function of input size.

After a couple of benchmarks, it seems that there is a non-linearity in the speed of the ESLIF built-in JSON parser. In the benchmarks included, note the relative speeds of SLIF JSON, JSON::PP and JSON::XS remain roughly the same, while the performance of ESLIF JSON plummets from better than JSON::PP to worse than the SLIF JSON.

Small file benchmark: https://gist.github.com/jeffreykegler/cdf4f8eceb07fcedb9d5dc52bd69a56a

Large file benchmark: https://gist.github.com/jeffreykegler/895e4136578ec9563468e6c103d1b9ae

SEGVs on some perl configurations with 6.0.26

Sample fail report: http://www.cpantesters.org/cpan/report/7605e376-36be-11ed-993c-b921912ee776

These SEGVs are reproducable and happen with many perl versions but only with few configurations. -DDEBUGGING and usemultiplicity=define and uselongdouble=undef seem required configuration options.

Sample production of a core file:

% /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.36.0/29fb/bin/perl -Mblib t/advent.t                       [CPANSHELL]
ok 1 - require MarpaX::ESLIF;
ok 2 - An object of class 'MarpaX::ESLIF' isa 'MarpaX::ESLIF'
2022/09/18 09:55:38 INFO  747118 Testing input: 2♥ 5♥ 7♦ 8♣ 9♠, expected result: Parse OK, expected value: Hand was 2♥ 5♥ 7♦ 8♣ 9♠
2022/09/18 09:55:38 INFO  747118 Testing suite line: SUIT ~ [\x{2665}\x{2666}\x{2663}\x{2660}]:u, type: hex
ok 3 - Description '2♥ 5♥ 7♦ 8♣ 9♠' have the utf8 flag
ok 4 - descriptionByLevel(0) '2♥ 5♥ 7♦ 8♣ 9♠' have the utf8 flag
ok 5 - descriptionByLevel(1) 'Grammar level 1' have the utf8 flag
zsh: segmentation fault (core dumped)   -Mblib t/advent.t

The stacktrace:

% gdb /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.36.0/29fb/bin/perl core                                [CPANSHELL]
GNU gdb (Debian 12.1-3) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.36.0/29fb/bin/perl...
[New LWP 747118]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.36.0/29fb/bin/per'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fdb6203a6d5 in marpaESLIFPerl_recognizerContextInitv (my_perl=my_perl@entry=0x5621a8d7e2a0, MarpaX_ESLIF_Grammarp=MarpaX_ESLIF_Grammarp@entry=0x5621ab8a49f8, 
    Perl_MarpaX_ESLIF_Grammarp=Perl_MarpaX_ESLIF_Grammarp@entry=0x5621a9e80df8, Perl_recognizerInterfacep=Perl_recognizerInterfacep@entry=0x5621a9e81170, 
    MarpaX_ESLIF_Recognizerp=MarpaX_ESLIF_Recognizerp@entry=0x5621aac045d8, Perl_recognizer_origp=Perl_recognizer_origp@entry=0x0, constantsp=0x5621a99f5b88, MarpaX_ESLIFp=0x5621a99f5b68)
    at /tmp/loop_over_bdir-bQg7EB/MarpaX-ESLIF-6.0.26-0/ESLIF.xs:1641
1641      MarpaX_ESLIF_Recognizerp->marpaESLIFRecognizerp         = NULL;
(gdb) bt
#0  0x00007fdb6203a6d5 in marpaESLIFPerl_recognizerContextInitv (my_perl=my_perl@entry=0x5621a8d7e2a0, MarpaX_ESLIF_Grammarp=MarpaX_ESLIF_Grammarp@entry=0x5621ab8a49f8, 
    Perl_MarpaX_ESLIF_Grammarp=Perl_MarpaX_ESLIF_Grammarp@entry=0x5621a9e80df8, Perl_recognizerInterfacep=Perl_recognizerInterfacep@entry=0x5621a9e81170, 
    MarpaX_ESLIF_Recognizerp=MarpaX_ESLIF_Recognizerp@entry=0x5621aac045d8, Perl_recognizer_origp=Perl_recognizer_origp@entry=0x0, constantsp=0x5621a99f5b88, MarpaX_ESLIFp=0x5621a99f5b68)
    at /tmp/loop_over_bdir-bQg7EB/MarpaX-ESLIF-6.0.26-0/ESLIF.xs:1641
#1  0x00007fdb6203b38f in XS_MarpaX__ESLIF__Recognizer_allocate (my_perl=0x5621a8d7e2a0, cv=<optimized out>) at /tmp/loop_over_bdir-bQg7EB/MarpaX-ESLIF-6.0.26-0/ESLIF.xs:4856
#2  0x00005621a7ca39f4 in Perl_pp_entersub (my_perl=0x5621a8d7e2a0) at pp_hot.c:5352
#3  0x00005621a7c5807a in Perl_runops_debug (my_perl=0x5621a8d7e2a0) at dump.c:2677
#4  0x00005621a7ba68af in S_run_body (oldscope=1, my_perl=0x5621a8d7e2a0) at perl.c:2716
#5  perl_run (my_perl=0x5621a8d7e2a0) at perl.c:2644
#6  0x00005621a7b6746e in main (argc=<optimized out>, argv=<optimized out>, env=<optimized out>) at perlmain.c:110

Issue with parameterized rules

A grammar like e.g.:

"
:default ::= action => ::shift
top ::= . => parameterizedRhs->(1)

with the following generator action (writen in C#):

        public string parameterizedRhs(object[] args)
        {
            int parameter = Convert.ToInt32(args[0]);
            int origParameter = parameter;

            this.nbParameterizedRhsCalls++;
            string output;

            if (this.nbParameterizedRhsCalls == 5)
            {
                output = "'5'";
            }
            else if (this.nbParameterizedRhsCalls > 5)
            {
                output = "'no match'";
            }
            else
            {
                ++parameter;
                output = $". => parameterizedRhs->({parameter})";
            }

            logger.LogInformation($"parameterizedRhs({origParameter}) returns: {output}");
            return $"{output}\n";
        }

will fail to compile like this:

info: Program[0] parameterizedRhs(1) returns: . => parameterizedRhs->(2)
info: Program[0] parameterizedRhs(2) returns: . => parameterizedRhs->(3)
info: Program[0] parameterizedRhs(3) returns: . => parameterizedRhs->(4)
fail: Program[0] Symbol Internal[2] is defined multiple times with different number of parameters (0 != -1)
fail: Program[0] Action 063_G1_action_start_rule failed for rule: <start rule> ::= ':start' <op declare> <start symbol> action => 063_G1_action_start_rule name => 'G1_rule_start_rule'
fail: Program[0] Rule No 20 value callback failure
fail: Program[0] parameterizedRhs callback returned a string that cannot be converted to a grammar
fail: Program[0]
fail: Program[0] --------------------------------------------
fail: Program[0] Recognizer progress (grammar level 0 (Grammar level 0)):
fail: Program[0] [[email protected]] top ::= . .
fail: Program[0]
fail: Program[0] Expected symbol: . (symbol No 1)
fail: Program[0]
fail: Program[0] <<<<<< FAILURE HERE: >>>>>>
fail: Program[0] --------------------------------------------

This is because a generated grammar inherits a parent grammar in _marpaESLIFGrammar_newp:

  /* Do we have a parent bootstrap ? This can happen only when we process a generator action */
  marpaESLIFGrammar_bootstrapp = marpaESLIFGrammarp->marpaESLIFGrammar_bootstrapp =
    (marpaESLIFGrammar_bootstrapOrigp == NULL)
    ? _marpaESLIFGrammar_bootstrap_newp(marpaESLIFp)
    : _marpaESLIFGrammar_bootstrap_clonep(marpaESLIFp, marpaESLIFGrammar_bootstrapOrigp);
  if (MARPAESLIF_UNLIKELY(marpaESLIFGrammarp->marpaESLIFGrammar_bootstrapp == NULL)) {
    goto err;
  }

And later on grammar validation reuses the bootstrap's internalRuleCounti variable like this:

    sprintf(tmps, "Internal[%d]", marpaESLIFGrammarBootstrapp->internalRuleCounti++);

So if a bootstrap grammar is used and have e.g. internalRuleCounti == 2, the validation will try to reuse Internal[2] instead of consuming a new Internal[3] meta symbol.