Self-explanatory.

In this first implementation, the goal is to make it easy to parse and apply transformations on the parsed items.

Combinators:

• (:char #\a) test the next element on the stream with the given char.
• (:one #'fn) test using a function to get the next char in the stream.
• (:string "abc") test if the string is in the stream.
• (:and exp ...) collect many expressions.
• (:or exp ...) collect the expression that matches.
• (:many exp) collect all matches of the expression, if any.
• (:maybe exp) turns the failed test into a valid one.

Declaration

A macro is used to declare all rules for the BNF.

(:= word (:many (:one #'alpha-char-p)) :call #'stringify)

Valid transformations:

• :call apply a function to the results using funcall.
• :apply apply a function to the results using apply.
• :tag return a cons like (cons TAG RESULTS).
• If none where specified, it return a nested list of all matches.

Example

Parsing a valid json number:

(load #P"~/projects/cl-bnf/cl-bnf.lisp")

(:= decimal-number (:many (:one #'numeric-char-p)))
(:= real-number (:or (:and #'decimal-number
                           (:char #\.)
                           #'decimal-number)
                     (:and #'decimal-number
                           (:char #\.))))
(:= signed-part (:or (:char #\+) (:char #\-)))
(:= exp-chars (:or (:char #\e)
                   (:char #\E)))
(:= exp-part (:or (:and #'exp-chars
                        #'signed-part
                        #'decimal-number)
                  (:and #'exp-chars
                        #'decimal-number)))
(:= numeric (:or #'real-number
                 #'decimal-number))
(:= number-literal (:or (:and #'numeric
                              #'exp-part)
                        #'numeric)
    :call (lambda (matches)
            (cons :number (stringify matches))))

(parse #number-literal "1e3")

Maybe there is something to do to simplify the rule declaration.

In the file src/bnf.lisp it uses sequence:position several times.

Where is the package "SEQUENCE" supposed to come from?

Isn't it just the standard function position?

Improve documentation for the project.

https://github.com/diasbruno/cl-bnf/blob/master/src/bnf.lisp#L180 should be moved to the defined-rule macro.

The macro := is named by the symbol named = in the keyword package. The exported symbol cl-bnf:|:=| has nothing to do with it.

As soon as the system is loaded, the macro is available from the keyword package:

CL-USER> (ql:quickload "cl-bnf")
To load "cl-bnf":
  Load 1 ASDF system:
    asdf
  Install 1 Quicklisp release:
    cl-bnf
; Fetching #<URL "http://beta.quicklisp.org/archive/cl-bnf/2018-07-11/cl-bnf-20180711-git.tgz">
; 4.94KB
==================================================
5,054 bytes in 0.00 seconds (0.00KB/sec)
; Loading "cl-bnf"
[package cl-bnf].
("cl-bnf")
CL-USER> (:= foo (:char #\a))
FOO

CL-USER> (describe 'cl-bnf:|:=|)
CL-BNF:|:=|
  [symbol]
; No value
CL-USER> (describe :=)
:=
  [symbol]

= names a constant variable:
  Value: :=

= names a macro:
  Lambda-list: (LABEL RULE &KEY CALL TAG APPLY)
  Documentation:
    Generate a function LABEL to parse RULE. Later,
    you can apply a TRANSFORMATION which can be a function
    or a keytword.
  Source file: /home/svante/quicklisp/dists/quicklisp/software/cl-bnf-20180711-git/bnf.lisp
; No value
CL-USER>

I do not think that this was intended, and I feel that it is less than ideal that the (global) keyword package gets filled with library code.

**define-grammar ** in src/bnf.lisp defines the rules only at compile time, so it doesn't work if you load the compiled file without actually compiling.

For example, (ql:quickload "cl-bnf-examples") so it compile and load everything. Quit the image you are running and start it again. Then (ql:quickload "cl-bnf-examples") again. This loads the compiled files, and errors because none of the rules in the json grammar are defined.

Other operators we can bring in:

:one-of

(:one-of "abc" #'number-char-p)

(:or (:char #\a) (:char #\b) (:char #\c) #'number-char-p)

:between

(:between (:char #\a) #'number)

(:and (:char #\a) #number (:char #\a))

Maybe we can borrow some ideas from this package and try compile to be more efficient.