Git Product home page Git Product logo

apg-js's Introduction

JavaScript APG

4.4.0 Release Notes

Version 4.4.0 adds an option, --typescript, to the APG genererator which will generate a typescript-compatible grammar object. The new script regen will regenerate all the grammar objects in place anytime a change is made to any of the ABNF grammars or if the generator is modified in any way.

4.3.0 Release Notes

Version 4.3.0 adds support for apg-lite, a new, light-weight version of the parsers built from this library. The parser generator (src/apg/apg.js) has a new option, --lite, that will generate grammar objects for the apg-lite parsers. See the documentation for apg-lite for a complete description.

Version 4.3.0 makes no other changes to the previous version other than adding the new apg-lite feature.

4.2.1 Release Notes

Some ABNF grammar authors will use zero repetitions as an explicit empty string acceptor. That is, one or the other of something like,

char = %d33-127
empty1 = 0char
empty2 = 0"x"

Previous version of APG have rejected zero repititions as an empty string acceptor in favor of the more intuitive and simpler to produce and simpler to process empty literal string

empty = ""

apg-js version 4.2.0 and lower neglects to reject the zero repetitions form and, unfortunately, it only sometimes actually accepts an empty string and sometimes fails. Version 4.2.1 corrects this. Zero repetitions are allowed as explicit empty string acceptors and are processed correctly.

Note, however, that zero repetitions is deprecated in that not all implementations of APG accept it and that is it is slightly less efficient to implement. Favored is the more streamlined empty literal string, ""

4.2.0 Release Notes

Version 4.2.0 fixes some issues that have been causing problems for some bundlers and/or development tools. ./src/apg-conv-api/converter.js, ./src/apg-conv-api/transformers.js and ./src/apg-lib/utilities.js now refer explicitly to the exported functions rather than relying on a saved copy of the this reference. Also, use of the global Buffer object has been replaced with

const { Buffer } = require('buffer');

and similary for fs and other global node.js properties, as recommended in the Node.js v20.5.1 documentation. (Note that require('node:buffer') causes problems with using browerify to bundle the libraries for web page use. Removing the node: prefix solves that problem without causing problems with the bundle usage. Thanks to contributions in the GitHub issue #13 thread for this suggestion.)

This updated version of apg-js has been tested with all of the examples in apg-js-examples. All examples including the web page examples work as expected.

Also, all dependencies have been moved to devDependencies, removing the need for any globally installed packages.

Overview

apg-js is the JavaScript version of APG, an ABNF Parser Generator. APG generates recursive-descent parsers directly from a superset of ABNF (SABNF). Visit the APG website for a complete overview of APG and SABNF.

apg-js obsoletes apg-js2, apg-js2-lib, apg-js2-exp, apg-js2-api, apg-conv and apg-conv-api. It changes them in two significant ways.

  • It fixes a major problem in the attributes algorithm. Previous versions could fail on large grammars with many or large sets of mutually-recursive rules.
  • It combines all of those packages into a single, easier to manage and maintain package.

Documentation

This package is meant to assist other parsing applications and is normally not installed by itself, rather installed along with those other applications. For details and many examples of using of the libraries, both in node.js and browser web page applications, see apg-js-examples at GitHub or npmjs. However, it does provide access to two, node.js applications, apg and apg-conv.

Applications

apg is the parser generator. To see its usage run,

npm run apg -- --help

apg-conv is a data conversion application. To see its usage run,

npm run apg-conv -- --help

Libraries

This package contains four libraries that can be used in either node.js or browser applications. The libraries depend upon one another and the dependency tree looks like so:

apg-exp
|- apg-api
|-|- apg-lib
|-|-|- apg-conv-api

Each of the libraries is bundled for browser use along with some special styles. Each bundle contains all of its dependencies explicitly. That is, if a browser application needs both apg-api and apg-lib, only the apg-api bundle need be scripted into the page.

The library and css bundles are in the ./dist directory.

./dist/apg-exp-bundle.js
./dist/apg-api-bundle.js
./dist/apg-lib-bundle.js
./dist/apg-conv-api-bundle.js
./dist/apg-lib-bundle.css

The bundles can all be regenerated with the scripts:

npm run bundle-apg-conv-api
npm run bundle-apg-lib
npm run bundle-apg-api
npm run bundle-apg-exp
npm run bundle-apg-lib-css

Code Documentation

The code documentation is in docco format and can be generated with:

npm run docco

The documentation is then at ./docs/index.html or see it here at the APG website.

apg-js's People

Contributors

ldthomas avatar matthewleon avatar ralfhandl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

apg-js's Issues

parsing of concatenation with initial variable repetition fails incorrectly

This is a hard one (for me) to explain, so let's take an example. Take the following minimal grammar:

const GRAMMAR = String.raw`full = *ab b
ab = "a" / "b"
b = "b"
`;

The following string should successfully be parsed by the grammar:

const EXAMPLE = "b";

like so:

const api = new apgApi(GRAMMAR);
api.generate();
if (api.errors.length) {
  console.error(api.errorsToAscii());
  throw `grammar has errors: ${GRAMMAR}`;
}
const schema = api.toObject();
const parser = new apgLib.parser();
const result = parser.parse(schema, 0, EXAMPLE);
console.log(result);

This fails, though, with the following result:

{
  success: false,
  state: 103,
  length: 1,
  matched: 0,
  maxMatched: 1,
  maxTreeDepth: 6,
  nodeHits: 9,
  inputLength: 1,
  subBegin: 0,
  subEnd: 1,
  subLength: 1
}

The reason for the failure seems to be that the initial repetition rule, *ab, is applied greedily, swallowing the single character b. The parser never then checks for the possibility of zero matches for repetition *ab, followed by a successful match of rule b.

This appears to be inconsistent with RFC 5234.

Improve error logging

Currently the command-line tool apg.sh first outputs errors in the input grammar, then lists the complete input, which can be rather long.

It took me a while to realize that the tool gives more hints on the errors than just

EXCEPTION THROWN: Error: apg.js: grammar has syntax errors

because the detailed error message was scrolled out of sight ๐Ÿ˜„

It would improve use on the command-line if the error details would be written after the input grammar.

This change is proposed in #4

parsing of concatenation with initial optional sequence fails incorrectly

This is a follow up to #6

Take this example grammar:

const GRAMMAR = String.raw`full = [ab] b
ab = "a" / "b"
b = "b"
`;

and test string:

const EXAMPLE = "b";

The string should parse successfully. It does not, due to the same reasoning as the issue above. [ab] produces a match, and the parser never seems to "back up" and consider whether rule b alone can produce a success, without the application of [ab].

`function exports(` is controversial.

Hi! ๐Ÿ‘‹

Firstly, thanks for your work on this project! ๐Ÿ™‚

Today I used patch-package to patch [email protected] for the project I'm working on.

I've been trying to build the project using parcel and one of the parsers involved in that was exploding because exports has a meaning and naming something else that was tripping it up. Giving a function a name means it's available under that name in the scope even if it was declared inline, so some parsers will have trouble with it as redefining exports. Meanwhile, the name is not useful for anything as far as I can see.

Here is the diff that solved my problem:

diff --git a/node_modules/apg-js/src/apg-api/scanner.js b/node_modules/apg-js/src/apg-api/scanner.js
index bde1a2b..63ab73e 100644
--- a/node_modules/apg-js/src/apg-api/scanner.js
+++ b/node_modules/apg-js/src/apg-api/scanner.js
@@ -13,7 +13,7 @@
 // - catalog the lines - create an array with a line object for each line.
 // The object carries information about the line number and character length which is used
 // by the parser generator primarily for error reporting.
-module.exports = function exports(chars, errors, strict, trace) {
+module.exports = function (chars, errors, strict, trace) {
   const thisFileName = 'scanner.js: ';
   const apglib = require('../apg-lib/node-exports');
   const grammar = new (require('./scanner-grammar'))();
diff --git a/node_modules/apg-js/src/apg-api/semantic-callbacks.js b/node_modules/apg-js/src/apg-api/semantic-callbacks.js
index d57125b..7407ed1 100644
--- a/node_modules/apg-js/src/apg-api/semantic-callbacks.js
+++ b/node_modules/apg-js/src/apg-api/semantic-callbacks.js
@@ -7,7 +7,7 @@
 // See:<br>
 // `./dist/abnf-for-sabnf-grammar.bnf`<br>
 // for the grammar file these callback functions are based on.
-module.exports = function exports() {
+module.exports = function () {
   const apglib = require('../apg-lib/node-exports');
   const id = apglib.ids;
 
diff --git a/node_modules/apg-js/src/apg-api/show-rules.js b/node_modules/apg-js/src/apg-api/show-rules.js
index 7121cb3..281d08c 100644
--- a/node_modules/apg-js/src/apg-api/show-rules.js
+++ b/node_modules/apg-js/src/apg-api/show-rules.js
@@ -2,7 +2,7 @@
  *   copyright: Copyright (c) 2021 Lowell D. Thomas, all rights reserved
  *     license: BSD-2-Clause (https://opensource.org/licenses/BSD-2-Clause)
  *   ********************************************************************************* */
-module.exports = (function exports() {
+module.exports = (function () {
   const thisFileName = 'show-rules.js';
   // Display the rules.
   // This function may be called before the attributes calculation.
diff --git a/node_modules/apg-js/src/apg-api/syntax-callbacks.js b/node_modules/apg-js/src/apg-api/syntax-callbacks.js
index 992ecce..c007a4b 100644
--- a/node_modules/apg-js/src/apg-api/syntax-callbacks.js
+++ b/node_modules/apg-js/src/apg-api/syntax-callbacks.js
@@ -7,7 +7,7 @@
 // See:<br>
 // `./dist/abnf-for-sabnf-grammar.bnf`<br>
 // for the grammar file these callback functions are based on.
-module.exports = function exports() {
+module.exports = function () {
   const thisFileName = 'syntax-callbacks.js: ';
   const apglib = require('../apg-lib/node-exports');
   const id = apglib.ids;

This issue body was partially generated by patch-package.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.