foo123 / editor-grammar Goto Github PK

invariant codebase of editor-grammar (codemirror-grammar, ace-grammar,prism-grammar,syntaxhighlighter-grammar, highlightjs-grammar,..)

Home Page: https://foo123.github.io/

JavaScript 100.00%

grammar-specification bnf peg syntax-highlight-parser

editor-grammar's Introduction

editor-grammar

Invariant Codebase of editor-grammar

Used as a git submodule, for common invariant code, to:

see also:

Abacus advanced Combinatorics and Algebraic Number Theory Symbolic Computation library for JavaScript, Python
Plot.js simple and small library which can plot graphs of functions and various simple charts and can render to Canvas, SVG and plain HTML
HAAR.js image feature detection based on Haar Cascades in JavaScript (Viola-Jones-Lienhart et al Algorithm)
HAARPHP image feature detection based on Haar Cascades in PHP (Viola-Jones-Lienhart et al Algorithm)
FILTER.js video and image processing and computer vision Library in pure JavaScript (browser and node)
Xpresion a simple and flexible eXpression parser engine (with custom functions and variables support), based on GrammarTemplate, for PHP, JavaScript, Python
Regex Analyzer/Composer Regular Expression Analyzer and Composer for PHP, JavaScript, Python
GrammarTemplate grammar-based templating for PHP, JavaScript, Python
codemirror-grammar transform a formal grammar in JSON format into a syntax-highlight parser for CodeMirror editor
ace-grammar transform a formal grammar in JSON format into a syntax-highlight parser for ACE editor
prism-grammar transform a formal grammar in JSON format into a syntax-highlighter for Prism code highlighter
highlightjs-grammar transform a formal grammar in JSON format into a syntax-highlight mode for Highlight.js code highlighter
syntaxhighlighter-grammar transform a formal grammar in JSON format to a highlight brush for SyntaxHighlighter code highlighter
SortingAlgorithms implementations of Sorting Algorithms in JavaScript
PatternMatchingAlgorithms implementations of Pattern Matching Algorithms in JavaScript

editor-grammar's People

Contributors

Stargazers

Watchers

Forkers

rastapopougros snipervld jendas1 ingydotnet totalamd tubbz-alt tjdev7

editor-grammar's Issues

Left-recursive in syntax not working

In the manual, it says

a syntax token can contain (direct or indirect) recursive references to itself ( note: some rule factoring may be needed, to avoid grammar left-recursion or ambiguity )

When I define left-recursive rule in Syntax, I got the exception: Uncaught RangeError: Maximum call stack size exceeded

What do you mean by " some rule factoring may be needed"? Do you mean I need to convert left-recursion to non-left-recursion following rules like this: https://web.cs.wpi.edu/~kal/PLT/PLT4.1.2.html ?

Correct use of sequence

In the following line, I am trying to catch 'var', '=' and 'somevar' separately and assign styles.

var=somevar

I have defined in the lex section 'var', eq and 'subs', and these work well.

    ,"var"                      	: "RE::/(var)(?:(?=\\s)|(?=\\t)|(?==)|$)/gim"
    ,"eq"                      		: "RE::/\\s*=\\s*/gim"
    ,"subs"                   		: "RE::/[A-Za-z$][A-Za-z0-9$]*/"

I am having trouble with the 'somevar' part as I believe I can only identify it accurately as a subs type following after types var and eq.

My sequence is currently

"Syntax"                            : {
    "subs1"                  : {"sequence":["var", "eq", "subs"]}
    ,"js"                           : " comment | eq | operator | var | subs1 | subs | keyword  "
},

Am I being too hopeful, or is there a correct approach to the above requirement.

recursive grammar

Having the following grammar:

// a partial javascript grammar in simple JSON format
{
        
// prefix ID for regular expressions used in the grammar
"RegExpID"                          : "RE::",
    
// Style model
"Style"                             : {
    
     "comment"                      : "comment"
    ,"atom"                         : "atom"
    ,"keyword"                      : "keyword"
    ,"this"                         : "keyword"
    ,"builtin"                      : "builtin"
    ,"operator"                     : "operator"
    ,"identifier"                   : "variable"
    ,"property"                     : "attribute"
    ,"number"                       : "number"
    ,"string"                       : "string"
    ,"date"                         : "string-2"
    ,"regex"                        : "string-2"
    
},

// Lexical model
"Lex"                               : {
    
     "comment"                      : {"type":"comment","tokens":[
                                    // line comment
                                    // start, end delims  (null matches end-of-line)
                                    [  "//",  null ],
                                    // block comments
                                    // start,  end    delims
                                    [  "/*",   "*/" ]
                                    ]}
    ,"identifier"                   : "RE::/[_A-Za-z$][_A-Za-z0-9$]*/"
    ,"this"                         : "RE::/this\\b/"
    ,"property"                     : "RE::/[_A-Za-z$][_A-Za-z0-9$]*/"
    ,"date"                         : "RE::/\\d{4}-\\d{2}-\\d{2}$]*/"
    ,"number"                       : [
                                    // floats
                                    "RE::/\\d*\\.\\d+(e[\\+\\-]?\\d+)?/",
                                    "RE::/\\d+\\.\\d*/",
                                    "RE::/\\.\\d+/",
                                    // integers
                                    // hex
                                    "RE::/0x[0-9a-fA-F]+L?/",
                                    // binary
                                    "RE::/0b[01]+L?/",
                                    // octal
                                    "RE::/0o[0-7]+L?/",
                                    // decimal
                                    "RE::/[1-9]\\d*(e[\\+\\-]?\\d+)?L?/",
                                    // just zero
                                    "RE::/0(?![\\dx])/"
                                    ]
    ,"string"                       : {"type":"escaped-block","escape":"\\","tokens":
                                    // start, end of string (can be the matched regex group ie. 1 )
                                    [ "RE::/(['\"])/",   1 ]
                                    }
    ,"regex"                        : {"type":"escaped-block","escape":"\\","tokens":
                                    // javascript literal regular expressions can be parsed similar to strings
                                    [ "/",    "RE::#/[gimy]{0,4}#" ]
                                    }
    ,"operator"                     : {"tokens":[
                                    "+", "-", "++", "--", "%", ">>", "<<", ">>>",
                                    "*", "/", "^", "|", "&", "!", "~",
                                    ">", "<", "<=", ">=", "!=", "!==",
                                    "=", "==", "===", "+=", "-=", "%=",
                                    ">>=", ">>>=", "<<=", "*=", "/=", "|=", "&="
                                    ]}
    ,"delimiter"                    : {"tokens":[
                                    "(", ")", "[", "]", "{", "}", ",", "=", ";", "?", ":",
                                    "+=", "-=", "*=", "/=", "%=", "&=", "|=", "^=", "++", "--",
                                    ">>=", "<<="
                                    ]}
    ,"atom"                         : {"autocomplete":true,"tokens":[
                                    "null", "undefined", 
                                    "NaN", "Infinity"
                                    ]}
    ,"boolean"                      : {"autocomplete":true,"tokens":["true", "false"]}
    ,"keyword"                      : {"autocomplete":true,"tokens":[ 
                                    "if", "while", "with", "else", "do", "try", "finally",
                                    "return", "break", "continue", "new", "delete", "throw",
                                    "var", "const", "let", "function", "catch", "void",
                                    "for", "switch", "case", "default", "class", "import", "yield",
                                    "in", "typeof", "instanceof"
                                    ]}
    ,"builtin"                      : {"autocomplete":true,"tokens":[ 
                                    "Object", "Function", "Array", "String", 
                                    "Date", "Number", "RegExp", "Math", "Exception",
                                    "setTimeout", "setInterval", "parseInt", "parseFloat", 
                                    "isFinite", "isNan", "alert", "prompt", "console", 
                                    "window", "global", "this"
                                    ]}

},

// Syntax model (optional)
"Syntax"                            : {
    
    "dot_property"                  : {"sequence":[".", "property"]},
    "prim"                          : "number | string | boolean",
    "kv"                            : {"sequence": ["string", "':'", "json"]},
    "json"                          : "object | array | prim",
    "object"                        : "'{' kv (',' kv)* '}'",
    "array"                         : "'[' json (',' json)* ']'",
    "a" : "'\"and\"'",
    "relation"                      : "'[' op ',' operand ',' operand ']'",
    "op"                            : "'\"=\"' | '\">\"' | '\">=\"' | '\"<\"' | '\"<=\"'",
    "operand"                       : "identifier | date | string | number | boolean",    
    "criteria"                      : "relation | and | or",
    "and"                           : "'['  '\"and\"' (',' criteria)+ ']'",
    "or"                             : "'['   '\"or\"' (',' criteria)+ ']'",
    "query"                         : "'{' '\"query\"'    ':' string ',' '\"where\"' ':' criteria '}'",
    "top"                           : "query"
},

// what to parse and in what order
"Parser"                            : [ ["top"] ]

}

I'd expect the input :

{"query": "sdasdas",
 "where": 
  ["and",
     ["=", 1, 1],
     ["=", 1, 1],
     ["or", 
        ["=", 1, 1],
        ["=", 1, 1]
     ]
  ]   
}

would have no syntax errors, but the there is an error reported.
How do I define recursive grammar like this one?

SOF or OEF

I see in the document that ^^ defines start of file

I tried to define a language that only contains foo and bar
But the input is "foo bar foo bar", the second "foo bar" is still matched and styled, the ^^ does not seemed to work.

I wanna change syntax rule to allow trailing comma in JSON arrays, so I change
"literal_array" : "'[' (literal_value (',' literal_value)*)? ']'"
to
"literal_array" : "'[' (literal_value (',' literal_value)* (',')? )? ']'"
But seems it not works. What I do wrong?
Error:

error: Token "atom": true|false|null Expected | Token "string" Expected | Token "number" Expected | Token "[" Expected | Token "{" Expected

A textual transformation tool to transform usual EBNF specification to equivalent JSON specification (grammar)

This is to remind and propose (for anyone interested) that since EBNF grammar notations/specifications do not use JSON format, but a rather similar textual format, for example

EBNF

a_rule ::= symbol1
| symbol2
| 'literal symbol'
| ..
;

or even:

EBNF

a_rule ::= <symbol1>
| <symbol2>
| 'literal symbol'
| ..
;

Although it is rather straightforward to transform the above examples into a JSON grammar for highlight. It is even better to have a transformation tool for that.

Given that JSON grammar (as used by *-grammar highlighter add-ons) can accept and parse EBNF/PEG notation. The task is even easier. For example, just split each rule as rule_name => rule_definition and create a JSON-like object from that (more correctly the Syntax part of the overall JSON grammar).

Textmate

I would like to convert textmate JSON language definitions (like those used in VSCode) to this format (so I can convert to other things.) Is this possible? Has someone already made a tool for this?

foo123 / editor-grammar Goto Github PK

editor-grammar's Introduction

editor-grammar

editor-grammar's People

Contributors

Stargazers

Watchers

Forkers

editor-grammar's Issues

Left-recursive in syntax not working

Correct use of sequence

recursive grammar

SOF or OEF

syntax rule not working

A textual transformation tool to transform usual EBNF specification to equivalent JSON specification (grammar)

Textmate

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent