foo123 / codemirror-grammar Goto Github PK

View Code? Open in Web Editor NEW

80.0 9.0 8.0 4.87 MB

Transform a JSON grammar into a syntax-highlight parser for CodeMirror

Home Page: https://foo123.github.io/examples/codemirror-grammar/

JavaScript 94.90% CSS 1.17% HTML 3.74% Batchfile 0.19%

bnf peg codemirror-editor grammar-parser syntax-highlighter

codemirror-grammar's People

Contributors

Stargazers

Watchers

Forkers

marcosvega91 mcanthony snipervld sammerset tubbz-alt

codemirror-grammar's Issues

grammar-scheme.html is incorrect in v2.3.0

I guess scheme.js needs some thing like this at line 69

//"combine": false,
"combine-before": "\s, (, ), [, ]",
"combine-after": "\s, (, ), [, ]",

Getting Grammar mode through npm

Hello,

Would you considere to provide this library as a npm package ?
It will be great to be able to do that, because it makes it difficult to use it when you are going use it through an angular/react application for example.

Thanks :).

Case insensitive keywords etc

Hi - great utility. One question - how can I set the keywords, for example, to be case-insensitive ?

Using the code like in example throws CodeMirror.Pos is not a function

Using
codermirror-grammar: master
codemirror: 5.37.0

Trying this code to have autocomplete work:

// vendors
import React from 'react';
import PropTypes from 'prop-types';
import CodeMirror from 'codemirror';
import 'codemirror/addon/hint/show-hint';
import 'codemirror/addon/lint/lint';
import CodeMirrorGrammar from '../../../../../../vendors/codemirror_grammar';
import grammar from './Scenario.grammar.json';

class ReactCodeMirror extends React.PureComponent {
  constructor(props) {
    super(props);
    this.state = {};
    this.editor = null;
    this.textAreaRef = React.createRef();
    this.mainLang = 'scenario-lang';
    this.codeMode = CodeMirrorGrammar.getMode(grammar);
    this.codeMode.supportAutoCompletion = true;
    this.codeMode.autocompleter.options = {
      prefixMatch: true,
      caseInsensitiveMatch: true,
      inContext: true,
    };
    CodeMirror.defineMode(this.mainLang, this.codeMode);
    this.autoCompleteCommand = `autocomplete_grammar_${this.mainLang}`;
    this.editorOptions = {
      hint: CodeMirror.hint.auto,
      lineNumbers: false,
      theme: 'the-matrix',
      readOnly: false,
      indentUnit: 1,
      indentWithTabs: false,
      mode: this.mainLang,
      lint: true,
      extraKeys: {
        'Ctrl-Space': this.autoCompleteCommand,
      },
      [this.codeMode.matchType]: true,
    };
    CodeMirror.registerHelper('lint', this.mainLang, this.codeMode.linter);
    CodeMirror.defineOption(this.codeMode.matchType, false, (cm, val, old) => {
      if (old && (old !== CodeMirror.Init)) {
        cm.off('cursorActivity', this.codeMode.matcher);
        this.codeMode.matcher.clear(cm);
      }
      if (val) {
        cm.on('cursorActivity', this.codeMode.matcher);
        this.codeMode.matcher(cm);
      }
    });
    CodeMirror.commands[this.autoCompleteCommand] = (cm) => {
      console.debug('Showing autocomplete', cm, this.codeMode.autocompleter);
      try {
        CodeMirror.showHint(cm, this.codeMode.autocompleter);
        // cm.showHint(this.codeMode.autocompleter);
        console.debug('>> hints shown');
      } catch (error) {
        console.error('Could not show hint', error);
      }
    };
    console.log(CodeMirror);
  }
  componentDidMount() {
  }
  componentDidUpdate() {
    this.injectCodeMirror();
  }
  injectCodeMirror() {
    if (this.editor) return;
    this.editor = CodeMirror.fromTextArea(this.textAreaRef.current, this.editorOptions);
  }
  updateCodeMirror() {
  }
  render() {
    return (
      <div className="ReactCodeMirror">
        <textarea ref={this.textAreaRef} value={this.props.value} onChange={this.props.onChange} />
      </div>
    );
  }
}
ReactCodeMirror.defaultProps = {
  options: {},
  value: '',
};
ReactCodeMirror.propTypes = {
  options: PropTypes.object, // eslint-disable-line
  value: PropTypes.string,
  onChange: PropTypes.func.isRequired,
};

Then an exception occurs:

Could not show hint TypeError: CodeMirror.Pos is not a function
    at CodeMirrorParser.autocomplete (codemirror_grammar.js?933b:5098)
    at autocompleter (codemirror_grammar.js?933b:5336)
    at fetchHints (show-hint.js?8d07:363)
    at Completion.update (show-hint.js?8d07:113)
    at CodeMirror$1.eval [as showHint] (show-hint.js?8d07:46)
    at Function.CodeMirror.showHint (show-hint.js?8d07:24)
    at

It seems that the CodeMirror object has no Pos property

What could it be ?

Getting context for autocompletion

Hi. I'm currently writing grammar for one language using CodeMirror Grammar and it's awesome!
Now, i'm implementing autocompletion hints.
Ex, type "SELECT * FROM" -> Press "Space" -> Will show list with keywords:

Table1
Table2
FooTable
and others

I implemented this behaviour (with lint-"magic"), but HOW to do this after press "Space" without typing any char? In particularly, how to get expected keyword groups (for example, grouped by meta property)?

lint-"magic": when user types, for example, "SE", hint-interceptor call editor.performLint() to get all errors and parses this list to understand current "context" (appropriate words). (i.e. keyword "SET" cannot be at the start of SQL statement, only "SELECT" (and others))
Thanks! ~~Sorry for my bad English...~~

problem with Сyrillic letters

Hello, Nikos!

Thank you very much for Codemirror Grammar on GitHub. It's brilliant.

Unfortunately, when I used it, I had one difficulty. When I define the "keywords" of a grammar I use Cyrillic letters. For such words, auto-substitution does not work.

Could you please suggest how to solve this problem?

Thank you very much in advance!

Michael

code completion

I have the following grammar:

{
  "RegExpID": "RE::",
  "Style": {
    "string": "string",
    "this": "keyword",
    "operator": "operator",
    "regex": "string-2",
    "atom": "atom",
    "identifier": "variable",
    "keyword": "keyword",
    "number": "number",
    "property": "attribute",
    "date": "string-2",
    "comment": "comment",
    "builtin": "builtin"
  },
  "Lex": {
    "boolean": {
      "autocomplete": true,
      "tokens": [
        "true",
        "false"
      ]
    },
    "string": {
      "type": "escaped-block",
      "escape": "\\",
      "tokens": [
        "RE::/(['\"])/",
        1
      ]
    },
    "this": "RE::/this\\b/",
    "operator": {
      "tokens": [
        "+",
        "-",
        "++",
        "--",
        "%",
        ">>",
        "<<",
        ">>>",
        "*",
        "/",
        "^",
        "|",
        "&",
        "!",
        "~",
        ">",
        "<",
        "<=",
        ">=",
        "!=",
        "!==",
        "=",
        "==",
        "===",
        "+=",
        "-=",
        "%=",
        ">>=",
        ">>>=",
        "<<=",
        "*=",
        "/=",
        "|=",
        "&="
      ]
    },
    "regex": {
      "type": "escaped-block",
      "escape": "\\",
      "tokens": [
        "/",
        "RE::#/[gimy]{0,4}#"
      ]
    },
    "atom": {
      "autocomplete": true,
      "tokens": [
        "null",
        "undefined",
        "NaN",
        "Infinity"
      ]
    },
    "delimiter": {
      "tokens": [
        "(",
        ")",
        "[",
        "]",
        "{",
        "}",
        ",",
        "=",
        ";",
        "?",
        " ",
        "+=",
        "-=",
        "*=",
        "/=",
        "%=",
        "&=",
        "|=",
        "^=",
        "++",
        "--",
        ">>=",
        "<<="
      ]
    },
    "identifier": "RE::/[_A-Za-z$][_A-Za-z0-9$]*/",
    "keyword": {
      "autocomplete": true,
      "tokens": [
        "if",
        "while",
        "with",
        "else",
        "do",
        "try",
        "finally",
        "return",
        "break",
        "continue",
        "new",
        "delete",
        "throw",
        "var",
        "const",
        "let",
        "function",
        "catch",
        "void",
        "for",
        "switch",
        "case",
        "default",
        "class",
        "import",
        "yield",
        "in",
        "typeof",
        "instanceof"
      ]
    },
    "number": [
      "RE::/\\d*\\.\\d+(e[\\+\\-]?\\d+)?/",
      "RE::/\\d+\\.\\d*/",
      "RE::/\\.\\d+/",
      "RE::/0x[0-9a-fA-F]+L?/",
      "RE::/0b[01]+L?/",
      "RE::/0o[0-7]+L?/",
      "RE::/[1-9]\\d*(e[\\+\\-]?\\d+)?L?/",
      "RE::/0(?![\\dx])/"
    ],
    "property": "RE::/[_A-Za-z$][_A-Za-z0-9$]*/",
    "date": "RE::/\\d{4}-\\d{2}-\\d{2}$]*/",
    "comment": {
      "type": "comment",
      "tokens": [
        [
          ";",
          "null"
        ],
        [
          "/*",
          "*/"
        ]
      ]
    },
    "builtin": {
      "autocomplete": true,
      "tokens": [
        "Object",
        "Function",
        "Array",
        "String",
        "Date",
        "Number",
        "RegExp",
        "Math",
        "Exception",
        "setTimeout",
        "setInterval",
        "parseInt",
        "parseFloat",
        "isFinite",
        "isNan",
        "alert",
        "prompt",
        "console",
        "window",
        "global",
        "this"
      ]
    }
  },
  "Syntax": {
    "json": "object | array | prim",
    "object": "'{' kv (',' kv)* '}'",
    "js": "comment | number | string | regex | keyword | operator | atom | (('}' | ')' | this | builtin | identifier | dot_property) dot_property*)",
    "kv": {
      "sequence": [
        "string",
        "':'",
        "json"
      ]
    },
    "top": "query",
    "prim": "number | string | boolean",
    "query": "'{' '\"query\"'    ':' string ',' '\"where\"' ':' criteria '}'",
    "operand": "identifier | date | string | number | boolean",
    "operator2": "'\"=\"' | '\">\"' | '\">=\"' | '\"<\"' | '\"<=\"'",
    "or" :          "\"or\" (',' criteria)+",
    "and":          "\"and\" (',' criteria)+",
    "criteria": "'[' (or | and | relation)    ']'",
    "dot_property": {
      "sequence": [
        ".",
        "property"
      ]
    },
    "array": "'[' json (',' json)* ']'",
    "relation": "'[' operator2 ',' operand ',' operand ']'"
  },
  "Parser": [
    [
      "top"
    ]
  ]
}

When I paste it to the grammar editor on http://foo123.github.io/examples/codemirror-grammar/ and clean the code editor the Ctrl+SPACE suggest true | false which is not valid language according to grammar as it must start with {.
Furthermore, if I type { and then Ctrl+SPACE it correctly suggests "query" but when I subsequently use Ctrl+SPACE instead of : or : " it suggests "where" which again results in incorrect syntax.

Please advise whether it is a bug or I'm doing something wrong.
Many thanks for your work.

Autocomplete not work

Hi, I have download your project and I open the web page with the javascript-recursive grammar that has the autocomplete set to true but It doesn't work why?

Sequence query

In the following line, I am trying to catch 'var', '=' and 'somevar' separately and assign styles.

var=somevar

I have defined in the lex section 'var', eq and 'subs', and these work well.

    ,"var"                      	: "RE::/(var)(?:(?=\\s)|(?=\\t)|(?==)|$)/gim"
    ,"eq"                      		: "RE::/\\s*=\\s*/gim"
    ,"subs"                   		: "RE::/[A-Za-z$][A-Za-z0-9$]*/"

I am having trouble with the 'somevar' part as I believe I can only identify it accurately as a subs type following after types var and eq.

My sequence is currently

"Syntax"                            : {
    "subs1"                  : {"sequence":["var", "eq", "subs"]}
    ,"js"                           : " comment | eq | operator | var | subs1 | subs | keyword  "
},

Am I being too hopeful, or is there a correct approach to the above requirement.

Sample grammar for markdown (or other lightweight syntax)

Hello,
do you know if there is an example of a grammar syntax for markdown or any other lightweight markup ?

I want to describe a grammar for another lightweight syntax, but all examples is for programming languages (exept XML, but there is always start/end tags around, whereas in lightweight syntax there is only 1 or 2 char starting the line, for ex in lists, etc).

So if you have info on that…

Thanks in advance.

problem with highlighting

Hi, Nikos!

Could you help with one problem: i made json-file with grammar (include cyrillic symbols).
All works fine, but tokens "и", "И" behave strangely =)
In word "Привет" symbol "и" will be highlighted (but it's just a letter in this case) ((

Grammar:
`// 1. a partial javascript grammar in simple JSON format

var ldsl_grammar = // a partial javascript grammar in simple JSON format

{

// prefix ID for regular expressions used in the grammar

"RegExpID" : "RE::",

// Style model

"Style" : {

 "comment"                      : "comment"

,"atom"                         : "atom"

,"keyword"                      : "keyword"

,"builtin"                      : "builtin"

,"operator"                     : "operator"

,"identifier"                   : "variable"

,"property"                     : "attribute"

,"number"                       : "number"

,"string"                       : "string"

// Lexical model

"Lex" : {

 "comment"                      : {"type":"comment","tokens":[

                                // line comment

                                // start, end delims  (null matches end-of-line)

                                [  "//",  null ],

                                // block comments

                                // start,  end    delims

                                [  "/*",   "*/" ]

                                ]}

,"identifier"                   : "RE::/[_A-Za-z$][_A-Za-z0-9$]*/"

,"property"                     : "RE::/[_A-Za-z$][_A-Za-z0-9$]*/"

,"number"                       : [

                                // floats

                                "RE::/\\d*\\.\\d+(e[\\+\\-]?\\d+)?/",

                                "RE::/\\d+\\.\\d*/",

                                "RE::/\\.\\d+/",

                                // integers

                                // decimal

                                "RE::/[1-9]\\d*(e[\\+\\-]?\\d+)?L?/",

                                // just zero

                                "RE::/0(?![\\dx])/"

                                ]

,"string"                       : {"type":"escaped-block","escape":"\\","tokens":

                                // start, end of string (can be the matched regex group ie. 1 )

                                [ "RE::/(['\"])/",   1 ]

                                }

,"operator"                     : {"tokens":[

                                "+", "-", "*", "/", "не"

                                ]}

,"delimiter"                    : {"tokens":[

                                "(", ")", "[", "]", ",", ":"

                                ]}

,"atom"                         : {"autocomplete":true,"tokens":[

                                "ИСТИНА", "ЛОЖЬ", "и", "или", "ИЛИ", "И"

                                ]}

,"keyword"                      : {"autocomplete":true,"tokens":[

                                "если", "Если", "ЕСЛИ", "в случае", "иначе", "тогда", "содержит", "то", "хоть один из", "ни одного из"

                                ]}

,"builtin"                      : {"autocomplete":true,"tokens":[

                                "КАЖДЫЙ", "ЛЮБОЙ", "ИМПЕРАТИВ", "ТРЕБОВАНИЕ", "Требование"

                                ]}

// Syntax model (optional)

"Syntax" : {

"ldsl"                           : "comment | number | string | keyword | operator | atom | builtin | identifier"

// what to parse and in what order

"Parser" : [ ["ldsl"] ]

};`

Uncaught RangeError: Maximum call stack size exceeded

with the following grammar:

{
  "RegExpID": "RE::",
  "Style": {
    "string": "string",
    "this": "keyword",
    "operator": "operator",
    "regex": "string-2",
    "atom": "atom",
    "identifier": "variable",
    "keyword": "keyword",
    "number": "number",
    "property": "attribute",
    "date": "string-2",
    "comment": "comment",
    "builtin": "builtin"
  },
  "Lex": {
    "boolean": {
      "autocomplete": true,
      "tokens": [
        "true",
        "false"
      ]
    },
    "string": {
      "type": "escaped-block",
      "escape": "\\",
      "tokens": [
        "RE::/(['\"])/",
        1
      ]
    },
    "this": "RE::/this\\b/",
    "operator": {
      "tokens": [
        "+",
        "-",
        "++",
        "--",
        "%",
        ">>",
        "<<",
        ">>>",
        "*",
        "/",
        "^",
        "|",
        "&",
        "!",
        "~",
        ">",
        "<",
        "<=",
        ">=",
        "!=",
        "!==",
        "=",
        "==",
        "===",
        "+=",
        "-=",
        "%=",
        ">>=",
        ">>>=",
        "<<=",
        "*=",
        "/=",
        "|=",
        "&="
      ]
    },
    "regex": {
      "type": "escaped-block",
      "escape": "\\",
      "tokens": [
        "/",
        "RE::#/[gimy]{0,4}#"
      ]
    },
    "atom": {
      "autocomplete": true,
      "tokens": [
        "null",
        "undefined",
        "NaN",
        "Infinity"
      ]
    },
    "delimiter": {
      "tokens": [
        "(",
        ")",
        "[",
        "]",
        "{",
        "}",
        ",",
        "=",
        ";",
        "?",
        " ",
        "+=",
        "-=",
        "*=",
        "/=",
        "%=",
        "&=",
        "|=",
        "^=",
        "++",
        "--",
        ">>=",
        "<<="
      ]
    },
    "identifier": "RE::/[_A-Za-z$][_A-Za-z0-9$]*/",
    "keyword": {
      "autocomplete": true,
      "tokens": [
        "if",
        "while",
        "with",
        "else",
        "do",
        "try",
        "finally",
        "return",
        "break",
        "continue",
        "new",
        "delete",
        "throw",
        "var",
        "const",
        "let",
        "function",
        "catch",
        "void",
        "for",
        "switch",
        "case",
        "default",
        "class",
        "import",
        "yield",
        "in",
        "typeof",
        "instanceof"
      ]
    },
    "number": [
      "RE::/\\d*\\.\\d+(e[\\+\\-]?\\d+)?/",
      "RE::/\\d+\\.\\d*/",
      "RE::/\\.\\d+/",
      "RE::/0x[0-9a-fA-F]+L?/",
      "RE::/0b[01]+L?/",
      "RE::/0o[0-7]+L?/",
      "RE::/[1-9]\\d*(e[\\+\\-]?\\d+)?L?/",
      "RE::/0(?![\\dx])/"
    ],
    "property": "RE::/[_A-Za-z$][_A-Za-z0-9$]*/",
    "date": "RE::/\\d{4}-\\d{2}-\\d{2}$]*/",
    "comment": {
      "type": "comment",
      "tokens": [
        [
          ";",
          "null"
        ],
        [
          "/*",
          "*/"
        ]
      ]
    },
    "builtin": {
      "autocomplete": true,
      "tokens": [
        "Object",
        "Function",
        "Array",
        "String",
        "Date",
        "Number",
        "RegExp",
        "Math",
        "Exception",
        "setTimeout",
        "setInterval",
        "parseInt",
        "parseFloat",
        "isFinite",
        "isNan",
        "alert",
        "prompt",
        "console",
        "window",
        "global",
        "this"
      ]
    }
  },
  "Syntax": {
    "json": "object | array | prim",
    "object": "'{' kv (',' kv)* '}'",
    "js": "comment | number | string | regex | keyword | operator | atom | (('}' | ')' | this | builtin | identifier | dot_property) dot_property*)",
    "kv": {
      "sequence": [
        "string",
        "':'",
        "json"
      ]
    },
    "top": "query",
    "prim": "number | string | boolean",
    "query": "'{' '\"query\"'    ':' string ',' '\"where\"' ':' criteria '}'",
    "operand": "identifier | date | string | number | boolean",
    "operator2": "'\"=\"' | '\">\"' | '\">=\"' | '\"<\"' | '\"<=\"'",
    "or" :          "\"or\" (',' criteria)+",
    "and":          "\"and\" (',' criteria)+",
    "criteria": "'[' (or | and | relation)    ']'",
    "dot_property": {
      "sequence": [
        ".",
        "property"
      ]
    },
    "array": "'[' json (',' json)* ']'",
    "relation": "'[' operator2 ',' operand ',' operand ']'"
  },
  "Parser": [
    [
      "top"
    ]
  ]
}

the following input:

{"query": "foo",
  "where"   :  ["<", 1 ,2]
}

causes:

codemirror_grammar.min.js:11 Uncaught RangeError: Maximum call stack size exceeded
    at oe (codemirror_grammar.min.js:11)
    at X (codemirror_grammar.min.js:10)
    at oe (codemirror_grammar.min.js:11)
    at X (codemirror_grammar.min.js:10)
    at oe (codemirror_grammar.min.js:11)
    at X (codemirror_grammar.min.js:10)
    at oe (codemirror_grammar.min.js:11)
    at X (codemirror_grammar.min.js:10)
    at oe (codemirror_grammar.min.js:11)
    at X (codemirror_grammar.min.js:10)