usethesource / rascal-syntax-highlighting Goto Github PK

Currently all elements between two brackets [] are treated naïvely as lexical character set, which results in issues with highlighting list literals. Particularly, we must ensure that only things between brackets inside lexical or syntax scopes are treated as character sets, and those outside are treated as list literals.

Note that removing the support for lexical character sets ruins the highlighting of the rest of the programs since \" and "/", and other similar characters can appear freely in character set expressions.

Examples of mis-highlighted terms:

// comment 1

n = 1;  // comment 2
l = [1, // comment 3: non highlighted
     2, 
     3];

list[int] n

private MuExp translateStringLiteral(s: (StringLiteral) `<PreStringChars pre> <Expression expression> <StringTail tail>`) {
    preIndent = computeIndent(pre);
    return muBlock( [ muCallPrim3("template_open", translatePreChars(pre), pre@\loc),
    				  *translateExpInStringLiteral(preIndent, expression),
    				  *translateTail(preIndent, tail),
    				  muCallPrim3("template_close", [], tail@\loc)
					]   );
}

The following tutorial http://www.apeth.com/nonblog/stories/textmatebundle.html provides many useful suggestions on how to highlight entities in a correct way. Recheck grammar and see if it is possible to align it with the recommendations provided by the tutorial.

Currently, URL literals, e.g. |http://google.com| are not highlighted as they optimally should be.
It is not possible to just use begin: | end: | since operators like || exist and concrete and abstract syntax uses them, so one should be careful in the way they are recognized.

The best approach would be to follow the URL grammar, albeit it's a bit complex since it can contain interpolations as well as having various different parts.

This is a question rather than an issue.

At the moment we are working on new infrastructure for our Rascal documentation that is based on Asciidoctor (see http://asciidoctor.org). Asciidoctor supports highlighting in various ways (see http://asciidoctor.org/docs/user-manual/#source-code-blocks). Now the question is: can we convert the nice Textmate rules under development by @ahmadsalim to input for any of the supported systems by Asciidoctor:

• http://coderay.rubychan.de
• https://highlightjs.org
• https://github.com/google/code-prettify
• http://pygments.org

I would prefer to have less Ruby/Python and more Javascript in the solution we will adopt.

Any thoughts?

I discovered the issue while I was doing something else.
First of all there is a small bug in the grammar which was unnoticed:

Which is that in the regex: rule we should replace

begin: "/(?!/|*)"

with

begin: "/(?!/|\\*)"

Without fixing this issue the Atom highlighter breaks on certain files, but the TM/Sublime highlighter is also wrong.

Fixing this issue brings another issue with regards to division being accidentally seen as regex literals by the highlighter.

Most files contain a license header, and a lot of standard functions contain documentation tags.
The body of them should be marked as comments.

@license{
  BSD-2 License
}
module A

@doc{
   Textile markup for the documentation
}
@javaClass{org.rascalmpl.library.X}
public java str doStuff(int x);

they should be marked as comments instead of normal text