Failing examples for removing comments about stata_kernel HOT 10 CLOSED

https://gist.github.com/kylebarron/e46ce79911976c3aae9d7e98309b9222

from stata_kernel.

@mcaceresb Check out https://github.com/kylebarron/stata_kernel/blob/dev-stata-lexer/stata_kernel/stata_lexer.py

from stata_kernel.

I believe I've implemented correct comments. Working on strings then writing some tests.

from stata_kernel.

And yes I know some of those regexes could be cleaned up. It was mostly a port from language-stata.

from stata_kernel.

One issue that comes to mind with the lexer is that * in mata de-references a pointer, and is not a comment. I wonder if it has a way to implement different rules for different environments (this might also come in handy for #delimit ;, where the notion of a line changes).

from stata_kernel.

Yes that should be possible. Since Mata is only started with mata and ended with end (is this true?), I can easily find when Mata starts and ends and then just write a different lexer for Mata and use the Mata tokenizer on the Mata code.

from stata_kernel.

You need to be careful in that

1. mata can be preceded by any number of CAPture, QUIetly, or Noisily
2. mata can be followed by any number of spaces, :, and any number of spaces thereafter.
3. If there is anything else following mata, it is evaluated inline, and does not enter the mata environment
4. end can be preceded by any number of spaces and followed by anything
5. It doesn't matter where end appears. In a loop, a function, etc. The mata environment will end at that point.
6. mata is not nested, as mata itself is a valid mata command.

from stata_kernel.

Thanks. I'll have to look at the mata syntax more closely.

from stata_kernel.

Comments are just C-style comments, except they can be nested. Rules for locals, etc. are the same. I don't think there is much to look out for within a mata block per se, other than the prompts are different.

from stata_kernel.

I'll come back to Mata once I progress on some other stuff. You're welcome to submit a PR with a Mata lexer if you'd like. All it needs to support is comments and strings.

from stata_kernel.