Oliver claims expectedTokens() doesn't work on these examples about antlr4 HOT 5 CLOSED

Is there an ANTLR bug involved? This appears to be a unit test for another application which uses ANTLR but isn't testing the ANTLR library itself. The expectedTokens test doesn't contain any assertions, so it's unclear what the intent actually is.

from antlr4.

Tried to clarify the code a bit, please re-check. The test concerned is expectedTokens().

Unfortunately, the stuff lies in the output to stdout. I spit out what I have parsed so far and then what tokens are to be expected using Parser#getExpectedTokens(). All I see as the expected token is the token I just parsed.

Maybe I just do not grok how this is supposed to work.

from antlr4.

Parser.getExpectedTokens() uses the current state of the current context (parser._ctx.s) determine the set of expected tokens. As soon as the parser changes state, the set of expected tokens changes. Right before a call to match (which produces the call to visitTerminal), the state is set to the source state of the non-epsilon transition matching the symbol. If you call getExpectedTokens() from within visitTerminal of a parser listener, you'll get the set of tokens for the single transition that matched the terminal (which is exactly what you saw).

from antlr4.

Hey, Sam, thanks for being patient with me. I actually mixed up stuff here and what you say makes sense. For I actually want to do: Finding out the set of valid tokens that can follow on a given position inside a buffer. Like for an editor with auto-completion. Any pointers?

from antlr4.

In ANTLRWorks 2 (currently closed source due to licensing details I'm working on), to get this information I needed to extend ParserATNSimulator, CommonToken, ANTLRErrorStrategy, TokenSource, and use a special RuntimeException to bail out of the internals while preserving the interpreter state, as well as controlling the beast with a special controller. In addition, I had to add the -Xforce-atn option to the tool (and use it for any parser where I intend to derive this information).

It's gruesome, but what's possible here is without a doubt the most powerful solution to this problem I've ever seen. Proof of this should be clear in what I was able to accomplish in GoWorks in under 1 month.

from antlr4.