antlr / antlr4-lab Goto Github PK

This issue is specifically with the online lab. Minimum Repeatable example:

grammar Expr;		
prog:	expr EOF ;

WS : ' ' -> skip ; 
ASSIGNMENT : ':' ;
IDENTIFIER : [A-Za-z_]+ ;
DECLARE :
    'func' | 'int' | 'integer' |
    'float' | 'byte' |
    'str' | 'string' |
    'array' | 'list' |
    'dict'
    ;
expr : assignment ;
assignment: 
    IDENTIFIER ASSIGNMENT DECLARE IDENTIFIER
 ;

given the input

print: func value

Gives the errors

1:7 missing DECLARE at 'func'
1:12 extraneous input 'string' expecting

I'm not sure what these error messages are even supposed to mean in this context; DECLARE is clearly not missing, and assignment expects four terms, so I'm not sure how the input is "extraneous". I think there's something wrong with the lab.

ps2pdf gives weird Unrecoverable error: limitcheck in .putdeviceprops msg. Pass on to client by showing SVG with error.

Small point but it would be nice if one could change the width of the left hand window where the grammar is defined.
There's a scroll bar at the bottom but would be good to be able to see longer definitions / comments with reasonable font size.

It looks like I am generating invalid JSON with this grammar and input:

https://stackoverflow.com/questions/74102851/antlr-4-lexing-issues-10-mismatched-input-x-expecting-x

...
"kids": [{
	"ruleidx": 2,
	"alt": 1,
	"kids": [0, 1, "ruleidx": 3, "alt": 1, {  <=======
		"ruleidx": 5,
...

That indicated 1, {... is invalid... Must be some parse tree nesting issue.

Off the top of my head it looks like all threads are going to collide and use the same temporary Image file to create parse trees.

i think the key reset is the issue. must ignore control keys for wiping highlighting etc...

With the grammars-v4 update, I'm noticing most images are cropped. I traced this to the call to ps2pdf. The size of the image is not set for the call. The default of ps2pdf is to use the device characteristics for the server. We need to override that with the size of the PostScript image create.

The WritePS() call does not output the size of the image created. So, the file must be read, and the line that declares the image size used, e.g., %%BoundingBox: 0 0 3857 414. The values there need to be passed to the call to ps2pdf.

ps2pdf -dDEVICEWIDTHPOINTS=3857 -dDEVICEHEIGHTPOINTS=414 temp.ps temp.pdf

On the lab site.
There is nothing on the lexer tab.

The tool console returns BAD JSON RESPONSE when I click the run button. I understand this issue has to do with the server and not my code (let me know if I'm wrong about this!)

grammar program;

program: book EOF;

book: entry+;

entry:
	receta porciones (tiempoprep)? (tiempococ)? calorias ingredientes elaboracion;

receta: LINE_START REC_LBL SEP nombre_receta NL;
nombre_receta: TEXT;
porciones:
	LINE_START POR_LBL SEP cantidad_porciones WS unidad_porciones NL;
cantidad_porciones: NUMBER;
unidad_porciones: TEXT;
tiempoprep: LINE_START TMP_LBL SEP tiempo NL;
tiempococ: LINE_START TMC_LBL SEP tiempo NL;
tiempo: NUMBER WS TEXT;
calorias: LINE_START CAL_LBL SEP calorias_text NL;
calorias_text:
	NUMBER WS TEXT;
ingredientes:
	LINE_START ING_LBL SEP NL ingredient_list NL;

ingredient_list: ingredient_list_item (COM NL ingredient_list_item)*;
ingredient_list_item: TAB amount WS ingredient_name;
amount: NUMBER;
ingredient_name: TEXT;

elaboracion: LINE_START ELA_LBL SEP NL (elaboration_list)+ NL?;
elaboration_list:
	elaboration_list_item (NL elaboration_list_item)*;
elaboration_list_item: INT L_SEP TEXT;

LINE_START: GUION WS;

GUION: '-';
REC_LBL: 'RECETA';
POR_LBL: 'PORCIONES';
TMP_LBL: 'TIEMPO PREPARACION';
TMC_LBL: 'TIEMPO COCCION';
CAL_LBL: 'CALORIAS';
ING_LBL: 'INGREDIENTES';
ELA_LBL: 'ELABORACION';

FEC_LBL: 'FECHA';
HOR_LBL: 'HORA';
DUR_LBL: 'DURACION';
TEM_LBL: 'TEMA';

L_SEP: ')';
COM: ',';
SEP: ':';
WS: ' '+;
NL: '\n';
TAB: '\t';

NUMBER: INT | FLOAT;
INT: [0-9]+;
FLOAT: [0-9]+ '.' [0-9]+;

LISTA: [0-9]+ ')';
TEXT: WORD (WS WORD)*;
WORD: [A-Za-z_]+;

- RECETA: Pudin de pan
- PORCIONES: 3 personas
- TIEMPO PREPARACION: 20 min
- TIEMPO COCCION: 60 min
- CALORIAS: 535 kcal
- INGREDIENTES:
	2.5 tazas leche evaporada,
	2 huevos,
	2 cucharadita extracto de vainilla,
	1 cucharadita canela en polvo,
	0.5 cucharadita jengibre rallado o en polvo,
	0.5 cucharadita clavo dulce en polvo,
	0.25 cucharadita sal,
	0.5 taza azucar morena,
	3 tazas pan viejo en trocitos,
	0.25 taza pasas,
	0.25 taza mantequilla
- ELABORACION:
	1) Buscar una licuadora
	2) Echar todos los liquidos
	3) Echar todos los solidos
	4) Licuar
	5) Beberselo en el vaso de la licuadora
	6) Fin
- RECETA: Pudin de pan
- PORCIONES: 2 jartones
- CALORIAS: 2000 calorias
- INGREDIENTES:
	2 Pizzas Grandes de Pizza Hut
- ELABORACION:
	1) Colocar la Pizza en la mesa
	2) Comersela

Looking at the console, there's some text Bad JSON: followed by a JSON too big to add here. Let me know if you need more information regarding this.

Go to lab.antlr.org and do the following:

• Start from a freshly opened browser. You can do it in different browsers, the result is the same.
• Select the "Java" grammar.
• Press "Run".
• The input is correct, but the parse tree is completely wrong, as though for some some stale input, rather than fetched directly from the control.

Go to http://134.209.209.215/ and repeat the steps. It works correctly. This is my website with my original code before the latest changes.

The input text box looks correct but the parser input is something else!

Lets separate to 2 different repositories UI and server. This was it will be easier to build UI using modern technologies like React or VUE for example. With ability to automate CI\CD.

Server on the other had might be not even used while UI development as if http://lab.antlr.org/antlr/ would work, that would be enough. Or one could clone a server repo and start it locally separately.

Right now I would hesitate to start massive changes in UI, because there are Java files there and I do not know anything about that, and scared to break something.

People are grumbling that the indexer for grammars-v4 are checked into the repo. And, honestly, anything that is generated from the repo should be a Github Action Artifact. I will be changing the indexer to leave grammars-v4.json in the artifact area for the repo at some point. Once I change this, lab.antlr.org should use that. The main problem here is that there is no API to traverse the repo and tell what grammars are and their properties such as start symbol, targets, etc. grammars.json is the best solution for the moment. antlr/grammars-v4#2919

https://stackoverflow.com/questions/31009230/how-to-implement-dragdrop-functionality-in-ace-editor

useful to see how things are tokenized even if you will use a handbuilt parser.

• I'm using 20.04 Ubuntu in WSL2 on Windows. The instructions say to use the "headless" version of Java (sudo apt install openjdk-11-jre-headless), but after doing that, the server would crash with Can't load library: /usr/lib/jvm/java-11-openjdk-amd64/lib/libawt_xawt.so. It worked once I used the "normal" version sudo apt install openjdk-11-jre.
• I also had a problem loading some tool dependencies. I needed to add these as well: for pdfcrop, sudo apt install texlive-extra-utils; for pdf2svg, sudo apt install pdf2svg.

I was using the antlr4-lab and noticed an issue, so I reflexively clicked the 'issues' link and inadvertently lost some edits.

Perhaps the links could open another tab by default, or at least prompt you that you will be lose any changes you've made.

This is cool. I saw this project is built with an ACE editor and a parser in the backend.

I wanted to ask whether you are interested in having editor support (like inline validation, completion etc.) through a language server running in the browser?

There is this open source project called Langium, which can provide a language server derived from a grammar. It is like Xtext for Typescript. It works well with the Monaco editor, which can be used as an LSP client.

I was recently migrating big parts of the Antlr4 grammar to Langium. It still needs some work on it, but when it is ready I could help with creating and integrating the language server and the editor on your site.

WDYT? :)

Hello, I dont see much BigQuery support. I found one grammar file but it's very old, is there any way i can get BigQuery support? Thank you!

It would be helpful to be able to resize the different regions. For example , the example input box is a little small.

The tree rendering does not show non-ASCII chars (\u0080-\uFFFF) correctly.

Simple grammar:

grammar trivial;
options { caseInsensitive = true; }

WORD: ([A-Z_0-9]  | [\u0080-\uFFFF])+ ;

script: WORD* EOF ;

Input text: aouÄÖÜ

Result:

Hierarchy screen is fine.

Create a link so people can pass around not only grammar but also the state of the parse. Similar to the awesome pythontutor.com

I am complete noob in Java. I do not even have a general knowledge. I want to test my contribution locally. I followed the steps and when I run sudo nohup java -cp ~/.m2/repository/org/antlr/antlr4-lab/0.1-SNAPSHOT/antlr4-lab-0.1-SNAPSHOT-complete.jar org.antlr.v4.server.ANTLRHttpServer I have output

nohup: ignoring input and appending output to 'nohup.out'

And terminal is waiting.

I fill like it was successful as url http://localhost/antlr/ shows

HTTP ERROR 404 Not Found
URI:	/antlr/
STATUS:	404
MESSAGE:	Not Found
SERVLET:	org.antlr.v4.server.ANTLRHttpServer$ParseServlet-6b4a4e18
Powered by Jetty:// 11.0.11

And when I test I have this error in console.

Access to XMLHttpRequest at 'http://localhost/antlr/' from origin 'http://localhost:5500' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

My http://localhost:5500 is a live server on a static ащдвукю

How to fix this problem?

Ideally we should dedupe based on content, temporary mitigations include TTL for data in stored in CloudStorage, number/size of items stored (eviction using FIFO), etc.

I got sick of trying to figure out the various libraries and build my own which is simple enough but I can't quite get the > and V arrows to look right when you select tree node. It would be nice to make that look a little better without resorting to another library.

It's hard to know what version lab.antlr.org is actually running. It would be a nice to let me know so I can see what has changed and where it is going.

I assume this would be extremely simple versus a full syntax highlighting mechanism. Here is what the intellij plugin does:

Comments, keywords, rule names, token names, and strings.

try to make lab look like an app.

See comment by @Serhioromano

antlr/antlr4#3717 (reply in thread)

I'm investigating the PHP target, for antlr/antlr-php-runtime#12. For agc, input PINBALL_GAME_BUTTONS_AND_LIGHTS.agc, the http://lab.antlr.org/ website outputs an error "Can't create SVG tree; ps2pdf says: Unrecoverable error: limitcheck in .putdeviceprops".

Some of the grammars and resulting parse trees could be quite large so let's compress to get better response speed.

hard to tell extent of each token.

or make it an img tag referring to url so users can open in another tab?

Hmm... can use window zoom now...maybe don't need.

>cat /etc/os-release 
NAME="Linux Mint"
VERSION="20.3 (Una)"
ID=linuxmint
ID_LIKE=ubuntu
PRETTY_NAME="Linux Mint 20.3"
VERSION_ID="20.3"
HOME_URL="https://www.linuxmint.com/"
SUPPORT_URL="https://forums.linuxmint.com/"
BUG_REPORT_URL="http://linuxmint-troubleshooting-guide.readthedocs.io/en/latest/"
PRIVACY_POLICY_URL="https://www.linuxmint.com/"
VERSION_CODENAME=una
UBUNTU_CODENAME=focal

• commit f8cac5485b47e3b0015e843c4f14da7a3eb6f851 builds into 0.2-SNAPSHOT (not 0.1-SNAPSHOT as per documentation)
• running produces a NoClassDefFoundError:

>java -cp ~/.m2/repository/org/antlr/antlr4-lab/0.2-SNAPSHOT/antlr4-lab-0.2-SNAPSHOT.jar org.antlr.v4.server.ANTLRHttpServer
Error: Unable to initialize main class org.antlr.v4.server.ANTLRHttpServer
Caused by: java.lang.NoClassDefFoundError: org/eclipse/jetty/util/thread/ThreadPool

• the maven dependency tree for jetty libs looks ok to me:

>mvn dependency:tree -Dincludes=org.eclipse.jetty
[...]
[INFO] ------------------------< org.antlr:antlr4-lab >------------------------
[INFO] Building ANTLR 4 Server 0.2-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ antlr4-lab ---
[INFO] org.antlr:antlr4-lab:jar:0.2-SNAPSHOT
[INFO] +- org.eclipse.jetty:jetty-server:jar:11.0.11:compile
[INFO] |  +- org.eclipse.jetty:jetty-http:jar:11.0.11:compile
[INFO] |  |  \- org.eclipse.jetty:jetty-util:jar:11.0.11:compile
[INFO] |  \- org.eclipse.jetty:jetty-io:jar:11.0.11:compile
[INFO] \- org.eclipse.jetty:jetty-webapp:jar:11.0.11:compile
[INFO]    +- org.eclipse.jetty:jetty-servlet:jar:11.0.11:compile
[INFO]    |  \- org.eclipse.jetty:jetty-security:jar:11.0.11:compile
[INFO]    \- org.eclipse.jetty:jetty-xml:jar:11.0.11:compile
[INFO] ------------------------------------------------------------------------

For several grammars, I'm getting "BAD JSON RESPONSE" after trying to run the parser. It looks like this is because the grammar contains actions in some target language.

• It might be best to give a better message saying that the grammar contains target-specific code.
• I think grammars-v4 requires a Java target if there is target-specific code. Is it possible to compile the java code so the grammar could be run. I.e., only allow java code from grammars-v4, and show only a read-only version of those source code?

It would be great to be able to point the tool to the local FS where a grammar is located and work on that grammar