[1.1.5] Cannot finish indexing when using this plugin for EU4 modding about paradox-language-support HOT 32 CLOSED

Hi, I have made some optimization, please install the following plugin package manually, check and profile indexing for eu4 game directory again, and then send me the .jfr file for about 5~10 minutes.

This update should optimize the indexing logic for definitions, but there are many other things to be indexed.

Paradox-Language-Support-1.1.6.zip

from paradox-language-support.

Will try it out in a second, thank you very much!!

from paradox-language-support.

It did not help unfortunately. Looks about the same. One difference I noticed is that RAM usage is lower. It capped out around 4.5 GB even tho I have 8 GB limit in the IDE settings.

Here is the jfr file:
idea64_exe_2023_08_11_084202.zip

from paradox-language-support.

from paradox-language-support.

During this jfr, it takes 60% time (4000s+) to parse script files, 30% time (2000s+) to indexing. at max 20% time (1400s+) can be optimized by optimizing indexing logic.

from paradox-language-support.

So this is just because there are a lot of files in the EU4 game, or at least mostly because of that. (and the files are long)

from paradox-language-support.

Do you think it is possible for the plugin to work for this game?

from paradox-language-support.

Not sure, optimize indexing logic helps nearly nothing, but perhaps scripts files to parse during indexing can be optmized and reduced. Should install eu4 and check manually.

from paradox-language-support.

This finally should be a out of memory problem. Over 60,000,000+ config objects are generated by deep copy, deep merge and inline. Approximately 4G of memory is occupied. There should be no other problems. Indexing process should be finished in 20 minutes If this problem are resolved.

from paradox-language-support.

So you are going to be able to fix this issue?

from paradox-language-support.

Not sure for those generated config objects are hard to cache.

from paradox-language-support.

After lastest commit IDE will freeze during indexing event files.

from paradox-language-support.

Does it go through folders alphabetically?

from paradox-language-support.

Yeah IDEA index files (that should be indexed) in project one by one alphabetically.

from paradox-language-support.

So there is quite a lot more to index.

from paradox-language-support.

I thought it could be fixed by giving more ram to the IDE but it just made it consume all of it regardless of how much you gave it.

from paradox-language-support.

Cwtools shall not index expressions (e.g. a event target) but only almost definitions and localisations. So it's enough fast. But for my plugin, it shall index expressions. It take too much time to get configs from expressions, to infer it's a event target or some other.

from paradox-language-support.

It should only use the memory you give to IDEA. For Stellaris, 4G shall be ok.

from paradox-language-support.

Until this problem is resolved, maybe thoese additional indices can be disabled, to use PLS together with eu4.

from paradox-language-support.

It would be spectacular if you managed to fix this plugin for EU4.

from paradox-language-support.

Should be very hard after many updates.

from paradox-language-support.

Indexing for eu4 game directory is almost finished after 15 minutes (CPU: 8 cores, RAM: 8G)

from paradox-language-support.

Does it work normally after the process is finished? Last time I waited for it to finish, the process would start over every time I had added a new file to the project structure. Does it reindex quickly now (15 minutes is quick)?

from paradox-language-support.

If you'd like to, I could test it on my end, but you would need to send the new plugin version.

from paradox-language-support.

1.1.7 still does not work fine for me.

from paradox-language-support.

Need some more optimization

from paradox-language-support.

The config objects generated during indexing process are too many (3000,000+, 3G+), also the time to consume.
The eu4 game directory is just too crazy. Plugin should parse all script files and localisation files to index definitions, localisations, flags, event targets, and more.
Other files should not be indexed by plugin ,they are just shown by IDEA during scanning.

from paradox-language-support.

I'm sorry but this time I cannot resolve this performance problem. Since there are few remaining features to be implemented, Next several versions should try to do some optimization.

from paradox-language-support.

Sure, I hope someday it will be fixed. Thank you!

from paradox-language-support.

Indexing process for eu4 game directory is once finished over 1 hour under 8 cores CPU & 8 G RAM from my last test.
Note that cwt configs exclude Stellaris is not very perfect, so it may show a lot of errors when open a script file.
You may need to send PR to cwtools-eu4-config by yourself as I don't have so much time and vigor.

from paradox-language-support.

although it was once done, since it's too slow, if you open game directory by IDE twice or make lots of changes, it may still cause a soft IDE freeze (you can operate anything but some IDEA background process is just cannot be finished for a long time)

from paradox-language-support.

Indexing process for eu4 game directory is finished over about 4 minutes at 1.1.12.

This version I merge several indices (that require resolving cwt configs) into one index. So the speed to indexing should be much faster when there are too many files to indexing. but the speed to access index should be slower since there are more index data to visit.

Please still note that cwt configs exclude Stellaris is not very perfect, so it may show a lot of errors when open a script file.

Since loading local cwt configs is not supported by PLS currently, you may still need to send PR to cwtools-eu4-config by yourself once 1.1.12 is released.

from paradox-language-support.