Currently the translating the frame to RGB format SUCKS. It takes 60% of the single threaded frame loading. I believe using YUV444P as well as using YUV444 precompiled color lookup table is the only way to eliminate the multi threaded video player.
The multithreaded video player also SUCKS. It is filled with bug prone code, that could crash the JVM, it is written in a horrific way. The illumination of the multithreaded is a critical step in improving this plugin. However to be able to that the single threaded player must be able to handle 4K 60 FPS. The previous frame splitting is a step in the right direction (I also coincided rewriting this entire splitting in C with openMP (#pragma omp) but i do not know if i will actually do that). Currently sws_scale is the performance killer. YUV444P Fixes that.
Currently most implementations of the video player pass prepared video into java. Is is inefficient. Creating a lot packets containing video data does not allow of effective use of zlib compression of said packets. By preparing packets on the native side we could use more efficient zlib compression + create an algorithm to prevent sending redundant data. This could greatly improve client side performance.
When playing on a remote server the screen flickers due to movement packet delay. This can be mitigated by using the /minecraft:effect give PLAYER minecraft:slowness 300 5 true command however I is important that this gets integrated into the plugin itself
Currently windows users have to setup FFmpeg manualy. This is bad because it requires user to follow a complicated guide. We need to create an easy way to automatically install FFmpeg. My idea:
Try to load library normally
if it fails check if a "bootstrap" option is enabled in config
If so load a rust windows bootstrap lib
in this lib we will check if FFmpeg is downloaded if so go to step 6