wcaleniewolny / projectayaya Goto Github PK

Currently the translating the frame to RGB format SUCKS. It takes 60% of the single threaded frame loading. I believe using YUV444P as well as using YUV444 precompiled color lookup table is the only way to eliminate the multi threaded video player.

The multithreaded video player also SUCKS. It is filled with bug prone code, that could crash the JVM, it is written in a horrific way. The illumination of the multithreaded is a critical step in improving this plugin. However to be able to that the single threaded player must be able to handle 4K 60 FPS. The previous frame splitting is a step in the right direction (I also coincided rewriting this entire splitting in C with openMP (#pragma omp) but i do not know if i will actually do that). Currently sws_scale is the performance killer. YUV444P Fixes that.

Moving camera causes a game (flappy_bird) to start to flicker.

Video:
https://user-images.githubusercontent.com/50914789/208758556-e3483a4a-921d-44be-97f5-1cdedeb8d117.mp4

Possible fix:

• Use NMS to set items in item frame

Pseudo code:

val data = SynchedEntityData(entity as Entity)
data.set(EntityDataAccessor(8, EntityDataSerializers.ITEM_STACK), "")
val metadataPacket = ClientboundSetEntityDataPacket((entity as CraftItemFrame).entityId, )

Currently only some of the 4 available rotations work. Some of are created the wrong way.

Currently most implementations of the video player pass prepared video into java. Is is inefficient. Creating a lot packets containing video data does not allow of effective use of zlib compression of said packets. By preparing packets on the native side we could use more efficient zlib compression + create an algorithm to prevent sending redundant data. This could greatly improve client side performance.

When playing on a remote server the screen flickers due to movement packet delay. This can be mitigated by using the /minecraft:effect give PLAYER minecraft:slowness 300 5 true command however I is important that this gets integrated into the plugin itself

Currently the splitter stores the necessary information as a global atomic number. This means that only 1 video can be safely played at the time.

Currently windows users have to setup FFmpeg manualy. This is bad because it requires user to follow a complicated guide. We need to create an easy way to automatically install FFmpeg. My idea:

1. Try to load library normally
2. if it fails check if a "bootstrap" option is enabled in config
3. If so load a rust windows bootstrap lib
4. in this lib we will check if FFmpeg is downloaded if so go to step 6
5. Download and unzip prebuilt FFmpeg
6. Call SetDllDirectoryA to add FFmpeg bin folder to path temporary
7. Use libloading to load the lib
8. If it succeed use get the library symbols and use register_native_methods to register native JNI method
9. If nothing failed return to java. If something failed throw an exception

Currently the CI does not compile due to this. As soon as that this issue gets fixed I will migrate to the fixed version so that CI can work again.