Question .core files vs .stream files about decima HOT 7 CLOSED

Hi @nsacerdote,

Thank you for your interest!

Your understanding of the difference is mostly correct.

The application gathers human-readable file paths from the file called prefetch/fullgame.prefetch.core (you can look it up through the search dialog). This file contains a list of files that are "prioritized" by the game to be loaded first during startup, I believe.

Unfortunately, this file does not contain streaming files or even some core files, including the one mentioned before. So the only solution that proved to be working was to scan all archives and generate a name table manually. The only issue with this approach is the performance. It takes several minutes to complete, even on high-end hardware, making it not vial to be used during startup. A simple solution would be to dump them into a separate file and ship them with other metadata files, but at the time, it sounded wild because of its size (60MB, yet 3MB compressed).

Since now we have a reason to do so, we might take this approach.

from decima.

Cool! I made some notes on your PR.

Core files contain references to streams, so by scanning every core file, including every field of every contained class, it is possible to generate a complete list of names.

from decima.

Thank you for the answer!

One thing that I don't get is why the "Find files" tool does find the stream files? (and only if you search for the exact full path with .stream as extension)

Seems that somehow the program already knows the path of the "unnamed" category files.

Also, a little offtopic, what's your position on contributions? I might want to try and create a PR at some point if I see the chance to contribute.

from decima.

The Find Files dialog doesn't find stream files for the same reason why they aren't shown in the navigator under human-readable names. But It still manages to find such files by exact match if their path hashes match. You can see the code responsible for this part here and here where it builds an "index" with all files it later uses for searching.

The <unnamed> folder contains files with no entries in the prefetch file, meaning that we don't know their string representation. The absence of readable paths wasn't the reason not to include such files at all, so they now reside under that special folder.

We are widely open to contributions! You can get technical help on our Discord server. The link is in the repo's description.

from decima.

I created a proof of concept of a simple approach to have the stream files in the navigator and in the search, seems to work, but not sure if this breaks something 😄

I didn't understand exactly how it could be possible to generate the name table manually as you suggested on your first comment. I guess that the actual stream files have their path embedded inside their binary data?

If that's the case then what you said about dumping and embeddinf them would be probably better, as it will find all stream files instead of only the ones that we can guess 😄

from decima.

Another question, and sorry for asking that much 😄.

I see that you also participated in the previous project "ProjectDecima". How was this issue solved there?

I don't see any big metadata file provided in that project. And it shows the stream files in the tree:

from decima.

Ah, don't worry. I'm glad to be helpful!

It wasn't solved in any way. It has the same issue, at least the original version :)

I believe this is a fork judging by the application's title. I don't remember adding support for streams in particular, so maybe it was done later in the same way you did later.

from decima.