Speed up normalisation code about faker HOT 8 CLOSED

Before we jump into the caching rabit hole, we should try async IO first.

from faker.

There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors.

☝️

from faker.

For reference:

• At the moment, pnpm run generate:locales takes about 6s on my machine, a M1 Macbook Air (with only normalization for food module)
• if i comment out the other modules from definitionsToSkip this increases to 29s on the first run and 19s for subsequent runs.

So maybe the ~13 seconds slowdown is tolerable for now if it will make the code too complex to fix it.

from faker.

Could we do some form of caching (like the eslint cache) so it doesn't bother trying to re-normalize unchanged files that have already been normalised?

from faker.

Could we do some form of caching (like the eslint cache) so it doesn't bother trying to re-normalize unchanged files that have already been normalised?

This was also my idea 🤔
Before implementing something on our own, we should look if we can bring a caching dev-dependency for that 👀

Edit: a really quick search brought e.g. https://www.npmjs.com/package/file-entry-cache to the table 👀

from faker.

Before we into the caching rabit hole, we should try async IO first.

Oh! That might be a really good idea 👍

from faker.

For me the baseline is:

• next-limited: 12s
• next-all: 34s

from faker.

I created a first refactor for async io:

• #2826

from faker.