Chunkgen large areas leads to Out-Of-Memory exceptions. about chunkgen HOT 8 OPEN

Can you try again with 1.3.0 from the releases?
I've just merged in a bunch of commits from @Baughn which may improve this.

from chunkgen.

I generated the same world and seed twice (same one that crashed every time before) with v1.3.0. Doing a full world wipe and server restart each time for all my testing. No OOM exceptions and believe it or not the gen was nearly 2x faster. 10 minutes to generate the same command in my OP. Before It was taking 10 minutes to get a little past half way then OOM crashing. I reverted to v1.2.3 and the crashing came back at roughly the same place as before so definitely not a fluke.

/chunkgen 0 0 100 100 0

[19:58:23] [Server thread/INFO] [chunkgen/]: Loaded Chunk at 1 1 0
--SNIP--
[20:07:24] [Server thread/INFO] [chunkgen/]: Loaded Chunk at -49 50 0
[20:07:24] [Server thread/INFO] [chunkgen/]: percentage: 1.0

Looking at the start and end chunks I think there is a off by one error some where but it is largely insignificant to the original issue. I'll take the off by one in this case. :D

@Baughn Did good work!

from chunkgen.

I'm pretty sure it's still doing the same thing. I had chunkgen running overnight, and the server's basically frozen.

Look at https://goo.gl/kEeRB7. It's pretty obvious where I started chunk-genning. (Unfortunately I'm not tracking number of loaded chunks in prometheus-integration yet.)

It didn't OOM, and was eventually able to exit, but that may just have been my GC parameters.

The flat bit shortly after the initial spike corresponds to when I was actively playing, so chunkgen was paused. Unfortunately, it appears that simply giving the server time doesn't make it recover.

from chunkgen.

I'm trying to fix the generation of large area. So far, I think i am able to do so by unloading each chunk after it's generated instead of dropping all chunk after the whole thing is done. So far, i was succesfully able to generate 400 by 400 of ungenerated chunk without crashing the server. I'll do more test before proposing a pull request.

from chunkgen.

Just a thought, but it may be more efficient to, say, unload chunks every 256 chunks generated.

I don't actually know this. Time for since tests!

from chunkgen.

In my experience in doing large chunkgen back in ... 164? 1710? Don't recall, the most important thing to do was to switch the garbage collector back to throughput (the normal parallel pause the world collector). While CMS made playing much better, it had horrible performance doing garbage collection as space got filled up, while the parallel collector gave great performance all the way to 98% full.

... I think (from memory) I was doing an r=50 generation.

from chunkgen.

I'm currently trying with 1875 by 1875 for a 30000 x 30000 blocks and it's
going good so far. We can unload all chunks or unload chunks by coordinate.
Generate then unload seems good to me.

On Thu, Apr 28, 2016, 2:18 PM keybounce [email protected] wrote:

In my experience in doing large chunkgen back in ... 164? 1710? Don't
recall, the most important thing to do was to switch the garbage collector
back to throughput (the normal parallel pause the world collector). While
CMS made playing much better, it had horrible performance doing garbage
collection as space got filled up, while the parallel collector gave great
performance all the way to 98% full.

... I think (from memory) I was doing an r=50 generation.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#12 (comment)

from chunkgen.

Ahh. Well, I had a 4 GB machine, about 3.something assigned to Java, and was doing a mystcraft/COG combo to try to determine how many ores COG was actually generating.

from chunkgen.