memory map large files on windows about mmap-io HOT 4 OPEN

@ozra

Update... I tried just mapping the file as 1gb chunks and I run into the same error after gig number 2...
*note, mapping the file works fine with python's numpy.memmap
Codez:

`// const file = path.join( remote.app.getAppPath(), 'pointclouds', 'las', 'test.las' )
const file = path.join( remote.app.getAppPath(), 'test.tif' )
const size = fs.statSync( file ).size
const fd = fs.openSync( file, 'r' )
const pages = []
const bufferSize = 2**30

console.time('test')
for ( let offset = 0; offset < size; offset+=bufferSize ) {
console.log( 'start:', offset )
let pageSize = bufferSize;
if ( pageSize > size ) {
pageSize = size % bufferSize
}
console.log( pageSize )
const protection = window.MMAP.PROT_READ
const privacy = window.MMAP.MAP_PRIVATE
const advise = window.MMAP.MADV_RANDOM
const buffer = window.MMAP.map( pageSize, protection, privacy, fd, offset, advise )
pages.push( buffer )
console.log( 'end:', offset + pageSize )
console.log( '\n\n' )
}
console.timeEnd('test')`

results in:

Uncaught Error: mmap failed, 0

from mmap-io.

Another update... If I allocate 4 1gb buffers on the same file starting at offset 0, there is also no issue. It seems like maybe the issue is the number being passed into either offset or size?

from mmap-io.

@ozra So I actually fixed this by changing the type of the input arguments for size and offset to long longs.... idk if this breaks things on other architectures, but posting my answer here for anyone that runs into a similar problem.

`
constexpr void* hinted_address = nullptr; // Just making things uber-clear...

const size_t    size            = static_cast<size_t>(get_v<long long>(info[0]));

const int       protection      = get_v<int>(info[1]);

const int       flags           = get_v<int>(info[2]);

const int       fd              = get_v<int>(info[3]);

const size_t    offset          = static_cast<size_t>(get_v<long long>(info[4], 0));

const int       advise          = get_v<int>(info[5], 0);

`
*disclaimer, I haven't written C plus plus in like 5 years so be cautious

from mmap-io.

We made https://github.com/ipinfo/mmap-utils with a fix for this.

from mmap-io.