Git Product home page Git Product logo

netcdf4's Introduction

netcdf4-js

Build status NPM Version

NodeJS addon for reading and writing the files in the Network Common Data Form (NetCDF) version <= 4, built upon the C-library for netcdf.

Installation

netcdf4-js is built with nodejs >= 4.x

Install using npm:

$ npm install netcdf4

Prerequisites:

You will need libnetcdf >= 4.x installed.

On Linux/Unix/OSX

  • Install NetCDF4 using your package manager, e.g., on Ubuntu/Debian:
$ sudo apt-get install libnetcdf-dev

or download it here

  • Make sure your system fulfills all the prerequisites of node-gyp

On Windows:

  • Install NetCDF4 from here
  • Make sure to select at least "dependencies", "headers" and "libraries" to install in the NetCDF installation wizard
  • Install the build tools as described here
  • Set the environment variable NETCDF_DIR to your NetCDF installation, e.g.,
C:\> set NETCDF_DIR=C:\Program Files\netCDF 4.6.1

Usage

Open files with

var netcdf4 = require("netcdf4");

var file = new netcdf4.File("test/testrh.nc", "r");

File modes are "r" for "reading", "w" for "writing", "c" for "creation", and "c!" for "overwriting".

Then you can read variables using read or readSlice. The following example reads values at positions 5 to 15:

console.log(file.root.variables['var1'].readSlice(5, 10));

Classes

Properties marked (r/w) can be read and will be written to the file when set.

File

Properties:

  • root : Main Group-object in file

Methods:

  • close() : Close file
  • sync() : Sync (or "flush") file to disk

Group

Properties:

  • id : ID used by C-library
  • name : Name
  • fullname : Full name (path in file)
  • variables : Associative array of variables in group
  • dimensions : Associative array of dimensions in group
  • unlimited : Associative array of unlimited Dimensions in group
  • attribute : Associative array of attributes of group
  • subgroups : Associative array of subgroups of group

Methods:

  • addVariable(name, type, dimensions) : Add a new variable in group. type is one of "byte", "char", "short", "int", "ubyte", "ushort", "uint", "float", "double". dimensions is an array of ids of dimensions for the new variable. Returns new variable.
  • addDimension(name, length) : Add new dimension of length length (can be "unlimited" for unlimited dimension). Returns new dimension.
  • addSubgroup(name) : Add subgroup. Returns new subgroup.
  • addAttribute(name, type, value) : Add and set new attribute. Returns new attribute.

Dimension

Properties:

  • id : ID used by C-library
  • name : Name (r/w)
  • length : Length or currently used length for unlimited dimensions

Attribute

Properties:

  • id : ID used by C-library
  • name : Name (r/w)
  • value : Value (r/w)

Methods:

  • delete() : Delete attribute

Variable

Properties:

  • id : ID used by C-library
  • name : Name (r/w)
  • type : Type of variable
  • attributes : Associative array of attributes
  • dimensions : Array of dimensions used by variable
  • endianness : Endianness: "little", "big", or "native" (r/w)
  • checksummode : Checksum mode: "none", or "fletcher32" (r/w)
  • chunkmode : Chunk mode: "contiguous", or "chunked" (r/w)
  • chunksizes : Array of chunk sizes (one size per dimension) (r/w)
  • fillmode : Boolean switch for fill mode (r/w)
  • fillvalue : Fill value (r/w)
  • compressionshuffle : Boolean switch for shuffle (r/w)
  • compressiondeflate : Boolean switch for compression (r/w)
  • compressionlevel : Compression level (1-9) (r/w)

Methods:

  • read(pos....) : Reads and returns a single value at positions given as for write.
  • readSlice(pos, size....) : Reads and returns an array of values (cf. "Specify a Hyperslab") at positions and sizes given for each dimension, readSlice(pos1, size1, pos2, size2, ...) e.g. readSlice(2, 3, 4, 2) gives an array of the values at position 2 for 3 steps along the first dimension and position 4 for 2 steps along the second one.
  • readStridedSlice(pos, size, stride....) : Similar to readSlice(), but it adds a stride (interval between indices) parameter to each dimension. If stride is 4, the function will take 1 value, discard 3, take 1 again, etc. So for instance readStridedSlice(2, 3, 2, 4, 2, 1) gives an array of the values at position 2 for 3 steps with stride 2 (i.e. every other value) along the first dimension and position 4 for 2 steps with stride 1 (i.e. with no dropping) along the second dimension.
  • write(pos..., value) : Write value at positions given, e.g. write(2, 3, "a") writes "a" at position 2 along the first dimension and position 3 along the second one.
  • writeSlice(pos, size..., valuearray) : Write values in valuearray (must be a typed array) at positions and sizes given for each dimension, e.g. writeSlice(2, 3, 4, 2, new Int32Array([0, 1, 2, 3, 4, 5])) writes the array at position 2 for 3 steps along the first dimension and position 4 for 2 step along the second one (cf. "Specify a Hyperslab").
  • writeStridedSlice(pos, size, stride..., valuearray) : Similar to writeSlice(), but it adds a stride parameter to each dimension. So for instance writeStridedSlice(2, 3, 2, 4, 2, 1), new Int32Array([0, 1, 2, 3, 4, 5]) writes the array at position 2 for 3 steps with stride 2 (i.e. every other value) along the first dimension and position 4 for 2 steps with stride 1 (i.e. with no dropping) along the second dimension.
  • addAttribute(name, type, value) : Adds and sets new attribute. Returns new attribute.

netcdf4's People

Contributors

adam-nielsen avatar guigrpa avatar parro-it avatar swillner avatar waldner avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

netcdf4's Issues

Explore the file

Hi, if I do console.log(file.root.variables); I got a TypeError: Operation not permitted. Is that the spected behavior? and if so how can I explore and unknown formated file? Thanks

NodeJS added to netCDF documentation.

Not an issue, per se. Based on the email you sent, we've added netcdf4-js to our list of software and have highlighted it in the release notes for the upcoming 4.4.0 release. You can view this at https://github.com/Unidata/netcdf-c/blob/master/RELEASE_NOTES.md for the time being, and it will go live on the Unidata netcdf-c website (http://www.unidata.ucar.edu/software/netcdf/docs) when the upcoming release goes live.

Thanks for telling us about this, it looks great and we're very excited to have an interface for NodeJS :).

Operation not permitted errors

Hi,

first, thanks for this library. I have found it very useful. I have been processing some large NetCDF files and all has been going really well. However, I've now run into a NetCDF file which throws "Operation not permitted" errors, but I cannot work out why. The same operation on other NetCDF files works just fine. I am only trying to read the files, not write to them.

One specific example I've traced to trying to read the 'endianness' attribute from a variable object. The attribute is there (it is listed if I do a for (let k of Object.keys(nc.root.variables[latitude])) ... but if I try to access the value, I get an "Operation not permitted" error. If I ry to dump the objects to console.dir() I get the same error. All of theses operations work fine with other NetCDF files I'm processing, so it is something different with this version of NetCDF files, but I am not sure what. Any clue or help you can provide would be appreciated. Have you seen this before or are there some known limitations etc?

The file is 14.8Mb in size, so I cannot upload it, but happy to share it if that would help (just need somewhere I an send/upload it).

thanks,
Tim

err with node-gyp rebuild

Hey there! Not sure if it's an issue with the package, but I get an err with node-gyp rebuild where it says it can't find the file netcdf.h. I've got the proper netcdf-c version installed on my computer and can access it using the command line (originally had thought it might be that it couldn't locate the netcdf program).

I looked around online and it seems like it's come up in other npm packages; tried the usual suggestions of updating node, deleting node_modules and re-installing, etc. etc.

Thought I'd flag in case you might be able to assist. Here's the actual error:

node-gyp rebuild

CXX(target) Release/obj.target/netcdf4/src/Group.o
In file included from ../src/Group.cpp:3:
../src/Variable.h:4:10: fatal error: 'netcdf.h' file not found
#include <netcdf.h>
^
1 error generated.
make: *** [Release/obj.target/netcdf4/src/Group.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:276:23)
gyp ERR! stack at emitTwo (events.js:106:13)
gyp ERR! stack at ChildProcess.emit (events.js:191:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:204:12)
gyp ERR! System Darwin 16.1.0
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/AdamBehrens/coding/as-clear-sky/node_modules/netcdf4
gyp ERR! node -v v6.0.0
gyp ERR! node-gyp -v v3.3.1
gyp ERR! not ok

Air-quality slice level variable and re-write the file

Hi,

It's possible to re-write a netcdf file with your library ? I have a variable named level (0, 50, 250, 500, 1000, 2000, 3000, 5000 m) and I'd like to get the level 0 and re-write the .nc file with just the level 0.

I want to know if your library can write .nc file or we can just read .nc files.

unable to install it

I have been strugelling to install this moudle for more that two days and I couldn't. Could you please let me know which step I'm missing?

Here is the error I get on nodejs:

D:\master\nodeTut>npm install netcdf4

[email protected] install D:\master\nodeTut\node_modules\netcdf4
node-gyp rebuild

D:\master\nodeTut\node_modules\netcdf4>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\....\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
gyp ERR! configure error
gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
gyp ERR! stack at PythonFinder.failNoPython (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\configure.js:483:19)
gyp ERR! stack at PythonFinder. (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\configure.js:508:16)
gyp ERR! stack at C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\polyfills.js:284:29
gyp ERR! stack at FSReqWrap.oncomplete (fs.js:152:21)
gyp ERR! System Windows_NT 10.0.16299
gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" "rebuild"
gyp ERR! cwd D:\master\nodeTut\node_modules\netcdf4
gyp ERR! node -v v8.11.2
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
npm WARN enoent ENOENT: no such file or directory, open 'D:\master\nodeTut\package.json'
npm WARN nodeTut No description
npm WARN nodeTut No repository field.
npm WARN nodeTut No README data
npm WARN nodeTut No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\Ehsan\AppData\Roaming\npm-cache_logs\2018-06-11T22_02_31_364Z-debug.log

Undefined symbol: H5T_NATIVE_SHORT_g

If you have compiled the netcdf4 and hdf5 libraries statically (in my case, because I want to deploy to AWS Lambda), then the module is built but it cannot be loaded due to missing symbols:

Error: node_modules/netcdf4/build/Release/netcdf4.node: undefined symbol: H5T_NATIVE_SHORT_g
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:718:18)
    at Module.load (internal/modules/cjs/loader.js:599:32)

It turns out this is due to a missing library dependency in binding.gyp, and the solution is to specify the dependent library:

{
  "targets": [
    {
      "libraries": [
        "-lnetcdf",
        "-lhdf5"       # Must specify
      ],
...

I haven't been able to test this with shared libraries (as my environment is only building static ones) but if it doesn't break the shared library build, could this change be included for the benefit of those using static libraries?

compile errors running npm install ( [email protected], [email protected])

macOS 10.15.3

netcdf installed with homebrew:

$ brew info netcdf
netcdf: stable 4.7.3 (bottled)
Libraries and data formats for array-oriented scientific data
https://www.unidata.ucar.edu/software/netcdf
/usr/local/Cellar/netcdf/4.7.3_1 (89 files, 5.8MB) *
  Poured from bottle on 2020-02-15 at 15:56:20
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/netcdf.rb
==> Dependencies

Summary report from installing:

35 warnings and 5 errors generated.
make: *** [Release/obj.target/netcdf4/src/Group.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/stephen/.nvm/versions/node/v12.16.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (events.js:321:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)
gyp ERR! System Darwin 19.3.0
gyp ERR! command "/Users/stephen/.nvm/versions/node/v12.16.0/bin/node" "/Users/stephen/.nvm/versions/node/v12.16.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/stephen/dev/node-cdf/node_modules/netcdf4
gyp ERR! node -v v12.16.0
gyp ERR! node-gyp -v v5.0.5
gyp ERR! not ok 
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

Just the five errors.

./src/Group.cpp:36:49: error: too few arguments to function call, single argument 'context' was not specified
    constructor.Reset(isolate, tpl->GetFunction());
                               ~~~~~~~~~~~~~~~~ ^
../src/Group.cpp:118:36: error: too few arguments to function call, single argument 'context' was not specified
        len = args[1]->Uint32Value();
              ~~~~~~~~~~~~~~~~~~~~ ^
/Users/stephen/Library/Caches/node-gyp/12.16.0/include/node/v8.h:2707:3: note: 'Uint32Value' declared here
../src/Group.cpp:161:44: error: no matching member function for call to 'ToObject'
    v8::Local<v8::Object> array = args[2]->ToObject();
                                  ~~~~~~~~~^~~~~~~~
/Users/stephen/Library/Caches/node-gyp/12.16.0/include/node/v8.h:2672:44: note: candidate function not viable: requires single argument
../src/Group.cpp:162:88: error: too few arguments to function call, single argument 'context' was not specified
    size_t ndims = array->Get(v8::String::NewFromUtf8(isolate, "length"))->Uint32Value();
../src/Group.cpp:165:47: error: too few arguments to function call, single argument 'context' was not specified
        dimids[i] = array->Get(i)->Int32Value();

The debug.log produced:

0 info it worked if it ends with ok
1 verbose cli [
1 verbose cli   '/Users/stephen/.nvm/versions/node/v12.16.0/bin/node',
1 verbose cli   '/Users/stephen/.nvm/versions/node/v12.16.0/bin/npm',
1 verbose cli   'install',
1 verbose cli   'netcdf4'
1 verbose cli ]
2 info using [email protected]
3 info using [email protected]
4 verbose npm-session ed44b6729031d234
5 silly install loadCurrentTree
6 silly install readLocalPackageData
7 http fetch GET 200 https://registry.npmjs.org/netcdf4 17ms (from cache)
8 silly pacote tag manifest for netcdf4@latest fetched in 28ms
9 timing stage:loadCurrentTree Completed in 65ms
10 silly install loadIdealTree
11 silly install cloneCurrentTreeToIdealTree
12 timing stage:loadIdealTree:cloneCurrentTree Completed in 0ms
13 silly install loadShrinkwrap
14 timing stage:loadIdealTree:loadShrinkwrap Completed in 7ms
15 silly install loadAllDepsIntoIdealTree
16 silly resolveWithNewModule [email protected] checking installable status
17 timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 7ms
18 timing stage:loadIdealTree Completed in 15ms
19 silly currentTree [email protected]
19 silly currentTree ├── [email protected]
19 silly currentTree ├── [email protected]
19 silly currentTree ├── [email protected]
19 silly currentTree └── [email protected]
20 silly idealTree [email protected]
20 silly idealTree ├── [email protected]
20 silly idealTree ├── [email protected]
20 silly idealTree ├── [email protected]
20 silly idealTree ├── [email protected]
20 silly idealTree └── [email protected]
21 silly install generateActionsToTake
22 timing stage:generateActionsToTake Completed in 3ms
23 silly diffTrees action count 1
24 silly diffTrees add [email protected]
25 silly decomposeActions action count 8
26 silly decomposeActions fetch [email protected]
27 silly decomposeActions extract [email protected]
28 silly decomposeActions preinstall [email protected]
29 silly decomposeActions build [email protected]
30 silly decomposeActions install [email protected]
31 silly decomposeActions postinstall [email protected]
32 silly decomposeActions finalize [email protected]
33 silly decomposeActions refresh-package-json [email protected]
34 silly install executeActions
35 silly doSerial global-install 8
36 verbose correctMkdir /Users/stephen/.npm/_locks correctMkdir not in flight; initializing
37 verbose lock using /Users/stephen/.npm/_locks/staging-db43c5e244f989ba.lock for /Users/stephen/dev/node-cdf/node_modules/.staging
38 silly doParallel extract 1
39 silly extract [email protected]
40 silly tarball trying netcdf4@latest by hash: sha512-XU2uEKh7N6jzJUSqJGEm0C6f92ea3WrRXnGa4bcRh2jq9lKD2H1OCGxjjzKPjD31z15qL5S9y342gfeui/Uu3A==
41 silly extract netcdf4@latest extracted to /Users/stephen/dev/node-cdf/node_modules/.staging/netcdf4-73e7ef9a (205ms)
42 timing action:extract Completed in 208ms
43 silly doReverseSerial unbuild 8
44 silly doSerial remove 8
45 silly doSerial move 8
46 silly doSerial finalize 8
47 silly finalize /Users/stephen/dev/node-cdf/node_modules/netcdf4
48 timing action:finalize Completed in 2ms
49 silly doParallel refresh-package-json 1
50 silly refresh-package-json /Users/stephen/dev/node-cdf/node_modules/netcdf4
51 timing action:refresh-package-json Completed in 6ms
52 silly doParallel preinstall 1
53 silly preinstall [email protected]
54 info lifecycle [email protected]~preinstall: [email protected]
55 timing action:preinstall Completed in 1ms
56 silly doSerial build 8
57 silly build [email protected]
58 info linkStuff [email protected]
59 silly linkStuff [email protected] has /Users/stephen/dev/node-cdf/node_modules as its parent node_modules
60 timing action:build Completed in 1ms
61 silly doSerial global-link 8
62 silly doParallel update-linked 0
63 silly doSerial install 8
64 silly install [email protected]
65 info lifecycle [email protected]~install: [email protected]
66 verbose lifecycle [email protected]~install: unsafe-perm in lifecycle true
67 verbose lifecycle [email protected]~install: PATH: /Users/stephen/.nvm/versions/node/v12.16.0/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/stephen/dev/node-cdf/node_modules/netcdf4/node_modules/.bin:/Users/stephen/dev/node-cdf/node_modules/.bin:/Users/stephen/.nvm/versions/node/v12.16.0/bin:/Users/stephen/.rbenv/shims:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:/opt/X11/bin:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Xamarin Workbooks.app/Contents/SharedSupport/path-bin:/Users/stephen/dev/bin
68 verbose lifecycle [email protected]~install: CWD: /Users/stephen/dev/node-cdf/node_modules/netcdf4
69 silly lifecycle [email protected]~install: Args: [ '-c', 'node-gyp rebuild' ]
70 timing audit submit Completed in 667ms
71 http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/audits/quick 667ms
72 timing audit body Completed in 1ms
73 silly lifecycle [email protected]~install: Returned: code: 1  signal: null
74 info lifecycle [email protected]~install: Failed to exec install script
75 timing action:install Completed in 2281ms
76 verbose unlock done using /Users/stephen/.npm/_locks/staging-db43c5e244f989ba.lock for /Users/stephen/dev/node-cdf/node_modules/.staging
77 timing stage:rollbackFailedOptional Completed in 70ms
78 timing stage:runTopLevelLifecycles Completed in 2696ms
79 silly saveTree [email protected]
79 silly saveTree ├─┬ [email protected]
79 silly saveTree │ └─┬ [email protected]
79 silly saveTree │   └─┬ [email protected]
79 silly saveTree │     └── [email protected]
79 silly saveTree └── [email protected]
80 warn [email protected] No description
81 warn [email protected] No repository field.
82 verbose stack Error: [email protected] install: `node-gyp rebuild`
82 verbose stack Exit status 1
82 verbose stack     at EventEmitter.<anonymous> (/Users/stephen/.nvm/versions/node/v12.16.0/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
82 verbose stack     at EventEmitter.emit (events.js:321:20)
82 verbose stack     at ChildProcess.<anonymous> (/Users/stephen/.nvm/versions/node/v12.16.0/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
82 verbose stack     at ChildProcess.emit (events.js:321:20)
82 verbose stack     at maybeClose (internal/child_process.js:1021:16)
82 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)
83 verbose pkgid [email protected]
84 verbose cwd /Users/stephen/dev/node-cdf
85 verbose Darwin 19.3.0
86 verbose argv "/Users/stephen/.nvm/versions/node/v12.16.0/bin/node" "/Users/stephen/.nvm/versions/node/v12.16.0/bin/npm" "install" "netcdf4"
87 verbose node v12.16.0
88 verbose npm  v6.13.4
89 error code ELIFECYCLE
90 error errno 1
91 error [email protected] install: `node-gyp rebuild`
91 error Exit status 1
92 error Failed at the [email protected] install script.
92 error This is probably not a problem with npm. There is likely additional logging output above.
93 verbose exit [ 1, true ]

Publish new version?

IThe current npmjs.com version does not build under node v12. However, it appears the current git HEAD does. Are there plans to publish a new version on npmjs.com? Version 12 has been LTS for a time now and not having a published version is causing some problems.

How to use readSlice in 4 dimensional array

Hi, I have to read in my 4 dimensional array with readSlice(). When I'm using read() it's work :

// Time, Level, Latitude, Longitude
file.root.variables['pm10_conc'].read(0, 0, 0, 0)

This will output well my first cell.

Time max array index = 24
Level max array index = 0
Latitude max array index = 400
Longitude max array index = 700

But I want to read this :

Latitude -> from 69.95 to 69.85
Longitude -> from 335.05 to 335.25

As you can see in my image here I should get this data

69.95, 335.05 = 0.122270
69.95, 335.15 = 0.122270
69.95, 335.25 = 0.122015

69.85, 335.05 = 0.133071
69.85, 335.15 = 0.122697
69.85, 335.25 = 0.120764

I did this but this will returns me all my data.

file.root.variables['pm10_conc'].readSlice(0, 24, 0, 1, 0, 400, 0, 700)

Normal because max lat = 400 and max lng = 700. Could you tell me how can I use readSlice() in my case ? Thanks.

Dead link to source

I am interested in building this on a Windows machine.

The link provided to source is dead.

expose API surface for loading netcdf directly from a Buffer

I currently have an application that downloads NetCDF data from S3 and then uses this library to read that data. Unfortunately, with the API surface that's currently exposed, I'm forced to save the data I have to disk first and then read it back into memory.

Obviously, this isn't the ideal workflow, since I already have the object in memory when downloading from S3. So it would be preferable if I could simply pass my buffer to a netcdf method (e.g. new netcdf4.fromBuffer(data)) and skip the disk entirely.

Based on the C-library's documentation, this is already implemented in NetCDF4 (source docs) and it should just be a question of adding new bindings to this library.

@parro-it any chance of implementing this in the future? I'm not very familiar with this repo's code, but I could also try to put a PR together (definitely the slower option) if you'd be willing to review & then make a new release after it was merged.

Error while installing ( cannot convert ‘v8::MaybeLocal<v8::String>’ to ‘v8::Local<v8::String>’)

These are the first few lines when trying to install this library with node 18.15.0

npm ERR! code 1                                                                                                                  
npm ERR! path /home/evert/src/city/server/node_modules/netcdf4                                                                   
npm ERR! command failed                                                                                                          
npm ERR! command sh -c node-gyp rebuild                                                                                          
npm ERR! make: Entering directory '/home/evert/src/city/server/node_modules/netcdf4/build'                                       
npm ERR!   CXX(target) Release/obj.target/netcdf4/src/Group.o                                                                    
npm ERR! make: Leaving directory '/home/evert/src/city/server/node_modules/netcdf4/build'                                        
npm ERR! gyp info it worked if it ends with ok                                                                                   
npm ERR! gyp info using [email protected]                                                                                           
npm ERR! gyp info using [email protected] | linux | x64                                                                               
npm ERR! gyp info find Python using Python version 3.11.2 found at "/usr/bin/python3"                                            
npm ERR! gyp info spawn /usr/bin/python3                                                                                         
npm ERR! gyp info spawn args [                                                                                                   
npm ERR! gyp info spawn args   '/usr/lib/node_modules_18/npm/node_modules/node-gyp/gyp/gyp_main.py',                             
npm ERR! gyp info spawn args   'binding.gyp',                                                                                    
npm ERR! gyp info spawn args   '-f',                                                                                             
npm ERR! gyp info spawn args   'make',                                                                                           
npm ERR! gyp info spawn args   '-I',                                                                                             
npm ERR! gyp info spawn args   '/home/evert/src/city/server/node_modules/netcdf4/build/config.gypi',                             
npm ERR! gyp info spawn args   '-I',                                                                                             
npm ERR! gyp info spawn args   '/usr/lib/node_modules_18/npm/node_modules/node-gyp/addon.gypi',                                  
npm ERR! gyp info spawn args   '-I',                                                                                             
npm ERR! gyp info spawn args   '/home/evert/.cache/node-gyp/18.15.0/include/node/common.gypi',                                   
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',                                                                       
npm ERR! gyp info spawn args   '-Dvisibility=default',                                                                           
npm ERR! gyp info spawn args   '-Dnode_root_dir=/home/evert/.cache/node-gyp/18.15.0',                                            
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/usr/lib/node_modules_18/npm/node_modules/node-gyp',                              
npm ERR! gyp info spawn args   '-Dnode_lib_file=/home/evert/.cache/node-gyp/18.15.0/<(target_arch)/node.lib',                    
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/home/evert/src/city/server/node_modules/netcdf4',                             
npm ERR! gyp info spawn args   '-Dnode_engine=v8',                                                                               
npm ERR! gyp info spawn args   '--depth=.',                                                                                      
npm ERR! gyp info spawn args   '--no-parallel',                                                                                  
npm ERR! gyp info spawn args   '--generator-output',                                                                             
npm ERR! gyp info spawn args   'build',                                                                                          
npm ERR! gyp info spawn args   '-Goutput_dir=.'                                                                                  
npm ERR! gyp info spawn args ]                                                                                                   
npm ERR! gyp info spawn make                                                                                                     
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]                                                              
npm ERR! In file included from ../src/Group.cpp:6:                                                                               
npm ERR! ../src/netcdf4js.h: In function ‘void netcdf4js::throw_netcdf_error(v8::Isolate*, int)’:                                
npm ERR! ../src/netcdf4js.h:11:77: error: cannot convert ‘v8::MaybeLocal<v8::String>’ to ‘v8::Local<v8::String>’                 
npm ERR!    11 |     isolate->ThrowException(v8::Exception::TypeError(v8::String::NewFromUtf8(isolate, nc_strerror(retval))));   
npm ERR!       |                                                      ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~      
npm ERR!       |                                                                             |                                   
npm ERR!       |                                                                             v8::MaybeLocal<v8::String>          
npm ERR! In file included from /home/evert/.cache/node-gyp/18.15.0/include/node/v8.h:30,                                         
npm ERR!                  from /home/evert/.cache/node-gyp/18.15.0/include/node/node.h:73,                                                                                                                                                                    

Windows Support?

Hi, are there any plans for supporting Windows?
Node is compiled with VS, thus requires modules to be built with that as well.
The problem is that VS does not compile C99, ie. variable size arrays like here:

Attribute.cpp(57): error C2131: expression did not evaluate to a constant

Get variable attributes

Hello,

First, thank you for your package, it's very useful !
I have an issue with variables attributes. it seems there are parameters (offset, scale_factor) that need to be applied on the data to get the expected value.
I thought it was stored on attributes (file.root.variables.[attributeName].attributes ) but I keep getting an "Operation not permitted".

Are they stored somewhere else or am I doing something wrong ?

Thank you

Lots of warnings during build

During the build, lots of warnings are emitted. See the attached file for the actual output. This is using node 11.12.0.
warnings.txt

The resulting module seems to be working fine, but I thought I'd just report this.

Build Failure on Install

Hello, I was trying to install the node package this morning, however, on install, the build fails due to being unable to include 'netcdf.h' I have NetCDF installed, so I don't know why it's having a hard time.

Cannot utilize in Windows 10

  • My NETCDF_DIR variable is set
  • It seems to compile properly (though it does warn a few variants of this during compilation):
xxx\node_modules\netcdf4\src\variable.cpp(961): warning C4996: 'v8::Value::NumberValue': was declared deprecatedc:\users\xxx\.node-gyp\10.15.0\include\node\v8.h(2478): note: see declaration of 'v8::Value::Int32Value' [xxx\node_modules\netcdf4\build\netcdf4.vcxproj])
  • When I run the require statement, I get:
The specified module could not be found. \\?\xxx\node_modules\netcdf4\build\Release\netcdf4.node Error: The specified module could not be found.
\\?\xxx\node_modules\netcdf4\build\Release\netcdf4.node
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:718:18)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at xxx\routes\api.js:6:19
  • The netcdf4.node file is present in the directory, and if I copy it and load it from another folder I get the same error.
  • I'm using Webpack 4, Node 10, Windows 10 Pro, and have used node-gyp successfully on other projects on this machine.

Any help you can provide would be hugely appreciated!

Install problem in ionic 4 app

Hi,
I'm trying to install netcdf4 in a ionic app giving:
npm install netcdf4 --save

When I compile, I get this error:
ERROR in ./node_modules/netcdf4/build/Release/netcdf4.node 1:0 Module parse failed: Unexpected character '' (1:0) You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
What can I do?
Thanks!

Node 10 support?

I recently tried installing this package with Node 10 and it failed with errors about gyp incompatibility. Before I dig into this further, I wanted to know if there are plans to make it compatible with node 10+? It is many many years since I did any C/C++ and I've not built any node modules which interface with C++, but happy to try and have a go, but thought I'd check to see if this has already been started or if it is a known issue with an easy fix etc.

thanks,

Tim

addAttribute() fails

addAttribute() apparently fails when creating a file from scratch:

const file = new netcdf4.File('./small.nc', 'c!');
const lonDimension = file.root.addDimension('lon', 4);
const latDimension = file.root.addDimension('lat', 4);
const timeDimension = file.root.addDimension('time', 'unlimited');
const lon = file.root.addVariable('lon', 'double', [lonDimension.id]);
lon.writeSlice(0, 4, new Float64Array([1, 2, 3, 4]));
lon.addAttribute('standard_name', 'longitude');

The last command fails with:

lon.addAttribute('standard_name', 'longitude');
    ^

TypeError: Variable type not supported yet

It also fails when modifying an existing file:

const filePath = 'test.nc';
const file = new netcdf4.File(filePath, 'w');
file.root.variables.lon.addAttribute('xxxx', 'aaaa');

New maintainer and plan

Hi, I'm the new maintainer of this repository.
Sven Willner is transferring repo and npm ownership to me.

I'm working in the meteorological field for almost an year, and I'll
use this module at work, so I think I could commit a fair number of hours to
work on this.

Within some days, I release a first minor version on npm,
where I'll:

  • merge PR #22 to allow to use Node 12
  • upgrade dependencies in order to fix security issues
  • port CI tests to GitHub actions

On a more long timeframe, I think I'll to port the code to NAPI, in order to leverage
it's ABI compatibility. This will avoid the necessity to change the code for every new
Node version that will be released, and will open the way to publish pre-compiled version
on NPM.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.