avh4 / binwrap Goto Github PK

View Code? Open in Web Editor NEW

39.0 39.0 17.0 134 KB

Distribute binaries via npm

JavaScript 97.09% Shell 2.11% Nix 0.80%

binwrap's People

Contributors

Stargazers

Watchers

Forkers

rtfeldman michaelglass sch pchambino unknowntraveller harrysarson henricavalcante daviddias benmerckx fleker razzeee lydell piot tamaracha sporto 00mjk zaptic

binwrap's Issues

binwrap-test should only download each unique URL once

currently it will download the same file multiple times if different arch/os identifiers map to the same URL.

Make transitive dependency on unzip-stream and tar optional

Users of this package should only need to have a transitive dependency on tar/unzip-stream if they actually use binaries that are compressed in those formats.

Updating packages

Hello, I found that you guys added on this commit a fs.mkdirSync(path); line of code but it's causing errors like:

error /Users/henricavalcante/contadev/front-end/node_modules/elm-format: Command failed.
Exit code: 1
Command: binwrap-install
Arguments:
Directory: /Users/henricavalcante/contadev/front-end/node_modules/elm-format
Output:
ERR { Error: EEXIST: file already exists, mkdir '/Users/henricavalcante/contadev/front-end/node_modules/elm-format/unpacked_bin'
    at Object.mkdirSync (fs.js:750:3)
    at /Users/henricavalcante/contadev/front-end/node_modules/binwrap/binstall.js:45:8
    at new Promise (<anonymous>)
    at untgz (/Users/henricavalcante/contadev/front-end/node_modules/binwrap/binstall.js:21:10)
    at binstall (/Users/henricavalcante/contadev/front-end/node_modules/binwrap/binstall.js:11:12)
    at install (/Users/henricavalcante/contadev/front-end/node_modules/binwrap/install.js:20:10)
    at Object.install (/Users/henricavalcante/contadev/front-end/node_modules/binwrap/index.js:14:14)
    at Object.<anonymous> (/Users/henricavalcante/contadev/front-end/node_modules/binwrap/bin/binwrap-install:18:9)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:741:12)
    at startup (internal/bootstrap/node.js:285:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:739:3)
  errno: -17,
  syscall: 'mkdir',

so, can I send a pull request wrapping it on :

if (!fs.existsSync(path)) {
   fs.mkdirSync(path);
}

Issue related:
avh4/elm-format#602

Document the node API for getting the path to installed binaries

For example, this feature was used in elm-review: jfmengels/node-elm-review@887569d

binwrap-test should fail if the zip files can't be unzipped

Similar to #1

Critical severity vulnerability - json-schema

This package depends on request 2.88.0
https://github.com/avh4/binwrap/blob/main/package.json#L42

Which depends on form-data 2.3.3
https://github.com/request/request/blob/master/package.json#L36

Which depends on json-schema 0.2.3

json-schema 0.2.3 has a critical vulnerability:
GHSA-896r-f27r-55mw

It would be great if this dependecy could be changed.

Specifying a different binary for Linux platforms without libc

Hi! Elm test currently fails on alpine Linux (a popular Linux distribution for docker images) due to musl being used rather than libc.

Is there a way to distribute a musl linked binary for this platform with binwrap?

I see that the Elm compile and elm-format use binwrap but do not specify a different binary, what approach do they take here?

Related issues:

when the end-user installs on an arch/os that doesn't have packaged binaries, a nice message should be given

elm-platform current uses a message like this:

var errorMessage = "Unfortunately, there are no elm-format " + packageInfo.version + " binaries available for your operating system and architecture.\n\nIf you would like to build elm-format from source, there are instructions at https://github.com/avh4/elm-format#building-from-source\n";

No binaries are available for: darwin-arm64 (Mac OS Monterey with M1 chip)

npm i binwrap

[email protected] install /node_modules/elmi-to-json
binwrap-install

/node_modules/elmi-to-json/node_modules/binwrap/install.js:18
throw new Error("No binaries are available for your platform: " + buildId);
^

Error: No binaries are available for your platform: darwin-arm64
at install (/node_modules/elmi-to-json/node_modules/binwrap/install.js:18:11)
at Object.install (/node_modules/elmi-to-json/node_modules/binwrap/index.js:14:14)
at Object. (/node_modules/elmi-to-json/node_modules/binwrap/bin/binwrap-install:18:9)
at Module._compile (internal/modules/cjs/loader.js:999:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
at Module.load (internal/modules/cjs/loader.js:863:32)
at Function.Module._load (internal/modules/cjs/loader.js:708:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
at internal/main/run_main_module.js:17:47

Binwrap slows execution down

My editor is executing elm-format a lot. It looks like binwrap is slowing down execution. Same effect with npm only. I think we need more comparisons on other systems and if binwrap is the cause we should look into something like the elm "no-deps" installer, which is using npm but looks like it isn't slowing execution down on my machine.

https://github.com/elm/compiler/tree/master/installers/npm

450ms vs 80ms

which -a elm-format                                                                                                                                
/tmp/fnm-shell-3625665/bin/elm-format
/home/as/bin/elm-format


time /tmp/fnm-shell-3625665/bin/elm-format src/Main.elm --yes                                                                                      
Processing file src/Main.elm
0.48user 0.04system 0:00.45elapsed 114%CPU (0avgtext+0avgdata 52876maxresident)k
0inputs+0outputs (0major+9793minor)pagefaults 0swaps

time /home/as/bin/elm-format src/Main.elm --yes                                                                                                    
Processing file src/Main.elm
0.07user 0.00system 0:00.08elapsed 89%CPU (0avgtext+0avgdata 14856maxresident)k
0inputs+0outputs (0major+1608minor)pagefaults 0swaps

if the package author provides a checksum or signature, it should be verified when downloading

Only one of these is necessary (either checksum or signature). Figure out which makes the most sense before implementing this.

Any option to signal that the binary is inside a folder and not the single file of the fetched url?

Going through the code, my understanding was that it could be done by putting in the config:

binaries: [
  "folder-name/myapp-cli"
],

But then I get an error:

Error: ENOENT: no such file or directory, open 'bin/go-ipfs/ipfs'

As it expects that the full folder to be moved to the bin.

Thanks in advance :)

Update: Some notes fro the digging

https://github.com/avh4/binwrap/blob/master/index.js#L8-L10 uses the binary name to check if it is already present on the bin folder

https://github.com/avh4/binwrap/blob/master/install.js#L22 uses the unpacked folder + binary name to find the binary. However, when a binary is one or more levels deep, it fails to find it.

https://github.com/avh4/binwrap/blob/master/bin/binwrap-install#L16 line that needs to be aware if the bin is somewhere else within the directory (or maybe here https://github.com/avh4/binwrap/blob/master/bin/binwrap-install#L16)

Allow configuring where to download binaries from

We would like to cache the binaries downloaded via binwrap via a repo in Artifactory. However, to do that we need to configure where to download the binaries from.

It would be preferable to have something similar to node-sass allowing you to configure the url to fetch the binary from.

If this is something that is interesting I could create a PR with this.

Running from packages that need to be installed globally, it didn't check root permissions.

When I was installing elm-format globally, which depends on binwrap to install, it crashes with permssions error:

output:

/usr/bin/elm-format -> /usr/lib/node_modules/elm-format/bin/elm-format

> [email protected] install /usr/lib/node_modules/elm-format
> binwrap-install

fs.js:653
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: EACCES: permission denied, open 'bin/elm-format'
    at Object.fs.openSync (fs.js:653:18)
    at Object.fs.writeFileSync (fs.js:1300:33)
    at /usr/lib/node_modules/elm-format/node_modules/binwrap/install.js:17:8
    at Array.forEach (<anonymous>)
    at install (/usr/lib/node_modules/elm-format/node_modules/binwrap/install.js:15:19)
    at Object.install (/usr/lib/node_modules/elm-format/node_modules/binwrap/index.js:8:14)
    at Object.<anonymous> (/usr/lib/node_modules/elm-format/node_modules/binwrap/bin/binwrap-install:16:9)
    at Module._compile (module.js:624:30)
    at Object.Module._extensions..js (module.js:635:10)
    at Module.load (module.js:545:32)
    at tryModuleLoad (module.js:508:12)
    at Function.Module._load (module.js:500:3)
    at Function.Module.runMain (module.js:665:10)
    at startup (bootstrap_node.js:201:16)
    at bootstrap_node.js:626:3
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `binwrap-install`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm-install-error.log.txt

Update to tar 4.4.2+ (Tar dependency has a vulnerability)

https://npmjs.com/advisories/803

binwrap-test should print a warning for each URL that uses http instead of https

Error: No binaries are available for your platform: darwin-arm64

is there an alternative to binwrap for arm? Or does anyone have any idea the work involved to add support?

npm ERR! node_modules/binwrap/install.js:18
npm ERR!     throw new Error("No binaries are available for your platform: " + buildId);
npm ERR!     ^
npm ERR!
npm ERR! Error: No binaries are available for your platform: darwin-arm64
npm ERR!     at install (node_modules/binwrap/install.js:18:11)
npm ERR!     at Object.install (node_modules/binwrap/index.js:14:14)
npm ERR!     at Object.<anonymous> (node_modules/binwrap/bin/binwrap-install:18:9)
npm ERR!     at Module._compile (node:internal/modules/cjs/loader:1097:14)
npm ERR!     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1151:10)
npm ERR!     at Module.load (node:internal/modules/cjs/loader:975:32)
npm ERR!     at Function.Module._load (node:internal/modules/cjs/loader:822:12)
npm ERR!     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
npm ERR!     at node:internal/main/run_main_module:17:47

If unzipping fails, the unpacked_bin file should be deleted

work without permissions to create unpacked_bin

From stoeffel/elmi-to-json#20 (comment), the issue seems to be that $NODE_HOME/<package name> is owned by root, but then the install script (or the wrapper script itself if you install with scripts disabled) can't create a folder there (and also couldn't download or modify files there).

The planned fix is that if it doesn't have write access there then it should fallback to unpacking the binaries to ${XDG_CACHE_HOME:-$HOME/.cache}/<package name>-binwrap/

use of `unzip` makes upgrade path to node 10 complicated

got this warning when running npm install

npm WARN deprecated [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN deprecated [email protected]: This module relies on Node.js's internals and will break at some point. Do not use it, and update to [email protected].

and subsequently, got this error

     ChildProcessError: Command failed: (cd test_app && ./node_modules/.bin/binwrap-install darwin x64)
node[3187]: ../src/node_contextify.cc:631:static void node::contextify::ContextifyScript::New(const v8::FunctionCallbackInfo<v8::Value>&): Assertion `args[1]->IsString()' failed.
 1: node::Abort() [node]
 2: 0x87b6c5 [node]
 3: node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo<v8::Value> const&) [node]
 4: v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo*) [node]
 5: 0xad5b9f [node]
 6: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [node]
 7: 0x13ae0b30427d
Aborted (core dumped)
 `(cd test_app && ./node_modules/.bin/binwrap-install darwin x64)` (exited with error code 134)
      at callback (node_modules/child-process-promise/lib/index.js:33:27)
      at ChildProcess.exithandler (child_process.js:289:5)
      at maybeClose (internal/child_process.js:947:16)
      at Socket.stream.socket.on (internal/child_process.js:368:11)
      at Pipe._handle.close [as _onclose] (net.js:598:12)

(see https://travis-ci.org/avh4/binwrap/jobs/374288327)

When I was using binwrap with [email protected], it transitively installed [email protected] which caused nodejs/node#20281

Because this package had installed [email protected] I didn't see the problem locally.

Binwrap install hangs on windows using npm and yarn

Binwrap install hangs when I try to install elm on windows.

I get these errors:
npm ERR! [email protected] install: binwrap-install
npm ERR! Failed at the [email protected] install script.

When I try with yarn it hangs on building fresh packages..
Can anyone help? Thanks!

Make it work with `npm --ignore-scripts`

See notes in elm-lang/elm-platform#205

Support .gz (non-tar) files

This should be done using node's built-in zlib API to prep for allowing users of this package to avoid dependencies on tar/unzip (#26).

Remove dependency on request

Assuming this can be done safely (that there aren't notable features we lose by using nodejs APIs directly), this should be done to remove transitive dependencies.

binwrap-test should print a warning if an unknown arch/os is given a URL

binwrap fails to install behind a corporate firewall (seems to ignore proxy settings)

Hi, when trying to install elmi-to-json (for elm-spa) from behind a corporate firewall I get the following binwrap-install error:

$ npm i elmi-to-json
> [email protected] install /home/trs/test-elm-spa/ri-elm/node_modules/elmi-to-json
> binwrap-install
ERR Error communicating with URL https://github.com/stoeffel/elmi-to-json/releases/download/1.3.0/elmi-to-json-1.3.0-linux.tar.gz Error: tunneling socket could not be established, statusCode=407
npm WARN [email protected] No repository field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `binwrap-install`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

This only happens for binwrap-install. Other packages not using it can be installed without problems, so it is not a problem with my proxy settings. It seems that binwrap-install does not honor the proxy settings.

binwrap-test should fail if the package zips don't all contain all the binaries

binwrap install fails (powershell/windows 10)

Hi! I got the following error installing the latest version of elm-format:

PS C:\Users\harry\node\elm-test> npm i -g elm-format
C:\Users\harry\AppData\Roaming\npm\elm-format -> C:\Users\harry\AppData\Roaming\npm\node_modules\elm-format\bin\elm-format

> [email protected] install C:\Users\harry\AppData\Roaming\npm\node_modules\elm-format
> binwrap-install

internal/buffer.js:53
  throw new ERR_OUT_OF_RANGE(type || 'offset',
  ^

RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 5. Received 9
    at boundsError (internal/buffer.js:53:9)
    at Buffer.readUInt32LE (internal/buffer.js:109:5)
    at UnzipStream._readExtraFields (C:\Users\harry\AppData\Roaming\npm\node_modules\elm-format\node_modules\unzip-stream\lib\unzip-stream.js:443:49)
    at UnzipStream.processDataChunk (C:\Users\harry\AppData\Roaming\npm\node_modules\elm-format\node_modules\unzip-stream\lib\unzip-stream.js:213:30)
    at UnzipStream._parseOrOutput (C:\Users\harry\AppData\Roaming\npm\node_modules\elm-format\node_modules\unzip-stream\lib\unzip-stream.js:651:28)
    at PassThrough.done (C:\Users\harry\AppData\Roaming\npm\node_modules\elm-format\node_modules\unzip-stream\lib\unzip-stream.js:713:18)
    at Object.onceWrapper (events.js:273:13)
    at PassThrough.emit (events.js:182:13)
    at finishMaybe (_stream_writable.js:641:14)
    at afterWrite (_stream_writable.js:481:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `binwrap-install`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\harry\AppData\Roaming\npm-cache\_logs\2019-08-11T11_03_04_421Z-debug.log
PS C:\Users\harry\node\elm-test> node -v
v10.15.0
PS C:\Users\harry\node\elm-test> npm -v
6.9.0