npm / git Goto Github PK
View Code? Open in Web Editor NEWa util for spawning git from npm CLI contexts
License: ISC License
a util for spawning git from npm CLI contexts
License: ISC License
npm install fails to clone branch, commit hash or tag when the repo does not have a HEAD reference.
Error Message:
`npm ERR! Cannot read properties of undefined (reading 'sha')
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\user\AppData\Local\npm-cache_logs\2023-01-26T12_33_01_788Z-debug-0.log`
It should still be able to clone the specified reference.
npm install ssh+git://git@private/private_repo#reference
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\user\AppData\Local\npm-cache_logs\2023-01-26T12_33_01_788Z-debug-0.log`
I originally ran into a very hard to trace issue that resulted from using a newer npm
. After much head scratching, I figured out that the problem is that npm
is setting $GIT_SSH_COMMAND
in a way that kills the script. In my case, the culprit was using a git config
setting that sets the ssh command, but $GIT_SSH_COMMAND
will override that setting. It's essentially the same problem reported in this comment. This is of course related to #2891, but a different problem. There's more relevant pointers in those issues, and I'll add this commit as the specific one that lead to this problem. (Though it looks like there was always an intention to set it.)
Having gone through a bunch of issues in this neighborhood, I see that this is not the first time that it bites someone. But more than that -- IMO it's actually bad to actually try to do the StrictHostKeyChecking
tweaking that this thing is doing. Yes, it can be annoying to figure out the ssh-keyscan
that is needed to pre-populate known hosts, but since this is potentially a security issue, I'd argue that disabling it is a BAD idea which should be left for people who want to do that. Even more so in a world where ephemeral runners are common, so you're constantly exposed to the risks that come with the disabling. (The original, plain usage of ssh makes it less dangerous, since you'd get the auto-trusted to happen on the first time you ssh to a new host, but then you'd have that persist. This is why disabling this and using it in an ephemeral runner is a bad idea.)
I therefore think that this particular setting should be removed. Or at least do it based on some setting that is not the default, so it's easier to do the ssh tweaking for people who opt into it (and the docs for that option would be a good place to warn people that it works by setting that env var).
As for the problem of existing things breaking, it shouldn't be too difficult to look at stderr, and if there's a message about an unknown host error, then add some text about the new option (whose page should describe the risk and what could be done to avoid it).
Hello!
I hope you are doing well!
We are a security research team. Our tool automatically detected a vulnerability in this repository. We want to disclose it responsibly. GitHub has a feature called Private vulnerability reporting, which enables security research to privately disclose a vulnerability. Unfortunately, it is not enabled for this repository.
Can you enable it, so that we can report it?
Thanks in advance!
PS: you can read about how to enable private vulnerability reporting here: https://docs.github.com/en/code-security/security-advisories/repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository
Currently GIT_SSH_COMMAND
has hardcoded ssh
path in https://github.com/npm/git/blob/v4.0.4/lib/opts.js#L5
But there is also GIT_SSH
env. var honored by standard git. The GIT_SSH_COMMAND
should use GIT_SSH
(and quoted) instead of ssh
when GIT_SSH
is non-empty.
Will fix npm/cli#2891 as I can confirm npm/cli#2891 (comment) fixes the problem.
GIT_SSH
other than ssh
/empty is honored.
Hi, would you be ok for a pull-request to replace mkdirp with the new native fs.mkdir recursive method ?
This is a Node.js 10.12.0 addition and Node.js 8 is EOL now so i guess the timing is ok.
Best Regards,
Thomas
As seen in the comments on npm/cli#3233 and #7, trying to install a git repository with the current npm CLI fails
when using OpenSSH <7.6 with the following:
npm ERR! command git --no-replace-objects ls-remote ssh://[email protected]/foo/bar.git
npm ERR! command-line line 0: unsupported option "accept-new".
npm ERR! fatal: Could not read from remote repository.
While the change is in theory a good one (I was caught off guard with the old behavior when cloning a repository failed due to a missing host key), it creates issues in practice.
Proposals I've seen include:
ssh -V
) before attempting to use accept-newThe library requires the lru-cache and semver packages but they are not declared as dependencies.
This is done at
Line 3 in 308ca27
Line 4 in 308ca27
Because yarn 2 uses PlugNPlay, it is not allowed to use non declared dependencies, causing my CI tools to fail
Temp fix is to add to .yarnrc.yml
packageExtensions:
"@npmcli/git@*":
dependencies:
lru-cache: "*"
semver: "*"
When running npm version ...
a commit/tag isn't created when you run within a git worktree. We should be able to detect when there's a ref to follow (ie. gitdir
) inside a relative .git
file. Need to update lib/is.js
to understand this scenario....
currently every operation performed through this module has to maintain its own cache. pacote half implements some form of caching, but now that we don't keep integrity values for locally built tarballs (i.e. git repositories) pacote will never actually use the cache.
in order to resolve this, i propose that we implement git specific caching semantics in this module.
these are just some thoughts i had about how this could work, that may or may not be useful when we implement.
ideally, we use cacache
so that we maintain consistency in our caching locations. this gives us a tiny bit of a challenge, however, in that we would need to cache specific entities in different ways
package.json
(including corgis, so this is really two entities in one key similar to how make-fetch-happen
handles different accept headers and content-typesnpm-shrinkwrap.json
if the package has one present, we need it accessiblenpm pack
where we start to venture into uncharted waters is with regards to how we identify and correctly handle stale repositories. to this end, i think one approach we could take is to also store a tarball comprising of the raw git repository in the cache. this tarball would be used as a means of determining if the requested reference has changed or not. the flow would look something like this:
cacache
or clone itgit fetch
, again skip if we just cloned)this would mean that every installation of a git repository will require us to extract a tarball of the repository, and do a git fetch
before retrieving anything but that's a very considerably better state than we have today where we clone the entire repository from scratch every time we need something from it.
npm run test
fails on machines where the tests take more than 30 seconds
Git cloning the repository on Windows and running npm run test
fails
On a Windows machine execute the following commands:
mkdir tests
cd tests
git clone [email protected]:npm/git.git git
cd git
npm install
npm run test
The expected result would be all tests passing. The obtained result is:
> @npmcli/[email protected] test
> tap
PASS test/find.js 3 OK 185.154ms
PASS test/is.js 5 OK 184.057ms
PASS test/opts.js 2 OK 134.044ms
PASS test/env.js 4 OK 15.652ms
PASS test/index.js 1 OK 16.549ms
PASS test/lines-to-revs.js 22 OK 91.419ms
PASS test/is-clean.js 6 OK 2s
PASS test/should-retry.js 3 OK 15.07ms
FAIL test/spawn.js
✖ expect resolving Promise
--- expected
+++ actual
@@ -1,3 +1,3 @@
Object {
- "stdout": "Initialized empty Git repository in c:/tests/git/test/spawn",
+ "stdout": "Initialized empty Git repository in C:/tests/git/test/spawn/.git/\n",
}
test: test/spawn.js setup repo
at:
line: 22
column: 12
file: test/spawn.js
type: Test
PASS test/which.js 5 OK 1s
PASS test/revs.js 20 OK 2s
FAIL test/spawn.js 1 failed of 6 4s
✖ expect resolving Promise
FAIL test/clone.js
✖ timeout!
test: test/clone.js again, with a submodule
expired: test/clone.js
FAIL test/clone.js
✖ timeout!
FAIL test/clone.js 2 failed of 33 31s
✖ timeout!
✖ timeout!
🌈 SUMMARY RESULTS 🌈
FAIL test/clone.js 2 failed of 33 31s
✖ timeout!
✖ timeout!
FAIL test/spawn.js 1 failed of 6 4s
✖ expect resolving Promise
Suites: 2 failed, 10 passed, 12 of 12 completed
Asserts: 3 failed, 107 passed, of 110
Time: 31s
npm run test
should succeed in an unchanged repository
npm run test
for this package fails (for instance) on the latest release of Ubuntu (20.10 groovy) fully upgraded. It happens due the latest stable version of the git
command line being 2.27.0 on that distro.
Fails for every attempt with a git version below 2.28.0
Git cloning the repository on any machine with a version of git installed below 2.28.0 and running npm run test
fails
On any machine with git below 2.28.0 installed (for instance, a fully upgraded Ubuntu 20.10 machine) execute the following commands:
mkdir tests
cd tests
git clone https://github.com/npm/git.git git
cd git/
npm install
npm run test
The expected result would be all tests passing. The obtained result is:
> @npmcli/[email protected] test /home/user/tests/git
> tap
PASS test/env.js 4 OK 111.436ms
PASS test/find.js 3 OK 130.919ms
PASS test/index.js 1 OK 55.775ms
PASS test/is-clean.js 6 OK 894.282ms
PASS test/is.js 5 OK 115.166ms
PASS test/lines-to-revs.js 22 OK 99.539ms
PASS test/opts.js 2 OK 77.576ms
FAIL test/revs.js
✖ command failed
if (code || signal)
rej(Object.assign(new Error('command failed'), result))
--------------------------^
else
res(result)
test: setup
stack: |
ChildProcess.<anonymous> (node_modules/@npmcli/promise-spawn/index.js:63:27)
at:
line: 63
column: 27
file: node_modules/@npmcli/promise-spawn/index.js
function: ChildProcess.<anonymous>
cmd: /usr/bin/git
args:
- init
- -b
- main
code: 129
signal: null
stdout: ""
stderr: >+
error: unknown switch `b'
usage: git init [-q | --quiet] [--bare] [--template=<template-directory>] [--shared[=<permissions>]] [<directory>]
--template <template-directory>
directory from which templates will be used
--bare create a bare repository
--shared[=<permissions>]
specify that the git repository is to be shared amongst several users
-q, --quiet be quiet
--separate-git-dir <gitdir>
separate git dir from working tree
--object-format <hash>
specify the hash algorithm to use
tapCaught: returnedPromiseRejection
FAIL test/revs.js
✖ command failed
if (code || signal)
rej(Object.assign(new Error('command failed'), result))
--------------------------^
else
res(result)
test: point latest at HEAD
stack: |
ChildProcess.<anonymous> (node_modules/@npmcli/promise-spawn/index.js:63:27)
at:
line: 63
column: 27
file: node_modules/@npmcli/promise-spawn/index.js
function: ChildProcess.<anonymous>
cmd: /usr/bin/git
args:
- ls-remote
- /home/user/tests/git/test/revs
code: 128
signal: null
stdout: ""
stderr: |
fatal: '/home/user/tests/git/test/revs' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
tapCaught: returnedPromiseRejection
FAIL test/revs.js
✖ command failed
if (code || signal)
rej(Object.assign(new Error('command failed'), result))
--------------------------^
else
res(result)
test: add a latest branch, point to 1.2.3 version
stack: |
ChildProcess.<anonymous> (node_modules/@npmcli/promise-spawn/index.js:63:27)
at:
line: 63
column: 27
file: node_modules/@npmcli/promise-spawn/index.js
function: ChildProcess.<anonymous>
cmd: /usr/bin/git
args:
- reset
- --hard
- version-1.2.3
code: 128
signal: null
stdout: ""
stderr: >
fatal: ambiguous argument 'version-1.2.3': unknown revision or path not in the
working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
tapCaught: returnedPromiseRejection
FAIL test/revs.js
✖ command failed
if (code || signal)
rej(Object.assign(new Error('command failed'), result))
--------------------------^
else
res(result)
test: check the revs
stack: |
ChildProcess.<anonymous> (node_modules/@npmcli/promise-spawn/index.js:63:27)
at:
line: 63
column: 27
file: node_modules/@npmcli/promise-spawn/index.js
function: ChildProcess.<anonymous>
cmd: /usr/bin/git
args:
- ls-remote
- /home/user/tests/git/test/revs
code: 128
signal: null
stdout: ""
stderr: |
fatal: '/home/user/tests/git/test/revs' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
tapCaught: returnedPromiseRejection
FAIL test/revs.js 4 failed of 4 921.343ms
✖ command failed
✖ command failed
✖ command failed
✖ command failed
PASS test/should-retry.js 3 OK 90.277ms
PASS test/clone.js 115 OK 22s
PASS test/which.js 5 OK 1s
PASS test/spawn.js 6 OK 4s
🌈 SUMMARY RESULTS 🌈
FAIL test/revs.js 4 failed of 4 921.343ms
✖ command failed
✖ command failed
✖ command failed
✖ command failed
Suites: 1 failed, 11 passed, 12 of 12 completed
Asserts: 4 failed, 172 passed, of 176
Time: 33s
npm run test
should succeed in an unchanged repository
npm run test
for this package fails on windows
Always
Git cloning the repository on Windows and running npm run test
fails
On a Windows machine execute the following commands:
mkdir tests
cd tests
git clone [email protected]:npm/git.git git
cd git
npm install
npm run test
The expected result would be all tests passing. The obtained result is:
> @npmcli/[email protected] test
> tap
PASS test/find.js 3 OK 185.154ms
PASS test/is.js 5 OK 184.057ms
PASS test/opts.js 2 OK 134.044ms
PASS test/env.js 4 OK 15.652ms
PASS test/index.js 1 OK 16.549ms
PASS test/lines-to-revs.js 22 OK 91.419ms
PASS test/is-clean.js 6 OK 2s
PASS test/should-retry.js 3 OK 15.07ms
FAIL test/spawn.js
✖ expect resolving Promise
--- expected
+++ actual
@@ -1,3 +1,3 @@
Object {
- "stdout": "Initialized empty Git repository in c:/tests/git/test/spawn",
+ "stdout": "Initialized empty Git repository in C:/tests/git/test/spawn/.git/\n",
}
test: test/spawn.js setup repo
at:
line: 22
column: 12
file: test/spawn.js
type: Test
PASS test/which.js 5 OK 1s
PASS test/revs.js 20 OK 2s
FAIL test/spawn.js 1 failed of 6 4s
✖ expect resolving Promise
FAIL test/clone.js
✖ timeout!
test: test/clone.js again, with a submodule
expired: test/clone.js
FAIL test/clone.js
✖ timeout!
FAIL test/clone.js 2 failed of 33 31s
✖ timeout!
✖ timeout!
🌈 SUMMARY RESULTS 🌈
FAIL test/clone.js 2 failed of 33 31s
✖ timeout!
✖ timeout!
FAIL test/spawn.js 1 failed of 6 4s
✖ expect resolving Promise
Suites: 2 failed, 10 passed, 12 of 12 completed
Asserts: 3 failed, 107 passed, of 110
Time: 31s
npm run test
should succeed in an unchanged repository
npm run test
leaves a test/is-clean folder behind, on Windows, after running
Git cloning the repository on Windows and running npm run test
leaves a test/is-clean folder behind even when the test is successful.
On a Windows machine execute the following commands:
mkdir tests
cd tests
git clone [email protected]:npm/git.git git
cd git
npm install
npm run test
npm run test
should leave no temporary folder after a successful run
There is already a description at length of this bug on npm/cli#2414 but a minimal use case can be provided if necessary
Whenever npm install
is run on a Windows system containing spaces in the path of the temporary folder when a git+https package is in package.json
npm/cli
npm install
fails with the following error:
npm ERR! code 129
npm ERR! command failed
npm ERR! command git clone <repo> <target containing spaces> --recurse-submodules --depth=1 --config core
A minimal reproducible usecase can be provided if necessary for this issue to be solved but, as #9 is a very straightforward fix, it may not be necessary
npm install
installs the package
Related to npm/cli#2414
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.