mapbox / secret-shield Goto Github PK
View Code? Open in Web Editor NEWEasily find secrets in files, directories, and repositories. Stop leaking secrets using git hooks.
License: MIT License
Easily find secrets in files, directories, and repositories. Stop leaking secrets using git hooks.
License: MIT License
What repository are you running secret-shield on?
https://github.com/mapbox/subdomain-docs/
What version of secret-shield are you running?
{
"version": "1.0.2",
"node": "v12.20.0",
"os_platform": "darwin",
"os_release": "20.4.0",
"installed_dir": "/Users/domlet/.nvm/versions/node/v12.20.0/lib/node_modules/@mapbox/secret-shield/bin"
}
How did you invoke secret-shield?
I followed these steps:
➜ subdomain-docs git:(db-redirectstyles) ✗ npm install -g @mapbox/secret-shield
npm WARN deprecated [email protected]: This module is now under the @mapbox namespace: install @mapbox/decrypt-kms-env instead
npm WARN deprecated [email protected]: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
npm WARN deprecated [email protected]: Backported compatibility to node > 6
/Users/domlet/.nvm/versions/node/v12.20.0/bin/secret-shield -> /Users/domlet/.nvm/versions/node/v12.20.0/lib/node_modules/@mapbox/secret-shield/bin/secret-shield.js
+ @mapbox/[email protected]
updated 1 package in 6.161s
➜ subdomain-docs git:(db-redirectstyles) ✗ secret-shield --add-hooks global
Successfully added global hooks!
I also followed the instructions at:
...but I still cannot commit to this repo:
➜ subdomain-docs git:(db-redirectstyles) ✗ git commit -m "update redirects for GSG edition"
husky > pre-commit (node v12.20.0)
/bin/sh: Studio: command not found
ERROR! You must have secret-shield installed and configured globally to commit to this repository. To set up secret-shield, follow these instructions: https://github.com/mapbox/secret-shield/blob/master/docs/enabledBadge.md
husky > pre-commit hook failed (add --no-verify to bypass)
How is secret-shield not working as expected?
I cannot commit. When I try:
➜ subdomain-docs git:(db-redirectstyles) ✗ git commit -m "update redirects for GSG edition"
I get this error:
husky > pre-commit (node v12.20.0)
/bin/sh: Studio: command not found
ERROR! You must have secret-shield installed and configured globally to commit to this repository. To set up secret-shield, follow these instructions: https://github.com/mapbox/secret-shield/blob/master/docs/enabledBadge.md
husky > pre-commit hook failed (add --no-verify to bypass)
secret-shield introduces a layer of protection against accidentally committing secrets to public repos. What if we want to protect against accidentally committing to public repos at all?
We should add a pre-push hook to the default secret-shield suites. This hook would:
If both are true, the hook would prompt with a confirmation message.
> git push -u origin foo
WARNING! This repository is PUBLIC, and you have never committed to this repo before.
Are you sure you want to push refs to '[email protected]:mapbox/secret-shield'? y/N
cc/ @mapbox/security-and-compliance @ectrotter @tmpsantos
Tracking issue for: utils/readfiles.js:26
:
Are you likely to extend this to npm publish
?
I'm embarrassed to admit I've accidentally leaked secret keys through npm not realising npm publish
takes the whole directory including files not staged!
What repository are you running secret-shield on?
https://github.com/mapbox/1tap-android
What version of secret-shield are you running?
{
"version": "1.0.2",
"node": "v12.22.1",
"os_platform": "darwin",
"os_release": "20.6.0",
"installed_dir": "/usr/local/lib/node_modules/@mapbox/secret-shield/bin"
}
How did you invoke secret-shield?
with AndroidStudio
terminal works as expected
but with SourceTree
I get error
How is secret-shield not working as expected?
SourceTree
output:
/Users/nikitos/.git/hooks/pre-commit/pre-commit: line 2: secret-shield: command not found
➜ ~ cat .git/hooks/pre-commit/pre-commit
#!/bin/sh
secret-shield --pre-commit -C verydeep --enable "Mapbox Public Key" --disable "High-entropy base64 string" "Short high-entropy string" "Long high-entropy string"
What repository are you running secret-shield on?
tried on multiple mapbox repositories
What version of secret-shield are you running?
There is no -v
flag, so I'm not sure
How did you invoke secret-shield?
`secret-shield --add-hooks local`
How is secret-shield not working as expected?
Errors with :
Could not find the local hooks to add -- is your secret-shield module correctly installed?
First reported by @chezzdev and I can reproduce the issue.
What repository are you running secret-shield on?
Private repo
What version of secret-shield are you running?
Unknown:
$ secret-shield --version
Unknown option: undefined
Usage: secret-shield <--string|--file|--directory|--repository> target
For detailed help run secret-shield --help or check the documentation.
How did you invoke secret-shield?
pre-commit via Husky - looks like it's running
#!/usr/bin/env node
'use strict';
const secretShield = require('@mapbox/secret-shield');
process.exit(secretShield.checkAndRun('2018-07-01'));
How is secret-shield not working as expected?
➜ my-repo (update-stuff) ✗ git commit -m 'Updates to stuff, remove dependency'
husky > pre-commit (node v8.10.0)
/bin/sh: Text.app/Contents/SharedSupport/bin:~/Applications/Sublime: No such file or directory
ERROR! You must have secret-shield installed and configured globally to commit to this repository. To set up secret-shield, follow these instructions: https://github.com/mapbox/secret-shield/blob/master/docs/enabledBadge.md
husky > pre-commit hook failed (add --no-verify to bypass)
⌦ my-repo (update-stuff) ✗ npm i -g @mapbox/secret-shield
/Users/andrewevans/.nvm/versions/node/v8.10.0/bin/secret-shield -> /Users/andrewevans/.nvm/versions/node/v8.10.0/lib/node_modules/@mapbox/secret-shield/bin/secret-shield.js
+ @mapbox/[email protected]
added 225 packages in 4.847s
➜ my-repo (update-stuff) ✗ git commit -m 'Updates to stuff, remove dependency'
husky > pre-commit (node v8.10.0)
/bin/sh: Text.app/Contents/SharedSupport/bin:~/Applications/Sublime: No such file or directory
ERROR! You must have secret-shield installed and configured globally to commit to this repository. To set up secret-shield, follow these instructions: https://github.com/mapbox/secret-shield/blob/master/docs/enabledBadge.md
husky > pre-commit hook failed (add --no-verify to bypass)
⌦ my-repo (update-stuff) ✗ which secret-shield
/Users/andrewevans/.nvm/versions/node/v8.10.0/bin/secret-shield
➜ my-repo (update-stuff) cat ~/.gitconfig | grep hooksPath
hooksPath = /Users/andrewevans/.nvm/versions/node/v9.5.0/lib/node_modules/@mapbox/secret-shield/config/hooks
➜ my-repo (update-stuff) ✗ git commit -m 'Updates to stuff, remove dependency' --no-verify
[update-stuff 3551437] Updates to stuff, remove dependency
3 files changed, 4522 insertions(+), 7 deletions(-)
create mode 100644 package-lock.json
I don't know what it wants, so I gave up and used --no-verify
. Error messages are totes bizarre. I expect PATH issues since I'm using nvm
, but I have no idea why it would be checking Text.app/Contents...
- that is for sure not on my PATH
Readme still recommends Node 4,6,8 - all of which have reached EOL 😓
We should upgrade this repo.
Tracking issue for: lib/analyzer.js:141
:
Tracking issue for: utils/readfiles.js:18
:
What repository are you running secret-shield on?
Multiple mapbox repos
What version of secret-shield are you running?
1.0.2
How did you invoke secret-shield?
pre-commit git hook
How is secret-shield not working as expected?
In projects with submodules where submodules are often changed and developed in parallel with the parent project there is common thing when submodule is in modified state.
Then secret-shield wrongly detects GitHub token
secret in Subproject commit *
string, e.g.
.
I've been able to disable this false positive by adding
{
"type": "remove",
"name": "Ignore submodule commits",
"pattern": "\\bSubproject commit [0-9a-zA-Z]+\\b"
},
entry to the "preprocess":
section of the json rule. Though it seem every rule will need to be modified similarly to avoid this falsepositive.
Initially I'd expected that it could be solved by just adding Submodule commit
string to the falsePositive json here, as docs state here string should be ignored completely https://github.com/mapbox/secret-shield/blob/main/docs/writingRules.md#bulk-ignore. But it didn't work for me, and from the source code it seems that items from falsePositive json are only replaced in the string so it works differently, not ignoring whole string but just the mathching part. Not sure if that's intended of a bug and should be fixed.
Why isn't installing secret-shield as a dependency in my repository sufficient? What's driving the demand that I manage a global npm secret-shield installation?
What repository are you running secret-shield on?
Mapbox private repo.
What version of secret-shield are you running?
How did you invoke secret-shield?
pre-commit
How is secret-shield not working as expected?
When I invoke command secret-shield --add-hooks global I got an error:
internal/modules/cjs/loader.js:1102
throw new ERR_REQUIRE_ESM(filename, parentPath, packageJsonPath);
^
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /home/pavel/.local/share/fnm/node-versions/v14.20.1/installation/lib/node_modules/@mapbox/secret-shield/node_modules/p-queue/dist/index.js
require() of ES modules is not supported.
require() of /home/pavel/.local/share/fnm/node-versions/v14.20.1/installation/lib/node_modules/@mapbox/secret-shield/node_modules/p-queue/dist/index.js from /home/pavel/.local/share/fnm/node-versions/v14.20.1/installation/lib/node_modules/@mapbox/secret-shield/searchRepo.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /home/pavel/.local/share/fnm/node-versions/v14.20.1/installation/lib/node_modules/@mapbox/secret-shield/node_modules/p-queue/package.json.
at new NodeError (internal/errors.js:322:7)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1102:13)
at Module.load (internal/modules/cjs/loader.js:950:32)
at Function.Module._load (internal/modules/cjs/loader.js:790:12)
at Module.require (internal/modules/cjs/loader.js:974:19)
at require (internal/modules/cjs/helpers.js:101:18)
at Object.<anonymous> (/home/pavel/.local/share/fnm/node-versions/v14.20.1/installation/lib/node_modules/@mapbox/secret-shield/searchRepo.js:3:16)
at Module._compile (internal/modules/cjs/loader.js:1085:14)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
at Module.load (internal/modules/cjs/loader.js:950:32) {
cod
```e: 'ERR_REQUIRE_ESM'
}
Used node v14.20.1.
What repository are you running secret-shield on?
Private repo
What version of secret-shield are you running?
npm list -g
throws a bunch of errors like:
npm ERR! missing: acorn@^5.5.3, required by @mapbox/[email protected]
npm ERR! missing: command-join@^2.0.0, required by @mapbox/[email protected]
npm ERR! missing: command-line-args@^5.0.2, required by @mapbox/[email protected]
npm ERR! missing: d3-queue@^3.0.7, required by @mapbox/[email protected]
npm ERR! missing: decrypt-kms-env@^3.0.0, required by @mapbox/[email protected]
npm ERR! missing: es6-promise@^4.2.4, required by @mapbox/[email protected]
npm ERR! missing: esprima@^4.0.0, required by @mapbox/[email protected]
npm ERR! missing: fast-fuzzy@^1.5.0, required by @mapbox/[email protected]
npm ERR! missing: find-in-files@^0.5.0, required by @mapbox/[email protected]
1.0.2
How did you invoke secret-shield?
global pre-commit hook:
$ git commit -m 'Fix typo in cfn template'
module.js:471
throw err;
^
Error: Cannot find module 'command-line-args'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/Users/andrewevans/Projects/secret-shield/bin/secret-shield.js:9:14)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
$ git commit -m 'Fix typo' --no-verify
[github-apps-v2 c9059f2] Fix typo
1 file changed, 1 insertion(+), 1 deletion(-)
$ node --version
v6.12.3
$ which secret-shield
/Users/agius/.nvm/versions/node/v6.12.3/bin/secret-shield
I have no idea what's going on with the hooks and nvm here, or even what the cwd
is when running secret-shield
Tracking issue for:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.