npm / normalize-package-data Goto Github PK
View Code? Open in Web Editor NEWnormalizes package metadata, typically found in package.json file.
License: Other
normalizes package metadata, typically found in package.json file.
License: Other
Current code does this:
function validRepo (file, data) {
// some other stuff
var r = data.repository.url || ""
// use the non-private urls
r = r.replace(/^(https?|git):\/\/[^\@]+\@github.com/,
'$1://github.com')
r = r.replace(/^https?:\/\/github.com/,
'git://github.com')
if (r.match(/github.com\/[^\/]+\/[^\/]+\.git\.git$/)) {
warn(file, data, "Probably broken git " +
"url: " + r)
}
}
I think that because var r
is a string, data.repository.url will never be updated. The only that could happen is a warning. Was that intended?
For example, the code currently sets the "_id" field. A concatenation of name and version.
It also adds all optional dependencies to dependencies. I have no experience with the "semantics" of optional dependencies, but it's certainly verbose.
People fields (author and such) are expanded to {name,email,url} objects. Handy when coding, but maybe less appropriate as a canonical version.
Normalization code like this might have two actually use cases: One for easy consumption by computers, one for fixing a human's mistake, but keeping the JSON somewhat humane.
Hi,
When i bundle my typescript to es6 project with webpack, i get the following error:
ERROR in ./~/normalize-package-data/lib/fixer.js Module not found: Error: Can't resolve './typos' in '<Path To>\node_modules\normalize-package-data\lib' @ ./~/normalize-package-data/lib/fixer.js 8:12-30 @ ./~/normalize-package-data/lib/normalize.js @ ./~/read-pkg/index.js @ ./~/read-pkg-up/index.js @ ./~/yargs/yargs.js @ ./bin/main.js
I can resolve the problem with changing the line 8 in fixer.js:
var typos = require("./typos")
to
var typos = require("./typos.json")
Is this a problem of my config or the fixer.js code?
By default (#31) shorthand URLs like
meryn/normalize-package-data
resolve to
git://github.com/meryn/normalize-package-data
Adding an option to override the git://github.com/ prefix would be nice as it would allow not to hardcode in-house project dependencies when used via npm.
What do you think ?
It's probably not a good idea to continue to accept semver before version 4.3.1, based on this security report:
repository.url
from '[email protected]:abc/def.git'
to 'git+ssh://[email protected]/abc/def.git%0A.git'
. Should it be 'git+ssh://[email protected]/abc/def.git'
?
I'm using the latest version of this module
Hi,
published distribution is 3.0.0 while github tags stops at 2.5.0. Could you add missing tags (at least the last) ?
Cheers, Xavier
Today I learned that code for package detail page of npm-www actually deduces a http url for repositories hosted on github. Up to now, I always had been copying and pasting a bit of the url to get to the repo. Stupid me.
This inference logic could also be put in normalize-package-json. In any case, it's better to happen somewhere in the model layer than in the view, I think.
In original async code, callback was immediately called with an error object
var ret = validName(file, data)
if (ret !== true) return cb(ret);
ret = validVersion(file, data)
if (ret !== true) return cb(ret);
I wonder if this is desirable if this function would be used inside npm-www for example. If requirements for name or versions get more strict (I admit, maybe far-fetched) the whole normalization process stops. Considering the packageData is modified in-place, the data after the throw is then unnormalized (or half-normalized, depending on where the throw happened).
hi all,
Currently, one of JSON files here is using a non-standard formatting with commas in front:
https://github.com/npm/normalize-package-data/blob/master/lib/typos.json
This causes minification to fail in Rollup + Babel + uglify-es scenarios.
I reported babel/babel#7858 it on the Babylon parser but please, do consider using a standard JSON notation, just like JSON formatters would produce. I know the benefits of leading colon, but the drawbacks with parsers later outweigh the extra inconvenience in Git diffs. I'm working with JSON's daily and I would never dare to format them with leading colons. Yes, the linter doesn't complain if you use leading colons but parsers actually do, later.
In general, you could switch to Prettier and let it manage the JSON formatting for you.
Thank you.
I'm dubious that this is still ideal, seeing as github itself no longer advertises "git" as a protocol option.
I'd like to suggest that we normalize to https or git+ssh urls instead.
What do you all think?
https://github.com/npm/normalize-package-data/blame/master/lib/normalize.js#L25-L29
Why is this assumption being made? It results in npm/npm#12133
I often come across a cool package on NPM and want to contribute, but I can't find the repo link. In the future, could we warn for packages that omit this link?
Would it be a good idea to email current authors of such packages and gently ask them to add these links?
lib/fixer.js#fixLicenseField doesn't take into account that the field
license
is also spelledlicence
, as it is spelled everywhere outside the US. This is already accounted for innpm-cli
.
package.json
file specifies licence
instead of license
as a field.npm WARN [email protected] No license field.
warning is displayed.package.json
file with the field licence
instead of license
.What is the purpose of this rule? It doesn't seem self-explanatory.
The values in optionalDependencies get added to dependencies. The optionalDependencies array is left untouched.
In original read-package-json, parseKeywords was defined, but never called.
Was this on purpose, or in error?
this seems undesirable
Was there a change regarding typos being a json file?
Getting errors in webpack:
Error: Cannot find module './typos'
at Function.Module._resolveFilename (module.js:325:15)
at Function.Module._load (module.js:276:25)
at Module.require (module.js:353:17)
at require (internal/module.js:12:17)
at Object.<anonymous> (node_modules/normalize-package-data/lib/fixer.js:8:13)
at Module._compile (module.js:409:26)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Module.require (module.js:353:17)
The previous version worked. Does a new version require a JSON loader added in webpack config or has it always been that way?
As part of the 2.4.1 release, you upgraded versions in your package.json. By upgrading is-builtin-module from 1.0.0 to 3.0.0, our build that uses Karma 0.5.5 is now breaking. We did not change any code or anything else...
Here is a picture comparing an npm install from yesterday and one from today on the same git commit hash. No code change was done but now the build is breaking
As you can see, our dependency version for karma did not change...karma's dependency version for meow did not change....meow's dependency version for normalize-package-data did increase by a patch version(0.0.1)...this caused is-builtin-modules to increase from 1.0.0 to 3.0.0
What I am getting at is that you updated the version for is-builtin-modules from 1.0.0 to 3.0.0, this is a backwards-incompatible change according to semantic version. See https://semver.org/. Thus by extension, your 2.4.1 release is also backwards incompatible with your 2.4.0 release
Idk if it is possible, but it would be extremely helpful if you deleted release 2.4.1 and re-released it as 3.0.0 so we do not have to resolve this backwards compatibility breaking change in our builds
This commit (99db55f) introduced the files
array but missed that lib
has .json
files and not just .js
.
The owner casing of a repo URL is important.
Currently if the repository field has a shorthand repo URL with case owner, the normalized version will have lower case owner name.
E.g, the owner of the shields
package has upper case letters in his username KenanY
. Hence, in the repository
property of his package.json he uses this shorthand repo name:
"repository": "KenanY/shields"
However, after normalizing the package, it is loosing the casing:
"repository": {
"url": "git+https://github.com/kenany/shields.git"
}
Live example: https://tonicdev.com/56fc202b81168d1100ac3b5d/57001370a2d934120019b64e
Owner-name casing is important! Shields from http://shields.io/ are case sensitive and with incorrect owner casing they return 404 errors.
I can prepare a pull request
Currently, there's a local "fixer.js" containing the value-inference code.
If this was put in a separate module, other interfaces to this code could be written.
Normalization of all package data is one application.
Another application could be to make a "class" that takes package data in its constructor, than has methods to get the inferred/fixed value for any field.
pkgData = new PkgData rawPkgData
console.log pkgData.getBugsURL()
Sometimes normalizing all fields isn't needed. For example, in the code for npm bugs
command.
I do consider this a "nice to have". Far from critical.
I wanted to use μήτρα (matrix) in my package field as name but it says invalid name. Doing the same tests as in the script doesn't give me this result. Any Ideas what to do (except changing the name)?
Remove it always.
Remove it if neither valid url or email were given, or could have been inferred. (current behavior)
Set it to an empty object.
Leave as much fields as is, but
a. remove offending properties
b. coerce offending properties to string
Properties are email and url, which must be strings. They could also be validated to be email addresses or http urls, respectively.
Currently, the objectifyDeps function goes over
var depTypes = [ "dependencies"
, "devDependencies"
, "optionalDependencies" ]
Hi,
try update: node-normalize-package-data
node-validate-npm-package-license dependency problem:
node-validate-npm-package-license : Depende: node-spdx-expression-parse (>= 1.0.0) which is a virtual package and is not provided by any available package
Sorry if invalid or duplicated.
Tell if needed.
Thanks
There's an interaction between hosted-git-info and normalize-package-data that's causing the repository URL to be output with a .git.git
extension.
Here's an example of data that fails:
{ type: 'git', url: 'git://github.com/balderdashy/waterline.git/' }
When there’s no description, the README will often start with badges. Either these could be removed (removing all images from the description), either it should be left blank, since as a reuser, it’s almost impossible to know a “real” description from an unpredictable one, that could be improved with other sources like GitHub.
It might be nice to have an interface like this:
var p = require('read-package-data').Parser()
p.on('warning', function(warning) {
doSomething(warning)
})
p.parse(data)
What do you think?
The original code only warns about typos. When consuming package data in npm-www for example, it may be desirable to actually fix the (known) typos.
On the other hand, this could promote sloppy coding of package.json by publishers.
This could be addressed by having "npm publish" reject package.json files that contain (known) typos.
https://github.com/npm/newww/issues/840
Somehow the mobile-detect package made it through the publish process (3 weeks ago) without its broken repository
value getting discarded or repaired:
repository: {
type: "git",
url: "github.com/hgoebl/mobile-detect.js"
}
For example, this is a syntax error.
Currently, the code corrects known fields, but leaves unknown fields alone. Is it a goal to give people the ability to put arbitrary fields in package.json?
Some of original code gave warnings but did not fix. For example, the various typos that were defined.
Other parts corrects things silently. For example (from original).
function bugsTypoWarn (file, data) {
var b = data.bugs
if (!b || typeof b !== "object") return
Object.keys(b).forEach(function (k) {
if (bugsTypos[k]) {
b[bugsTypos[k]] = b[k]
delete b[k]
}
})
}
scriptTyposWarn on the other hand, warns but does not correct.
Hello,
I'm packaging atom [1] for debian [2].
Your software is there included.
I found some issues in your software:
Regards,
Jörg Frings-Fürst
[1] http://atom.io
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=747824
[3]
// Copyright 20xx-20xx your name
// License: [your license eg. MIT / GPL-2 ....]
Some modules just have 'example.org' for their homepage in package.json. It would be great to normalize this by prepending 'http://' if no protocol is given.
This error is thrown in my project
Error: Invalid version: "207-alpha1-SNAPSHOT"
Our version in package.json is sentimental, not in semver.
In error stack trace I can see above error comes from
...
at Object.fixVersionField (.../node_modules/normalize-package-data/lib/fixer.js:191:13)
at .../node_modules/normalize-package-data/lib/normalize.js:32:38
at Array.forEach (<anonymous>)
at normalize (.../node_modules/normalize-package-data/lib/normalize.js:31:15)
at Function.module.exports.sync (.../node_modules/webpack-babel-env-deps/node_modules/read-pkg/index.js:43:36)
Not sure if this is related to your project. Feel free to close if I'm wrong here.
Thanks.
Since #58 hosted-git-info is used to normalize dependencies. To be consistent, I suggest doing the same for the repository
property.
This would mean that
"repository": "npm/normalize-package-data"
would be normalized to
"repository": "github:npm/normalize-package-data"
instead of
"repository": {
"type": "git",
"url": "git+https://github.com/npm/normalize-package-data.git"
}
Not guaranteed to be array: keywords, contributors, authors, man.
Guaranteed to be array (if set): files
Let's get this logic centralized. This'd replace ensureValidName
. Things that use fixNameField
to validate names should probably be updated to use validate-npm-package-name
directly too. cough npm-registry-client
cough
When we do an 'npm install' on this component, we get an extra file in the repository
normalize-package-data\test\normalize.js~
This is showing up because of our .gitignore settings.
I don't believe this file is needed and is only there because it's a back-file made by the text editor.
I request that it be removed in the next publish to npm registry
The file does not appear when you do a clone from github
npmjs.com has 2.4.0 as latest release but latest tag in github repo is 2.3.8. Please update the tags.
Currently, the function returns no values.
var ret = packageData(require('./package.json'));
ret; //=> undefined
I want it to return the normalized data.
var ret = packageData(require('./package.json'));
ret; //=> { ... } (normalized object)
Someone has managed to get themselves massively confused over in isaacs/npm#3826.
Originally, I thought that it would be best to separate the code for generating warnings from the code that make changes to data. But that was before I thought of the (quite elegant) solution of passing a warn function along.
However, it may be interesting to check for package.json validity (perhaps on several levels of correctness) while leaving the package data alone.
The consumer of this function could also do it by himself, by calling normalize on a cloned object, and pass a "warn" function that (for example) sets a "hasWarned" to true, then check this hasWarnings value afterwards.
Original code:
function validName (file, data) {
if (!data.name) {
data.name = ""
return true
}
// some more stuff
}
Perhaps in the whole npm flow, name and version are checked somewhere else as well. But as stand-alone module, I think it'd be better to throw here as well.
If you run array dependencies through normalizePackageData
(depObjectify
specifically), the leading @
in a scoped package name will be incorrectly interpreted as the beginning of a range instead of the beginning of the package name, leading to the "fixed" dependency name being the empty string and the dependency semver range being a github specifier.
While array dependencies are not encouraged, they are still allowed to be published, and if nothing else this leads to very strange error messages:
$ npm publish
$ 400 Bad Request - PUT https://registry.npmjs.org/your-package - "dependencies" dep "" is not a valid dependency name.
Again, non-scoped-package array dependencies can be published successfully, and if anything you'd want this message to say "scoped packages are not allowed in array dependencies".
Here is a reproducible test case: https://runkit.com/tolmasky/scoped-package-in-array-dependencies
Happy to submit a PR if it is agreed that this is not desired behavior.
The readme states:
You may combine strict validation with warnings by passing true as the second argument, and warnFn as third.
However, the code is the other way around:
function normalize (data, warn, strict) {
if(warn === true) warn = null, strict = true
Which one of these should be changed?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.