Git Product home page Git Product logo

all-the-package-names's Introduction

nice-registry [EXPERIMENTAL] ๐Ÿ”ฌ

A stateless proxy server to the npm registry that serves fancied up package metadata.

How it Works

This is what the server does when you fetch a package:

Raw Registry Data

First, the raw package metadata is fetched from registry.npmjs.com. This is the only network request made per package.

Dependents

The dependent-packages module adds dependency info for the package, without making a network request:

  • directDependents
  • directDependentsCount
  • directDevDependents
  • directDevDependentsCount
  • totalDirectDependentsCount

Downloads

The download-counts adds a averageDownloadsPerDay property to the package. This is also added without making a network request/

Owners

The owners and owner-profiles packages are used to create an owners array which includes the following metadata for each npm user, as available:

  • name - full name
  • email - public email address
  • homepage - a URL
  • github - GitHub handle
  • twitter - Twitter handle
  • gravatar - HTTPS gravatar url
  • packageCount - how many packages thes user owns

No network requests are made here either.

Niceties

Lastly, the resulting package object is turned into a [nice package], which has these qualities:

  • uses normalize-package-data as a baseline for cleanup, then:
  • uses the doc['dist-tags'].latest as the baseline for package metadata
  • derives starsCount from the users object
  • derives a versions array from the time object
  • renames _npmUser to lastPublisher, because it's a more intuitive name.
  • renames maintainers to owners, for consistency with the npm's CLI commands.
  • normalizes GitHub repository URLs to https format
  • moves internal bookkeeping properties like _id and _from into an other object that can easily be omitted.

HTTP API

The following routes are supported:

GET /package/:name

Fetches a single package. The following query params are allowed:

Examples:

GET /packages

Fetches multiple packages by name. The following query params are allowed:

  • names (required) - a comma-delimited string of package names to fetch
  • pick - a comma-delimited string of properties to include in the response.
  • omit - a comma-delimited string of properties to omit from the response.

Examples:

Public Server

There's a public instance running on Heroku.

Example URL: http://nice-registry.herokuapp.com/package/cheerio?omit=readme,versions,other,directDependents,

Running Your Own Instance

This server's design follows the 12 Factor development methodology popularized by Heroku. It is stateless: that is, it does not include a database, nor does it write to the filesystem.

Each package request makes just one network call, and that is to fetch the main package data from the npm registry. Supplementary data for download counts, dependents, owner profiles, etc, is provided by offline datasets that are automatically updated and published to npm by bots.

This server can optional cache responses in Redis. If you specify a REDIS_URL in the environment, then it will be used. If you don't, it won't. No configuration required.

Setting up your own instance on Heroku only requires a few commands:

git clone https://github.com/nice-registry/nice-registry
cd nice-registry
heroku create my-nice-registry
heroku addons:add heroku-redis
git push heroku master

Tests

npm install
npm test

License

MIT

all-the-package-names's People

Contributors

feross avatar ionicabizau avatar karlhorky avatar martinkolarik avatar zarianec avatar zeke avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

all-the-package-names's Issues

Figure out what to do with mixed-case package names

mixed-case packages names are no longer allowed on npm, but existing mixed-case packages are grandfathered in. I know of at least one package JSONStream that is mixed-case and actually legit, but for the most part they are probably junk.

Wondering if we should omit them?

Or maybe just mention in the README that they exist and should possible be omitted at require time.

https://github.com/npm/validate-npm-package-name#naming-rules
https://github.com/nice-registry/mixed-case-package-names

organization namespaced packages do not appear to be sorted

regular package names appear to be sorted by dependency count but organization namespaces appear to be out of order. E.g. these are the first 10 that appear in the list

 andrew ๎‚ฐ ~ ๎‚ฐ all-the-package-names | grep '^@' | head -10
@yuliy/supermodule
@kwiatkk1/native-dns
@eknkc/s3cp
@inversion-of-control/modular-electron-example-vendors
@beisen/skeleton
@johnhooks/dollar
@insin/babel-plugin-react-transform
@tonypythoneer/status
@xyezir/string-replace-loader
@caseywebb/hypernasa
@jialu/nodeapp
@react-material/button
@react-material/fab
@jwaterfaucett/animation_frame
@typed/is-promise

e.g. @types/is-promise has 1 dependent while @yuliy/supermodule has 0.

Package count as separate file

Hi,

I was interested in using this project, but in particular I only want the count of packages rather than the package names themselves. Would it be possible to save to another file the count of the packages so then I don't need to continually redownload a 50MB json file?

Is this still being updated daily?

Hello,

I love the idea of your package. It looks like it hasn't been running daily since May 25th according to the commit logs. Is this project still maintained?

Automatic updates have stopped

Just wanted to bring this to your attention in case you didn't notice. It seems the last update was 20 days ago. Thank you for making this project ๐Ÿ‘

line 20 makes lodash redundant?

Your is a lovely package and this is not a real issue, rather curiosity, but I was wondering the reason behind lodash as dependency.

In line 20

if (name && name.length && !names.includes(name)) names.push(name)

you ignore falsy values like compact would do, with a check on name.length that is also not needed since empty strings are falsy already so if ('') { alert(1); } never happens, and you filter duplicates via array.includes(name) like uniq, and after you get again all values you had before.

Not only you could've simply used an if (name) names.add(name) having names as new Set instead of an Array, you could also easily get rid of lodash as dependency.

You know, you're somehow responsible for its score in the top 10 ๐Ÿ˜„

Am I missing something obvious in your code?

Would be so kind to explain me why you need lodash here?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.