Git Product home page Git Product logo

statusboard's Introduction

๐Ÿ“Š statusboard

npm.github.io/statusboard

A single view to help monitor the status/health of the @npm/cli teams's Open Source Projects.

Data

Adding a project

GitHub search query for org:npm topic:npm-cli fork:true

To be included here a repo must be in the npm org and have the npm-cli topic added to it. The data is rebuilt daily so the repo should appear by the next day.

Repos are also searched for workspaces via the package.json#workspaces array and those are included if they do not have package.json#private.

Removing a project

Projects are removed from the list if they are archived on GitHub and deprecated on the npm registry, or if they are archived and moved to a workspace of another repo we track. The topic npm-cli does not need to be removed and should be kept on GitHub repos for historical reference.

Developing:

To update maintained projects:

Locally: npm run fetch:maintained -w data

CI: gh workflow run fetch-maintained.yml

To update daily data:

Locally: npm run fetch:data -w data

CI: gh workflow run fetch-data.yml

Serve / Publish Site

Serve: npm run dev -w www

Publish: The site is published to GitHub Pages on all pushes to main.

Forking

This project aims to have some portability, but it will not work out-of-the-box if forked. Here are the things we're aware of that you should change if you fork this:

  • Enable the GitHub Action workflows after forking. They are not enabled by default on forked repos
  • Update misc links and references to your org
  • Delete all historical data from workspaces/www/lib/data/**/*.json
  • Update the necessary config items in:
  • Have a AUTH_TOKEN=${{ GITHUB TOKEN}} in workspaces/data/.env if you are fetching data locally. All the data here is from open source repos and packages on the npm registry. So if you're data is private this token will need to have the proper scopes. You can look in data/lib/api/ to see all the calls that are made.
  • Run npm run fetch:maintained -w data and npm run fetch:data -w data to populate your new data

statusboard's People

Contributors

actions-user avatar darcyclarke avatar dependabot[bot] avatar gyan-droid avatar jamesmgreene avatar jumoel avatar lukekarrys avatar npm-cli-bot avatar npm-deploy-user avatar wraithgar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

statusboard's Issues

Add `funding` to package.json

  • Implement npm i "nag" at the end of output
    ex. 23 packages are looking for funding. Run "npm fund" to find out more.
  • Implement --no-fund flag to opt-out of the "nag" when installing
  • Implement npm fund subcommand which lists out all the funding references
  • Write tests for the "nag", --no-fund & the npm fund subcommand
  • Document the subcommand functionality
  • Document the schema change of package.json (can use/reference this spec once updated)
  • Add a visual representation for the funding field/value on package pages
  • Add funding to Pakument/Corgi objects: https://github.com/npm/validate-and-store it looks like we don't really need it, the reference implementation at npm repo is actually using the full metadata in which makes sense since we don't want to tax the entire registry just for users that are manually running the npm fund <package> command.

[Web] Improve Analytics Coverage

  • Add tracking on package page for outbound clicks (use GA to send events)
  • Add tracking on package page for tab clicks (use GA to send events)
  • Add tracking on search page for sort clicks (use GA to send events)
  • Add tracking for Stripe payment user flow (use GA to set "goals")
  • Add tracking for Email invite user flow (use GA to set "goals")

New Docs Website

  • Create a build pipeline/process, separate from the cli builds, to generate the docs site (use GitHub actions if possible)
  • Add persistent toggle options for sub navs (ie. closed/opened states persist page reloads)
  • File a ticket with SRE team to create a subdomain (ex. preview.npmjs.com)
  • Work with SRE team to determine proper origin
  • Talk with Ahmad about the future of the support.npmjs.com Zendesk help portal (ask about timelines/release schedule)

New CLI Docs

  • Migrate the new Gatsby site into @npm/cli/docs/
  • Clean up styles & the overall template's code
  • Copy existing cli docs into the new format
  • Remove irrelevant documentation (ie. anything that is not cli-specific)

Epic: Automated release spike

  • Identify a project (not cli) to automate the releases of
  • Set up CI integration to run tests/benchmarks
  • Set up CD integration to automate versioning/releasing/publishing

[FEATURE] `npm publish` more information / gated command

What / Why

Curerntly npm publish just works but mistakes can happen and people don't wall know about --dry-run which adds some additional details without publishing. It would be great if npm publish had a gated [N/y] input, and would also display the output of --dry-run by default. This would allow the user to inspect before publishing. A --yes flag should be added to allow ci to continue to function correctly.

When

  • n/a

Where

  • n/a

How

Current Behavior

  • n/a

Expected Behavior

  • n/a

Who

  • n/a

References

  • n/a

Create npm.community Landing Page

  • Design/developer mock page
  • Determine deployment/hosting/domain
  • Work with marketing on messaging & what links to use
  • Queue up for launch on Tuesday, November 5th

Archive legacy projects

  • Identify criteria for legacy projects (ex. anything that hasn't been touched in 2 years)
  • Query for projects meeting the criteria
  • Queue up engineering org about intent
  • Give grace period
  • Archive projects
[
  "https://github.com/npm/npm-lylog",
  "https://github.com/npm/package-audit",
  "https://github.com/npm/redsess",
  "https://github.com/npm/fastly-audit-purge",
  "https://github.com/npm/node-github-url-from-git",
  "https://github.com/npm/npm-fastly-purge",
  "https://github.com/npm/ansible-bashpocalypse",
  "https://github.com/npm/npm_conf",
  "https://github.com/npm/ec2tail",
  "https://github.com/npm/npm-skim-registry",
  "https://github.com/npm/npm-tutor",
  "https://github.com/npm/npm-registry-readme-trim",
  "https://github.com/npm/registry-cert-verification",
  "https://github.com/npm/npm-collection-staff-picks",
  "https://github.com/npm/lylog",
  "https://github.com/npm/normalize-license-data",
  "https://github.com/npm/rderby",
  "https://github.com/npm/shirts",
  "https://github.com/npm/git-repo-manager",
  "https://github.com/npm/css",
  "https://github.com/npm/npm-logbot",
  "https://github.com/npm/callback-tracker",
  "https://github.com/npm/couch-readonly-replica",
  "https://github.com/npm/keyword-context",
  "https://github.com/npm/npm-fullfat-registry",
  "https://github.com/npm/pingme",
  "https://github.com/npm/graphics",
  "https://github.com/npm/create-test-package",
  "https://github.com/npm/npme-trusty",
  "https://github.com/npm/npme-unknown",
  "https://github.com/npm/site-structure",
  "https://github.com/npm/oss-license-name-to-url",
  "https://github.com/npm/ansible-ftools",
  "https://github.com/npm/npme-centos65",
  "https://github.com/npm/readdir-scoped-modules",
  "https://github.com/npm/npme-ansible-module",
  "https://github.com/npm/Paquetes-de-Codigo-Abierto",
  "https://github.com/npm/uid-number",
  "https://github.com/npm/npmo-auth-ldap",
  "https://github.com/npm/scope-it",
  "https://github.com/npm/numbat-analyzer",
  "https://github.com/npm/npm-marketing-changes-feed",
  "https://github.com/npm/Paquetes-de-Codigo-Abierto",
  "https://github.com/npm/npmconf",
  "https://github.com/npm/newww-tests",
  "https://github.com/npm/talk-proposals",
  "https://github.com/npm/oss-weekly",
  "https://github.com/npm/npm-marketing-sidebar-blob",
  "https://github.com/npm/blankie",
  "https://github.com/npm/eloqua-client",
  "https://github.com/npm/divot",
  "https://github.com/npm/npmo-auth-sso",
  "https://github.com/npm/tarball-url-info",
  "https://github.com/npm/token-facilitator",
  "https://github.com/npm/node-oauth2-server",
  "https://github.com/npm/express-oauth-server",
  "https://github.com/npm/handlebars-helper-from-package",
  "https://github.com/npm/npm-tips",
  "https://github.com/npm/normalize-registry-metadata",
  "https://github.com/npm/realize-package-specifier",
  "https://github.com/npm/new-docs",
  "https://github.com/npm/private-pkgs-orgs-docs",
  "https://github.com/npm/publishing-pkgs-docs",
  "https://github.com/npm/npme-auth-bitbucket",
  "https://github.com/npm/installation-setup-docs",
  "https://github.com/npm/fstream-ignore",
  "https://github.com/npm/registry-docs",
  "https://github.com/npm/ansible-pg-failover-config",
  "https://github.com/npm/npmo-auth-token",
  "https://github.com/npm/ifttt-hook-translator",
  "https://github.com/npm/sign-payload",
  "https://github.com/npm/marky-markdown-integration-test",
  "https://github.com/npm/how-npm-works-docs",
  "https://github.com/npm/readme-tester",
  "https://github.com/npm/ansible-role-deploy-update-code-git",
  "https://github.com/npm/captain-hook",
  "https://github.com/npm/tumblr-theme",
  "https://github.com/npm/support-cli",
  "https://github.com/npm/external-fetch",
  "https://github.com/npm/redis",
  "https://github.com/npm/npm-camp",
  "https://github.com/npm/nicely-placed-modification-logs",
  "https://github.com/npm/ansible-pg-failover",
  "https://github.com/npm/handlebars-helper-icon",
  "https://github.com/npm/npm-explicit-installs",
  "https://github.com/npm/npmo-docker-compose",
  "https://github.com/npm/npme-vagrant",
  "https://github.com/npm/couch-login",
  "https://github.com/npm/using-pkgs-docs",
  "https://github.com/npm/dr-frankenstyle",
  "https://github.com/npm/npm-like-im-5",
  "https://github.com/npm/public-api",
  "https://github.com/npm/dyn-js",
  "https://github.com/npm/route66",
  "https://github.com/npm/kenny-logins",
  "https://github.com/npm/ansible-percona",
  "https://github.com/npm/pkgize",
  "https://github.com/npm/marky-compare",
  "https://github.com/npm/handlebars-react",
  "https://github.com/npm/next.js",
  "https://github.com/npm/postgresql",
  "https://github.com/npm/background-refresh-cache",
  "https://github.com/npm/npme-auth-oauth2",
  "https://github.com/npm/annotation-api",
  "https://github.com/npm/annotation-poller",
  "https://github.com/npm/ansible-influxdb",
  "https://github.com/npm/ansible-role-elasticsearch",
  "https://github.com/npm/decorate",
  "https://github.com/npm/forza",
  "https://github.com/npm/fstream-npm",
  "https://github.com/npm/google-container-engine",
  "https://github.com/npm/hapi-stateless-notifications",
  "https://github.com/npm/humans",
  "https://github.com/npm/multi-fs",
  "https://github.com/npm/nm",
  "https://github.com/npm/npm-authify",
  "https://github.com/npm/npm-docker-couchdb",
  "https://github.com/npm/npm-hook-receiver",
  "https://github.com/npm/npme-auth-github",
  "https://github.com/npm/npme-auth-oauth2-restricted",
  "https://github.com/npm/npme-installer",
  "https://github.com/npm/npmi-cli",
  "https://github.com/npm/nsqjs",
  "https://github.com/npm/oauth2-server-pg",
  "https://github.com/npm/pivotal-ui",
  "https://github.com/npm/read",
  "https://github.com/npm/redis-pool",
  "https://github.com/npm/rust-readme-client",
  "https://github.com/npm/security-holder",
  "https://github.com/npm/seq-file",
  "https://github.com/npm/spife-dev-logger",
  "https://github.com/npm/tarpit",
  "https://github.com/npm/team-info-cache",
  "https://github.com/npm/weallcontribute",
  "https://github.com/npm/with-fixtures"
]

[FEATURE] <title>

What / Why

n/a

When

  • n/a

Where

  • n/a

How

Current Behavior

  • n/a

Expected Behavior

  • n/a

Who

  • n/a

References

  • n/a

[Web] Clean Up Search Experience

  • Remove "powered by npms.io" at bottom of search pages
  • TBD: Fix, change or remove scoring/sorting if the downstream implementation is expected to continue to diverge from npms.io
  • Sync up with @djsauble on the right path forward for the above

Add pretty-printed WARN message upon non-2FA-enabled-publish

In order to better promote adoption of 2FA among package publishers we could add a catchy-looking warning message that gets printed to the user upon every publish (if that user has disabled 2FA publish). Some other optional ideas:

  • Add a Y/N prompt confirmation (in order to guarantee users read the warning info)
  • Add a direct link to: https://www.npmjs.com/settings/<username>/tokens/create

[FEATURE] add column for ci/cd information

What / Why

At a glance would be nice to see if the repository is using travis, appvoyer, or github actions. More specifically what matrix of builds it's running.

When

Should be displayed in a column on the status board.

Where

  • n/a

How

Current Behavior

Only the build status of the "default" branch is shown. No information about the repositories build matrixes are present.

Expected Behavior

Pulling the files content from a repository should be able to find at a top level if the .travis.yml or .appvoyer.yml or .github/workflow/ files exist. Travis and Appvoyer would be the easiest of the files to parse, you can just look for os: and nodejs versions, to determine the build matrix. The github actions will be a little more work. We'll have to find the on: field and look for push or pull_request for the "default" branch and review the strategy of that to determine the build matrix.

References

  • n/a

[FEATURE] <title>

What / Why

n/a

When

  • n/a

Where

  • n/a

How

Current Behavior

  • n/a

Expected Behavior

  • n/a

Who

  • n/a

References

  • n/a

Add monitoring/benchmarking tools

  • Add a benchmark suite to test against our builds (ex. pnpm benchmarks)
  • Create a build pipeline/process, separate from the cli builds, to run the benchmarks & generate a report (use GitHub actions if possible)
Requirements

Requirements

  • Implement benchmark/ folder with script to run benchmarking commands [R]
    • create execution pattern
    • create scenario's paradigm
  • Create github action to run benchmark script [R]
    • link current commit of repo to global node_modules/ [R]
    • run benchmark script which will run scenarios [R]
    • output benchmark results in console [R]
  • Make output from benchmark run more usable [R]
    • write output of scenarios to a results.yml file [R]
    • write benchmark output to comment in pull-request [R]
    • update the same comment in pull-request [S]
  • Integrate benchmarking run into release process [R]
    • write benchmark results to file and commit it [R]
Legend

[R]: required
[S]: stretch goal

Todos

TODO:

  • update comments in scenarios.js[DOCS]

The information about order is incorrect. The ordering doesn't matter as the result of each run is a full cache, full node_modules, and full lockfile. Each next scenario begins with "everything", and must remove the things it doesn't need to complete itself.

  • remove delay function [CHORE]

Was used for testing

  • Properly comment index.js [DOCS]; remove commented code [CHORE]

Description of what a "scenario" is would be helpful for adding new scenarios in the future. Commented code was a previous iteration and can be removed.

  • collect results from each scenario into an object in index.js; write this object into a yaml file in benchmarks/results [FEAT]

Currently just writing the results to stdout. We need to keep track of the results for comparison between releases.

  • update GitHub Action workflow to only run on pull-request [FIX]

Currently the workflow runs on push which is everything: branches and pull-requests. Need to narrow scope to just pull-requests so that we can be sure to be able to write the results to a comment on said pull-request.

  • write "results.yml" into a comment on a pull-request [FEAT]

Using actions/...@v1 should be able to write a comment for the pull-request this action ran for.

  • update release process to include creating "results.yml" and committing it to the repository [FEAT]

A final "results.yml" file should be generated at release time, the "released commit" has an idea of it's benchmark.

Stretch Goal Tasks

Stretch Goal Tasks

  • if pull-request comment exists; update said comment

We want to avoid spamming pull-requests with all the results from each commit pushed to a branch with a pull-request. We should update the same comment; if a comment exists, other write a new comment.

  • compare results from current run with those inside benchmarks/results

Need to handle the edge case where that file doesn't exist.

Pull-Requests

pnpm forked repository which has ci update to commit results to repo upon merge into master branch

pull-request to add ci workflow for pull-request events, and pushes to latest, release-next branches

pull-request for benchmarking work into the npm/benchmark repo

Fix up & improve statusboard

  • Set up a better development environment/workflow
  • Recursively follow paginated responses (currently we're not grabbing all repositories)
  • Add secondary metrics (ex. specific labels/count)
  • Store data snapshots
  • Allow for searching/filtering of projects by column value
  • Clean up styles/formatting
  • Add test coverage to columns
  • Set up GitHub actions to run fetches & builds

[BUG] GitHub Pages builds are failing

What / Why

The GitHub Pages builds are failing
image

When

The last published build was 11/04/2019.

Where

Affects the https://npm.github.io/statusboard/ GitHub Pages website.

How

Current Behavior

It's borked

Expected Behavior

Should deploy a new version of the page each day via the GitHub Actions workflow.

References

  • n/a

Evaluate competitive landscape & determine what features/commands are required vs. nice-to-have

Initial research

  • Read docs and community published material on Yarn workspaces
  • Read docs and community published material on Lerna
  • Build a simplistic local e2e example to manually test/validate/understand basic features

User stories interviews

  • Private/corporate workspaces users meeting with @wesleytodd
  • Node Foundation workspaces users meeting with @boneskull
  • OSS workspaces user meeting with @pedronauck
  • Lerna workspaces support meeting with @evocateur
  • Yarn workspaces support meeting with @arcanis

Fix make test-npm in node

Running make test-npm in node is currently failing. From a quick glance I'm able to recognize that most of these errors are related to recent changes (such as adding snapshot tests but not the snapshots itself, etc).

Migrate & archive npm.community

When we re-open the cli/issues on Tuesday, November 5th, we'd like to automatically generate tickets referencing any remaining #bug threads.

  • Backup forums (.sql has been saved to Dropbox as of Sunday Nov 3rd)
  • Nov 5th: Set the forums to read-only mode
  • Nov 5th: Run Discourse query to capture all open #bug tickets
  • Nov 5th: Run GitHub query to generate issues against the cli referencing their archived counterparts discussions & labeling where possible/appropriate
  • Nov 5th: Add pinned topic to each Discourse forum pointing to the announcement blog post & where to find support

Add new "Registry Outage/Issue" template to npm/cli repository

We need a new kind of issue template which should capture registry related outages/issues so that we can properly clean-up/triage those in a sane manor. Messaging within this template should direct user's to our contact form (ie. https://www.npmjs.com/support) first-and-foremost as it is the more sophisticated/streamlined support vector for these types of issues.

  • Investigate how probot inherits/sets settings
  • Add OSS template to npm/cli

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.