Possible stages for Compass tool:

1. Tools level (e.g. broad overview of available tools and respective maintainers in a single database with linked websites and nice label navigation) <-- already available, starting directly from the second point. Just requires indexing of existing projects
2. Github project/implementations level (e.g. detailed view of projects, developed label and navigation system + crawler for automatic indexing of existing and new projects)
3. Find projects using PL projects dependencies -> have an understanding of final users/consumers

Project stages readiness:

1. MVP
   • Web UI for viewing and filtering resources
   • Database for storing metadata
   • Automating labelling of resources
   • Github crawler + Web-Spider
   • DB design for our use-case (object, label, social fingerprint, link)
2. Next steps
   • Package manager crawler
   • Web on-page crawler
3. Long-term plan
   • Abstraction layer crawler (identifies where to crawl/what to crawl)
   • Social network crawler

Fleek only provides 250 build minutes for free. GitHub Actions allows unlimited build minutes for opensource repos. We should build with github actions, and publish to a branch/folder in a way that fleek can just grab the assets without needing any time-consuming build step.

Currently, https://aged-block-7445.on.fleek.co/ is showing the default create react app landing page. This issue can be closed once our page is updated and reflects that it's for our PL compass project.

• Run ecosystem-research project locally for ipfs org, start repo_miner process.
• display table of all repos' names in ipfs org

Schedule meeting with David Carter to discuss existing solutions & problems. What features/solutions would David propose/want?

we need to put together our UI after coming up with a few simple user stories #13

Currently, when github API gives back a 403, we completely STOP API requests.. We need to automatically resume based on the timestamp given by the x-ratelimit-reset response header.

We need to collect user stories so we can map out our feature plan:

• As a PL employee, I want to be able to see all of the repositories I am responsible for
• As a member of a PL team, I want to be able to see all of the repositories my team is responsible for
• As a PL employee interested in a github repo, I want to be able to see who is the owner/DRI for that github repo.

Initial motivation:

• response time to issue 30 days
• ecosystem dashboard -> as a single source of tracking of all issues -> tracking stuff across the whole organization
• tracking the response rate and having additional metrics on issues created

Next functionality

1. Understanding how different projects work together
2. Packages linking and tracking libraries
3. Triaging issues of projects
4. Understanding how people are using all this open-source stuff (coz no tracking code in open source stuff)
5. Tracking the activity of projects using our libraries
6. Automatic crawler through GitHub
7. Tracking Hackathon submissions for quality of using our tech (tracking contributors as well)
8. General issue triaging

The subdomain should support https and direct to the same site as the fleek hosted subdomain https://aged-block-7445.on.fleek.co/

Check out instructions at https://blog.fleek.co/posts/getting-started

KanBan board is at https://github.com/SgtPooki/PL-Compass/projects/1

Similar projects or efforts:

• ecosystem.ipfs.io
• ecosystem.filecoin.io
• https://storageapi.fleek.co/laurentsenta-team-bucket/carto-dist/index.html
• https://github.com/protocol, https://github.com/ipfs, ... + ideally pull info from public repos of all participants of PL
• https://dnsdumpster.com/ ; <-- Search ipfs.io, filecoin.io, etc.
• https://hackathons.filecoin.io/
• Airtable PL (many of them)
• https://repocheck.com/
• Notion PL (many of them) <- Is it needed?
• Gitbook PL (many of them) <- Is it needed?
• https://livablesoftware.com/tools-mine-analyze-github-git-software-data/
• https://www.reddit.com/r/MachineLearning/comments/ssfijc/p_database_for_ai_visualize_versioncontrol/
• PL twitter accounts
• ghcrawler tool from microsoft
• Tableland - Textile

This issue should maintain the list of potential crawling sources. i.e. root URLs or source-types in a graph of dependent PL things to base crawling from.

• https://pl-strflt.notion.site/Latest-Updates-448a9502996949369fdf7539dd4b6bc5 - Public EngRes SitRep index
• https://ecosystem-research.herokuapp.com/repositories
• https://ecosystem-research.herokuapp.com/collabs/repositories
• https://ecosystem-research.herokuapp.com/community/repositories

Todo items

• fix fleek deploy (contains sorting updates)
• dedupe repos
• add contributors column with link to collaborators page on github to repo table
• add top contributors column that lists top 3 contributors:
  • https://api.github.com/repos/scala/scala/contributors?q=contributions&order=desc
  • https://stackoverflow.com/questions/48874412/github-api-how-to-get-the-top-contributors-sorted-for-a-given-repository
• organization filtering to show only ipfs, or all: https://react-table.tanstack.com/docs/api/useFilters
• link to repo details on ecosystem-dashboard page (new column)
• add contributors table & fetching (from ecosystem-dashboard)
• Create contributor detail page that shows all PL repos they interact with (spoof data at first), including twitter handle, email, etc.. (konstantine to provide mock data)