LIBRARIES page about circuitpython-org HOT 28 CLOSED

@jwcooper @tylerdcooper I'd like work together with you to move forward with this as soon as is reasonable. I understand there may be other priorities.

from circuitpython-org.

@kattni I can start on it this week, I hope.

For the script component (if possible), I'd like to follow the same process we use for the files.json (https://github.com/adafruit/circuitpython-org/blob/master/_data/files.json) circuitpython release process. For libraries, we could simply add a _data/libraries.json file that is pushed to this repository on each release. Then, we can just pull in that data in jekyll, etc.

from circuitpython-org.

@jwcooper,

Using the same process as files.json shouldn't be an issue. The idea was to have the script in adabot anyway. It will most likely run in a Travis cron job, vs the release builds on circuitpython. Submitting PRs can be the same...

I'll try and get a schema together soon, though it honestly may just come straight out of GitHub's API replies.

from circuitpython-org.

@jwcooper,

How does the below schema work for you all?

Truncated Raw Example:

[
  {
    "library_updates": {
      "new": {
        "Adafruit_CircuitPython_LIFX": "https://api.github.com/repos/adafruit/Adafruit_CircuitPython_LIFX"
      },
      "updated": {
        "Adafruit_CircuitPython_INA260": "https://api.github.com/repos/adafruit/Adafruit_CircuitPython_INA260",
        "Adafruit_CircuitPython_RA8875": "https://api.github.com/repos/adafruit/Adafruit_CircuitPython_RA8875"
      }
    },
    "open_issues": {
      "Adafruit_CircuitPython_SK9822": [
        "https://api.github.com/repos/adafruit/Adafruit_CircuitPython_SK9822/issues/1"
      ],
      "Adafruit_CircuitPython_SI4713": [
        "https://api.github.com/repos/adafruit/Adafruit_CircuitPython_SI4713/issues/2"
      ],
      "Adafruit_CircuitPython_VEML6070": [
        "https://api.github.com/repos/adafruit/Adafruit_CircuitPython_VEML6070/issues/7"
      ],
      "Adafruit_CircuitPython_BluefruitSPI": [
        "https://api.github.com/repos/adafruit/Adafruit_CircuitPython_BluefruitSPI/issues/9",
        "https://api.github.com/repos/adafruit/Adafruit_CircuitPython_BluefruitSPI/issues/8",
        "https://api.github.com/repos/adafruit/Adafruit_CircuitPython_BluefruitSPI/issues/4"
      ]
    }
  }
]

I can change it up if need be. I have more experience reading JSON than encoding...

from circuitpython-org.

Ha! And in true fashion, now I notice that its using the API URL for library updates. Easy fix.

from circuitpython-org.

The formatting looks great other than maybe dropping the outer array (unless there will be multiple library_updates and open_issues sections)?

Should be quite easy to get that displayed with liquid. I'll use that as the test schema, and let you know if I need any other changes.

Thanks!

from circuitpython-org.

I don't imagine there will be multiple update/issues section. Will drop the outer array.

from circuitpython-org.

I'm going to get an initial PR for /_data/libraries.json in tonight, hopefully. That will at least start things.

But, as I'm constructing the adabot auto-update-majik part, I have a question. This kind of ties into my lack of knowledge with the plan for automating files.json updates.

@kattni, @tannewt:
Should I plan to utilize adabot's fork for cloning, updating, and then PRing? Or would direct commits be preferred?

The main difference I see between the two JSON updates is:

• files.json is based on core releases, and will presumably be run by the core's Travis on those triggers. Or, will it be more schedule based to update the download counts?
• libraries.json will be a cron based update. The cron can be set up on any of the repos. The "easiest" would be to simply add it to the current adabot daily cron (can still be limited to weekly inside the python script).

from circuitpython-org.

@kattni, @sommersoft
Should we display the open issues in the contributing section since they are included in the libraries.json? If so, it might be useful if we can include the title in the libraries.json with the url (I'm assuming this will be in the API already for github, if not, no big deal).

Also, can we add the release bundle download links in the libraries.json? I can pull them from the API, if we need to, but we're already building this json file, so it may make sense to include it (faster page loads as well, if we don't' have to wait for an API request/response in js).

Possible updated example file:

{
  "releases": [
    {
      "2.x": "https://github.com/adafruit/Adafruit_CircuitPython_Bundle/releases/download/20190414/adafruit-circuitpython-bundle-2.x-mpy-20190414.zip"
    },
    {
      "3.x": "https://github.com/adafruit/Adafruit_CircuitPython_Bundle/releases/download/20190414/adafruit-circuitpython-bundle-3.x-mpy-20190414.zip"
    },
    {
      "4.x": "https://github.com/adafruit/Adafruit_CircuitPython_Bundle/releases/download/20190414/adafruit-circuitpython-bundle-4.x-mpy-20190414.zip"
    }
  ],
  "examples": [
    "https://github.com/adafruit/Adafruit_CircuitPython_Bundle/releases/download/20190414/adafruit-circuitpython-bundle-examples-20190414.zip"
  ],
  "library_updates": {
    "new": {
      "Adafruit_CircuitPython_LIFX": "https://github.com/adafruit/Adafruit_CircuitPython_LIFX"
    },
    "updated": {
      "Adafruit_CircuitPython_INA260": "https://github.com/adafruit/Adafruit_CircuitPython_INA260",
      "Adafruit_CircuitPython_RA8875": "https://github.com/adafruit/Adafruit_CircuitPython_RA8875",
      "Adafruit_CircuitPython_CharLCD": "https://github.com/adafruit/Adafruit_CircuitPython_CharLCD",
      "Adafruit_CircuitPython_FeatherWing": "https://github.com/adafruit/Adafruit_CircuitPython_FeatherWing"
    }
  },
  "open_issues": {
    "Adafruit_CircuitPython_BluefruitSPI": [
      {
        "https://github.com/adafruit/Adafruit_CircuitPython_BluefruitSPI/issues/9": "read_packet needs to check for leftover data in self._buffer",
        "https://github.com/adafruit/Adafruit_CircuitPython_BluefruitSPI/issues/8": "ATI command works ok but errors when debug is used - gives Error (id:0xa180)",
        "https://github.com/adafruit/Adafruit_CircuitPython_BluefruitSPI/issues/4": "HID example code"
      }
    ],
    "Adafruit_CircuitPython_Hue": [
      {
        "https://github.com/adafruit/Adafruit_CircuitPython_Hue/issues/2": "PyPi Setup"
      }
    ]
  }
}

from circuitpython-org.

@jwcooper The contributing section will only contain a small blurb about contributing and a link to a second page. This is intentional - the library report page will be a lot of information and I want it separate from the main page which is meant to be a basic introduction to/download page for CircuitPython libraries. cpy.org/libraries will have a link to cpy.org/contributing or cpy.org/libraries/contributing which will contain the issues/PRs/repo issues.

from circuitpython-org.

@kattni That sounds good. I can move that functionality to /libraries/contributing. We'll probably want the issue titles for that page in the json, since we could re-use that functionality on the contributing, if possible.

from circuitpython-org.

can include the [issue] title in the libraries.json with the url

Yeah, that's easy enough. I can add that tonight.

Also, can we add the release bundle download links in the libraries.json?

The planned schedule to run the script for libraries.json is once a week. The Bundle releases are done daily (if there is an update). It may be better to JS with the API, or simply point to the release page https://github.com/adafruit/Adafruit_CircuitPython_Bundle/releases/latest.

from circuitpython-org.

@sommersoft Sounds good. I'll likely hard link the /latest page as well as build the release bundle zip links from the api with js.

from circuitpython-org.

@sommersoft Here is the latest version of the libraries.json file. The PR is nearly ready to go, just have a couple more things to add in.

https://github.com/adafruit/circuitpython-org/blob/79c6a4fdc566c659dbcfd2db030d3ce20028ba15/_data/libraries.json

from circuitpython-org.

@kattni, @sommersoft Is the repo level issues section automated? If so, I believe we'd need that in the libraries.json file as well.

adafruit/circuitpython#1246

from circuitpython-org.

@jwcooper Yes, it will need to be there. I was unclear with my expectation so @sommersoft didn't know the entirety of what I wanted for the contributions page. We discussed it last night and he's sorting how he wants to do it since it now resembles the current adabot script more closely. Sommersoft can explain better what his plans are now.

from circuitpython-org.

Sounds good, thank you @kattni!

from circuitpython-org.

@jwcooper,

Here's a current iteration (i reordered it a little): https://github.com/sommersoft/circuitpython-org/blob/new_lib_json/_data/libraries.json

I can push it to your PR if you'd like. Well, once I remember how to do it. 😄

from circuitpython-org.

@sommersoft At first glance, this looks great. I can copy it over to my fork and integrate the new components that were added.

from circuitpython-org.

@kattni, @sommersoft I think we just need the count for the pull requests merged at this point (based on latest PR review).

from circuitpython-org.

@jwcooper I believe you are correct! There is one more change that needs to be made to the Contributing page, but that's on our end, not yours (change needs to be made to the script).

from circuitpython-org.

Yeah, it'll just be another key. How does {"merged_pr_count": "n"} sound, @jwcooper?

from circuitpython-org.

@sommersoft That works for me.

from circuitpython-org.

@jwcooper We've updated the script with the rest of the necessary information to complete the libraries page.

I want to find out, how does the page get updated? Is it triggered by the script updating? Or is there something else to it?

I ask because I am considering having the authors/reviewers/newlibs/updatedlibs lists be updated daily as a rolling weekly list. I want to make sure that we're handling that appropriately if we decide to do it - I think it's simply a change on our end for when the script updates, but I don't know that for certain.

from circuitpython-org.

The site rebuilds on any commit to the master branch of this repository. Whenever the script is updated and the PR is merged, it should trigger a re-build.

I'll merge in the latest changes to the libraries json.

from circuitpython-org.

@jwcooper Are you okay with the Contributors and New/Updated Libraries lists being updated daily? The Contributing page would be updated weekly. I ask because updating it daily would increase the amount of activity on the repo, and you seem most likely to notice. It won't require any changes on your end, it's something we'd do, but it would mean daily builds.

from circuitpython-org.

I think that's fine as long as it's automated.

from circuitpython-org.

This was completed and merged.

from circuitpython-org.