The staging site and the API are currently returning an 'Internal Server Error', or Error 500.

This is causing MultiMC to fail as newer files are not found, causing download failures/json parsing failures.

Last entry in
https://cursemeta.dries007.net/history/index.json

"timestamp_human": "2018-02-23 21:58:27 UTC",
"timestamp": 1519423107,
"history": [
....
1519419991,
1519423107
]
}

So something may need kicking or Curse changed something again.

Texture packs added to modpacks result in 502 Bad Gateway

Example: https://cursemeta.dries007.net/270078/2457838.json

Modpack: https://minecraft.curseforge.com/projects/all-the-mods-3/files

Link: https://cursemeta.dries007.net/293425/2697986.json

Project: https://minecraft.curseforge.com/projects/alchemistry

File: https://minecraft.curseforge.com/projects/alchemistry/files/2697986

This URL was requested by multimc, and this issue has made it fail to download/install a pack I'm working on.

Hello, I've been looking into using this for MMD's curse info bot. Basically you give the bot a username, it will tell you how many downloads that user has, and link to their top 10 projects. I was wondering if you could add a few things to help us out.

This is what the current output looks like.

The main thing missing is the monthly downloads data for a specific project. Im not sure if you have that data available when you scrape it, but it would be nice to keep that feature. Some other feature requests would be a way to get all of the projects from a username. Grabbing the mods.json every few hours and processing it ourselves would allow for this, however it would make the bot less ram/storage efficient than it is now. I noticed that you can search using a username but scraping this has similar performance to what we have now.

Another thing we have been planning to implement, is a system that will announce when a new mod has been put on Curse. This is probably beyond your scope, but do you think it would be possible to have a way to get all mods that were created since the last time the data was scraped?

Thanks

The following pages return a stacktrace on top of a json error.

• https://cursemeta.dries007.net/228525/2534543.json
• https://cursemeta.dries007.net/281791/2534481.json

See also https://paste.ee/p/zjuT9 for the full error.

I noticed that the user profile has the ASPX extension appended when it shouldn't. This is easy to fix on the receiving end, but still a bit weird.

  "Authors": [
    {
      "Name": "NuclearHex",
      "Url": "https://mods.curse.com/members/NuclearHex.aspx"
    }
  ],

This leads to issues like these:
MultiMC/Launcher#2340 (comment)

Specific request:
https://cursemeta.dries007.net/278645/2514913.json

It should contain something like this to be usable:

"_Project": {
    "PackageType": "singlefile",
    "Path": "resourcepacks"
},

The CORS settings for https://staging_cursemeta.dries007.net prevent accessing the API from other domains in a browser. This makes it not possible to use the CurseMeta API in a front-end web app.

The API should support basic query syntax, for example:

GET /.json?ver=1.12&fields=name,versions,url

[
    {
        "name": "Project that supports 1.12",
        "versions": ["1.12", "1.0"],
        "url": "http://blah.blah"
    }
]

There should also be endpoints for searching, eg;

GET /search.json?q=tinkers

[
    {
        // Tinkers Construct
    },
    {
        // IDK something else with tinkers in the name/desc
    }
]

(Will continue updating this issue)

Appears that cursemeta.dries007.net is down.

While running a serverstarter script, it ran into this error:
java.io.IOException: Request to https://cursemeta.dries007.net/410127/3548273.json was not successful

The game version for releases is is missing an R in the key. Just spent a good amount of time trying to work out why I kept getting issues with undefined values lol.

		{
			"FileType": "release",
			"GameVesion": "1.4.2",
			"ProjectFileID": 642755
		},

Pretty basic use-case of resolving a slug to an mod id. No idea how to get it to work.

I tried minecraft/storage-drawers using GET & POST in various different forms, but I always just get 200 null back (yay)

Apparently something super bad happened and the server is responding with internal errors:
https://cursemeta.dries007.net/

See: MultiMC/Launcher#2283

It would be really nice to have a way to get server version of a modpack (imagine a self-updating modded server). Right now the API lists only client-version files.

Just my thoughts: AddonFile model has IsCompatibleWithClient property. Could be the thing.

You can watch this issue if you want to know if/when sections are finished.

V2 is now live, under beta label. A live-edit staging environment can also be found on staging_cursemeta.dries007.net.

With the v2 python rewrite, all the previous aggregation/stats/history functionality was lost.
Bringing all of it back will take time, but will be done over time.

Feel free to comment and note what you need. New functionality may also be requested here.

ToDo snapshot as of March 14th

• Referral link: https://workflowy.com/invite/4dfc7739.lnx (Gives me more free space to add stuff!)
• Keep historic DL counts
  • PopularityScores?
  • The sooner the better, less data loss
  • I have the historic data still (Write import script)
  • API
    • Diff downloads (from-to timestamp)
    • Get total downloads per game/type
  • Keep up to date with feeds only?
  • Fixed interval data points or detect a change on every update?
    • I think more fixed interval, probably less data. Maybe even only do changed data?
• Status page
  • Internal statistics such as
    • Threads running (?)
    • Uptime global/thread (?)
    • Requests served global/thread (?)
    • Task queue size
    • Server load
    • Approx bandwidth used
  • Any other stats?
  • Should this be public?
• Web UI
  • Preferably pretty version of the old thing
    "https://cursemeta2.dries007.net/search.html#mods-287538"
  • Per game, obviously.
  • Per category too, or allow selection of that? (mod, pack, world, addon, ...)
  • Per game version
  • Proper 'fuzzy' search! (partial client side maybe?)
• DB query API
  • Direct or Post processed output?
  • Options
    • GraphQL http://graphql.org/
      " http://docs.graphene-python.org/projects/sqlalchemy/en/latest/tutorial/"
      • Pro: Big, known & supported
    • Flask-Restless (https://flask-restless.readthedocs.io/en/stable/searchformat.html)
      • Pro: Direct integration with DB models
      • Con: Relies on internal DB structure
  • File Download Endpoint (redirect?) https://gist.github.com/NikkyAI/ce29d9ca12fee410c91add7b17bba31a
• Uniform output API (Post process everything)
  • Fix typos (GameVersion is missing the R in some places)
  • Add Addon data to File objects (so less requests are needed)
  • Selective return data
    • Jenkins API like ?
  • Re-processed (or similar but DIY) raw feeds, preferably compressed.
• Statistics pages (Graphs & tables!)
  • Old version: https://cursemeta2.dries007.net/stats/graphs.html
  • Maybe take some inspiration from MCStats.org ?
  • Stuff to stat (Global & per game, type, author(primary & all), gameversion)
    • nr of games
    • nr of addons
    • nr of files
    • nr of authors
    • Downloads
    • Filesizes
    • PopularityScores
    • GameVersions
    • Newest files
    • Newest addons
    • Newest authors (?)
    • Highest downloads 24h/7d/1m/
• Separate out UI and data
  • Now the analysis of the response gets triggered by the CurseClient module, but is processed in the Frontend module (as part of tasks)
  • Make a more modularized system (would also support better tests)
• File downloads (on the server)
  • This task must be locked! (only 1 dl / analyze at the same time)
  • Get and store correct download URL
    • Store that instead of the original one, the original is always this:
      "https://files.forgecdn.net/files/${FILEID/1000}/${FILEID%1000}/${FILENAME}"
  • Get the filesize
  • Hash the file
    • SHA256 ?
  • Crazy idea: If it's a mod, analyze it, if it's a pack, parse the manifest?
• Archive
  • We are not in the business of archiving, but supporting others is OK.
  • Write a script people can use to keep an archive (preferably with options).
• Crazy idea: Draw a huge (interactive?) graph of the authors - addons relations + file dependencies?
• Write the history of CurseMeta somewhere, because it's interesting.
  • https://gist.github.com/dries007/10d8dc05a6fc5d1e700404cdd4446d21
• Write tests (meh)
  • Get a list of troubling addon/file ids to test on (weird characters, no files, etc)
• Pick a theme song.
  • Sonic X Theme Song ''Gotta Go Fast'' https://www.youtube.com/watch?v=yfGjLIPRgDA ?

Some Project+FileID combinations MultiMC is trying to resolve via CurseMeta are returning host errors. It looks like it might be an intermittent issue, my guess is that the result sometimes manages to make it to the cache but that the backend requests are timing out. Sorry for hammering the heck out of the server 😅

Examples

• https://cursemeta.dries007.net/242436/3738133.json
• https://cursemeta.dries007.net/245211/3751424.json
• https://cursemeta.dries007.net/377448/3581611.json
• https://cursemeta.dries007.net/401955/3742705.json