Packages are hard to browse or search about metadata.jl HOT 31 CLOSED

This is probably the best resource:
http://docs.julialang.org/en/latest/packages/packagelist/

from metadata.jl.

Idea: could someone with owner privileges to the METADATA.jl update the website in the project description at the top with the package listing URL @JeffBezanson posted? /cc @StefanKarpinski

from metadata.jl.

The obvious problem with that list is that it is not up-to-date. We need to have a way to autogenerate it from metadata within the packages.

The other (related) problem is that it is not scalable to large numbers of packages, which will demand a kind of tagging and searching system. Which also requires metadata in the packages.

It is better to implement a scalable system of package documentation and tagging metadata now, when there are less than 100 packages to fix, than when you have 5000 packages.

from metadata.jl.

The package listing is auto-generated, and uses the description from the github description for now. What we need is an automatic way of running the script daily and updating the website. (Cc: @aviks)

Tagging has been talked about before, and is certainly necessary to have. I wonder if we can also structure the way we generate our package list so that it is search-engine friendly. Ideally a google search for "julia packagename" should be good enough to find a package, and would save us the headache of building something to search for packages.

We should also integrate search facilities into apropos() and help().

from metadata.jl.

Why not just have a file called tags in each package's directory of METADATA.jl that lists, one per line, a tag for that package? For example, METADATA.jl/DataFrames could have a file tags that lists statistics, data, etc. If the tag parser has a list of valid tags built into it, it should be trivial to use this file to generate a consistent classification of packages.

from metadata.jl.

We should really just have a per package metadata file, as we will want to add other package related metadata in the future.

from metadata.jl.

Yes, I've been arguing for that for a long time. I really doubt we're going to do better than an extension of something like R's DESCRIPTION file: http://cran.r-project.org/doc/manuals/R-exts.html#The-DESCRIPTION-file

from metadata.jl.

[ This is also discussed on Julia's issues: https://github.com/JuliaLang/julia/issues/1959 ]

from metadata.jl.

The package list is generated by a single julia script. I try and update it every few days manually. It is easy to set it up as a cron if there is a machine available to run it on. However, you'll need to figure out github keys management, especially if you want to script to auto-commit for you. [EDIT] ie, someone with commit rights needs to provide his github tokens, and trust the machine this is running on.

from metadata.jl.

As for links to docs, if you add an url to the github package description, it will display that link under the "More Info" header. See Ito or Gadfly for examples.

from metadata.jl.

The script should probably be checked into julia/contrib.

Also, at some point, it would be nice to have one page per package, where we pull in things like an introduction, tags, examples, screenshots, etc. This will also help indexing in search engines.

from metadata.jl.

Can we just have the package listing be a separate repo so that we can setup a commit hook on julia.mit.edu that automatically regenerates it and pushes it to github without to much noise in JuliaLang/julia?

from metadata.jl.

That seems like the best thing--easy enough to set up a webhook to trigger it, and will keep commit noise out of the main Julia repository.

from metadata.jl.

Yes. I have been wanting to have packages.julialang.org and this would be a good time to do it.

from metadata.jl.

@ViralBShah The script has been in doc/

https://github.com/JuliaLang/julia/blob/master/doc/listpkg.jl

from metadata.jl.

@IainNZ has done a great job with his package listing. I say we alias the site from packages.julialang.org and close this.

from metadata.jl.

That'd make me happy. I'd make the more developer focussed filters like licenses and test status hidden by default but otherwise its good to go

from metadata.jl.

We still need some kind of tags/hierarchy metadata for the packages, but maybe that's more for JuliaLang/julia#1959

from metadata.jl.

How about just pkg.julialang.org?

from metadata.jl.

That'd be even better

from metadata.jl.

An alias from packages.julialang.org would be nice for discoverability.

from metadata.jl.

Ok, what do I make the CNAME to?

from metadata.jl.

Ok, done. I'll make packages.julialang.org redirect to it as well.

from metadata.jl.

So, I have an idea, @IainNZ:

1. we make this the way to find packages and
2. we by default only show packages with passing tests

I feel like that would be a pretty effective, sneaky way to encourage both the usage and creation of packages with tests.

from metadata.jl.

Eventually, we could have a minimum coverage requirement too.

from metadata.jl.

So, for example, PyCall has a failing test because of a problem with Python 3 that I can't reproduce yet (JuliaPy/PyCall.jl#75), and which has little practical impact because most people use Python 2. If you don't show packages with failing tests, then I will simply disable that test. Hence you will get less test coverage, not more.

from metadata.jl.

Some packages (like AWS and ClusterManagers) have tests that depend on external resources and credentials.

If there is a requirement like "with passing tests", these packages will need to have a dummy runtests.jl

from metadata.jl.

I think everything should be shown.
I'm on holiday with no computer so test results will be stale

from metadata.jl.

I'll make packages.julialang.org redirect to it as well.

FYI, that one seems to be returning a 404.

from metadata.jl.

For some reason my hosting/domain provider (1&1) won't let me redirect that subdomain to pkg.julialang.org – I have no idea why.

from metadata.jl.

I think we are good to close now. Kudos to @IainNZ

from metadata.jl.