julialang / metadata.jl Goto Github PK

@StefanKarpinski, these docs are out of date now. e.g. Pkg.required() and Pkg.new() seem to be gone.

@timholy
@simonster

Pkg is saying that the current MAT.jl sha1 is referencing a commit that is "not a tree." Not sure if this is due to Fedora 17 packaging git version 1.7.11. Hopefully, Fedora 18 will come with a newer version in 2 days.

--Rob

The JuliaWebRepl package on METADATA uses git://github.com/JuliaLang/JuliaWebRepl.jl.git as its url. That url does not exists. This makes the package unusable.

There is github.com/loladiro/JuliaWebRepl.jl.git , but is it the same thing?

Should the url be changed or the package deleted?

cc: @loladiro @vtjnash

I have no problem using Pkg.clone() to install the package as an unregistered package. But when I tried to do Pkg.add(), I got the error shown below. Any advice or insight would be much appreciated. Thanks!

julia> Pkg.add("RobustStats")
INFO: Initializing package repository /Users/xiaohe/.julia/v0.3
INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl
INFO: Cloning cache of ArrayViews from git://github.com/lindahua/ArrayViews.jl.git
INFO: Cloning cache of BinDeps from git://github.com/JuliaLang/BinDeps.jl.git
INFO: Cloning cache of Blocks from git://github.com/tanmaykm/Blocks.jl.git
INFO: Cloning cache of Cairo from git://github.com/JuliaLang/Cairo.jl.git
INFO: Cloning cache of Color from git://github.com/JuliaLang/Color.jl.git
INFO: Cloning cache of DataArrays from git://github.com/JuliaStats/DataArrays.jl.git
INFO: Cloning cache of DataFrames from git://github.com/JuliaStats/DataFrames.jl.git
INFO: Cloning cache of Distributions from git://github.com/JuliaStats/Distributions.jl.git
INFO: Cloning cache of GLM from git://github.com/JuliaStats/GLM.jl.git
INFO: Cloning cache of GZip from git://github.com/kmsquire/GZip.jl.git
INFO: Cloning cache of Homebrew from git://github.com/JuliaLang/Homebrew.jl.git
INFO: Cloning cache of IniFile from git://github.com/JuliaLang/IniFile.jl.git
INFO: Cloning cache of NumericExtensions from git://github.com/lindahua/NumericExtensions.jl.git
INFO: Cloning cache of PDMats from git://github.com/lindahua/PDMats.jl.git
INFO: Cloning cache of Rmath from git://github.com/dmbates/Rmath.jl.git
INFO: Cloning cache of RobustStats from git://github.com/mrxiaohe/RobustStats.jl.git
INFO: Updating cache of RobustStats...
INFO: Cloning cache of SortingAlgorithms from git://github.com/JuliaLang/SortingAlgorithms.jl.git
INFO: Cloning cache of StatsBase from git://github.com/JuliaStats/StatsBase.jl.git
INFO: Cloning cache of Tk from git://github.com/JuliaLang/Tk.jl.git
INFO: Cloning cache of URIParser from git://github.com/loladiro/URIParser.jl.git
INFO: Cloning cache of Winston from git://github.com/nolta/Winston.jl.git
ERROR: Missing package versions (possible metadata misconfiguration):  RobustStats v(nothing,v"0.0.0") [9909afdb1b596dae093c889b2e7c4e8fdcee3dcc[1:10]]

 in wait at task.jl:273
 in wait at task.jl:188
 in wait at task.jl:48
 in sync_end at task.jl:300
 in add at pkg/entry.jl:308
 in add at pkg/entry.jl:71
 in anonymous at pkg/dir.jl:28
 in cd at file.jl:22
 in cd at pkg/dir.jl:28
 in add at pkg.jl:20

I think a few packages have been added and lots of things have been tagged :)

I'm planning to manually push a change to the TimeSeries METADATA directory to fix a mistake from merging a separate repo. It turns out the merge went ok except I didn't change the module name or the file name where the module is defined from TimeArrays to TimeSeries.

I manually fixed it on my local computer but I can't tag it since the package manager doesn't know what TimeSeries is.

The current registry has only a url entry, causing an installation failure:

julia> Pkg.add("DataArrays")

ERROR: unknown package DataArrays
 in add at pkg/entry.jl:45
 in add at pkg/entry.jl:48
 in anonymous at pkg/dir.jl:25
 in cd at file.jl:22
 in cd at pkg/dir.jl:25
 in add at pkg.jl:19

cc: @johnmyleswhite

There are two two issues I would like to see clarified:

1) sha1 bumping should (almost) always accompany a new version (with exception of version 0.0.0). (Does current policy allow bumping of any x.x.0 version? Should this only be in effect for released versions of Julia?)

Rationale: no matter how careful one is, some new version will break on someone's system, and it's important to allow the user to revert to a previous version. Without bumping the version, this is hard for the naive user.

2. Old versions of packages should (almost) never removed from METADATA.jl

Rationale: Removing a package version from METADATA.jl causes problems when 1) a package depends on a removed version of a package, or 2) a user wants to replicate some results produced with a particular version of a package, and that version is removed.

My impression is that these are policy, or at least should be, but I don't think they're documented, and both have been violated

Any suggestions/objections/refinements? Should I file an additional issue for Pkg2 to support updating packages?

Should the entire NumericFunctors directory be removed from METADATA?

The commit 2c9112d seems to have tried to do some cleanup, but the results seem inconsistent.

The NumericFunctors directory still exists, but without its url file.

cc: @lindahua @ViralBShah

TradeInstrument is a package that downloads financial data and puts them into types of financial instruments. It's the first of a group of four that will include TradeAccount, TradeSignal, TradeSystem.

I'm pretty sure this is an acceptable name. It's inspired by R's FinancialInstrument and quantmod packages.

Does the prefix "Trade" work here?

It is easy to get the REQUIRE file from
https://raw.githubusercontent.com/{{pkg.url}}/{{SHA1-hash}}/REQUIRE (if the package is hosted on github). It would be nice if we could make inconcistencies trigger a travis error.

See: https://groups.google.com/forum/#!topic/julia-dev/9T6SF3L9YUc

It'd be nice if the package list had the date of the last update of each package (i.e. the date of the given commit).

Actually, Julia 0.1.2 finds .julia directory of Julia 0.2.0
So, packages doesn't work fine in one of the installations.

Just noting so it gets fixed.

Pinging @dirk

Did I mess up the REQUIRE?

https://github.com/HarlanH/DataFrames.jl/blob/master/REQUIRE

Stefan, each *.jl package you've added should have a LICENSE file, presumably the MIT license.

Hi! There is a problem when BioSeq.jl is installed from Julia 0.1.2.
Only version 0.0.0 is installed (instead of 0.1.0)

diegozea/BioSeq.jl#8

(like #406)

INFO: Updating cache of GoogleCharts...
ERROR: Missing package versions (possible metadata misconfiguration): GoogleCharts v(nothing,v"0.0.1") [2f9b2b8b0ef80e9cb77fb232cbe7015f4f8503fa[1:10]]

Sometimes Pkg.publish() generates a "Tag MyPackage v0.0.2" PR title and other times "pull-request/2p9g7" type title. The "Tag" message is clearly more informative. How can one control how this message is generated?

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.0 (2013-11-16 23:48 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org release
|__/                   |  x86_64-linux-gnu

julia> Pkg.init()
INFO: Initializing package repository /home/idunning/.julia
INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl

julia> Pkg.register("Mosek")
INFO: Registering Mosek at git://github.com/JuliaOpt/Mosek.jl.git
INFO: Committing METADATA for Mosek

julia> Pkg.tag("Mosek")
INFO: Tagging Mosek v0.0.1
INFO: Committing METADATA for Mosek

julia> Pkg.publish()
Enter passphrase for key '/home/idunning/.ssh/id_rsa': 
INFO: Validating METADATA
ERROR: packages with unsatisfiable requirements found:
    Clustering v0.2.5 – no valid versions exist for package Distance
    DataFrames v0.5.0 – no valid versions exist for package Blocks
    Stats v0.0.1 – no valid versions exist for package StatsBase

 in error at error.jl:21
 in check_metadata at pkg/entry.jl:542
 in publish at pkg/entry.jl:263
 in anonymous at pkg/dir.jl:25
 in cd at file.jl:22
 in cd at pkg/dir.jl:25
 in publish at pkg.jl:53

In between the init() and register() I cloned Mosek manually.

I've been having trouble adding packages (I mentioned this error here). Below is a sample error message I got. Is there any recommended fix for this? Thanks!

EDIT: I am on OS Mountain Lion. My copy of Julia is version 0.2

julia> Pkg.add("GZip")
ERROR: MetadataError("required package VersionSet(\"DataFramesGZip\",[]) 
    has no version compatible with fixed requirements")
    in ReqsStruct at pkg/resolve.jl:67
    in resolve at pkg/resolve.jl:1163
    in _resolve at pkg.jl:343
    in anonymous at no file:163
    in cd at file.jl:25
    in cd_pkgdir at pkg.jl:42
    in add at pkg.jl:143
    in add at pkg.jl:175

The Available Packages document doesn't appear to be updating; packages are missing and version numbers are incorrect.

Version 0.3.0-prerelease+1070 (2014-01-18 19:46 UTC)

julia> Pkg.add("DataFrames")
INFO: Initializing package repository /home/idunning/.julia
INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl
INFO: Cloning cache of Blocks from git://github.com/tanmaykm/Blocks.jl.git
INFO: Cloning cache of DataFrames from git://github.com/JuliaStats/DataFrames.jl.git
INFO: Cloning cache of GZip from git://github.com/kmsquire/GZip.jl.git
INFO: Cloning cache of SortingAlgorithms from git://github.com/JuliaLang/SortingAlgorithms.jl.git
INFO: Cloning cache of StatsBase from git://github.com/JuliaStats/StatsBase.jl.git
INFO: Installing Blocks v0.0.1
INFO: Installing DataFrames v0.3.16
INFO: Installing GZip v0.2.5
INFO: Installing SortingAlgorithms v0.0.1
INFO: Installing StatsBase v0.3.5
INFO: Package database updated

julia> using DataFrames
ERROR: Stats not found
 in require at loading.jl:39
 in include at boot.jl:240
while loading /home/idunning/.julia/DataFrames/src/DataFrames.jl, in expression starting on line 5

Why is it installing DataFrames 0.3.16 but StatsBase 0.3.5?? This seems to be breaking anything dependent on DataFrames.

@johnmyleswhite

The README seems to be a mix of Pkg documentation, a users .julia, how to include packages in METADATA.jl, and technical details. What should it be - or perhaps, what should be here, what should be in the manual.
e.g. the "guidelines" we are mocking here
https://gist.github.com/IainNZ/6086173
should they be in the README here or in the docs?

This package has only two methods that users might find useful, that being yahoo and fred, which basically download data from Yahoo Finance or St. Louis Federal Reserve.

These free data sets can be downloaded with the Quandl package.

The plan for TradeInstrument is to fold it into an unregistered package named TradeModels. The two convenience data reader methods will go there.

(this is a more of public announcement to leave a trail for "whatever happened to package x?")

It seems that there are different versions of the packagelist pages:

• http://docs.julialang.org/en/release-0.2/packages/packagelist/
• http://docs.julialang.org/en/latest/packages/packagelist/

And that these are different. Should they be? I would expect versioning to be handled by the package manager, or at least for all packages compatible with a release to show up in the versioned packagelist (as-in, right now I should be able to write a package that's compatible with release-0.2 and see it show up in that packagelist).

I bring this up because I see a substantial amount of traffic coming from the release-0.2 packagelist, and it looks to be way out of date :(

I'm not sure where this should be filed, but the Profile library doesn't seem to have a REQUIRES file here (it has one in it's repository) and thus fails to pull in the required additional package Options.

METADATA just got back its master branch. I believe it was deleted when the branch based versioning was formalised.

metadata-v2 is still the default branch, so I suppose its no harm done, but this might be confusing to contributors.

@johnmyleswhite

@johnmyleswhite Do you mind if I do this? The packages themselves call using Stats and not using StatsBase, so they are broken, but now that we have a Stats package that just re-exports StatsBase, they could just use that and be fine.

Just to make sure I understand correctly: (I tried looking over various docs/threads, but I'm sure I missed a comment or two and probably out of order anyway)

-Pushing my package version/sha1 updates to the devel branch is the right way to go?

-These changes are then supposed to be folded into the master branch on a regular basis? (though how regular?)

This is under the assumption that when users, on any version of Julia, run Pkg.init()/update(), this is pulling from the master branch of METADATA.jl (please correct me if I'm wrong here). This assumption is driving my questions because for a user to see updates to my package, I would need to push an update to devel and then wait until it's merged to master for them to be able to Pkg.update() and see the changes appropriate to their julia version.

I've got a repo (https://github.com/WestleyArgentum/GeneticAlgorithms.jl) tracked here, but I have to switch the url to point at a different repo (https://github.com/forio/GeneticAlgorithms.jl) which is now the root.

What's the best practice here? Should I tag a new version and change the url at the same time? Or, if the sha could be the same in both cases, should I just switch the url?

Related to #347, but I didn't see this specifically addressed.

I am not sure what is the best way to do this (either in METADATA.jl or in each individual package repository), but it would be nice for all package tests to be run through travis-ci. At this stage, we can do it with the latest julia build - but eventually, packages can choose to depend on a specific released version of julia.

Pinging @staticfloat and @loladiro for ideas on continuous integration for packages.

I was getting this error because I had an extra space at the end of my sha1.

I've been thinking about how to deprecate packages a bit more smoothly. Let me throw out an idea:

I'm not sure what happens to Pkg if you flat out nuke a whole package in METADATA, but what thing you could do is set the maximum Julia version for all versions of the to-be-deprecated package to the current release.

E.g. Trie.jl is being merged into DataStructures. We can set the maximum julia version for Trie to julia 0.2, and then when julia 0.3 (or 0.4?) comes out we can safely delete it (presumably).

I ran into an issue where some of my packages would download through Pkg.add(), but only one would not: NumericExtensions.jl.git on ubuntu.

The fix for me was to modify ~/.julia/Metadata/NumericExtensions.url and change the url prefix from https:// to git://

I don't know the exact reason for why git:// works better for me - probably a proxy issue, but I do know the METADATA urls are inconsistent: I counted and there are 199 packages using git:// and 16 using https://

I was running my PackageEvaluator script and it died hard on MinimalPerfectHashes, which I confirmed is broken with Pkg.add

INFO: Updating cache of MinimalPerfectHashes...
ERROR: Missing package versions (possible metadata misconfiguration):  MinimalPerfectHashes v(nothing,v"0.0.0") [c2223cc0288f78729f3dc149b4026410b241bba8[1:10]]

 in error at error.jl:21
 in resolve at pkg/entry.jl:304
 in resolve at pkg/entry.jl:271
 in edit at pkg/entry.jl:15
 in add at pkg/entry.jl:22
 in add at pkg/entry.jl:26
 in anonymous at pkg/dir.jl:25
 in cd at file.jl:22
 in cd at pkg/dir.jl:25
 in add at pkg.jl:18
 in process_options at client.jl:230
 in _start at client.jl:371

On the Windows 0.1.2 binary, I get the following error message when running Pkg.update()

julia> Pkg.update()
Already up-to-date.
warning: LF will be replaced by CRLF in .gitmodules.
The file will have its original line endings in your working directory.
ERROR: no method isless(VersionNumber,ASCIIString)
in _resolve at pkg.jl:277
in anonymous at no file:524
in cd at file.jl:39
in cd_pkgdir at pkg.jl:34
in update at pkg.jl:499

I notice that the Mathematica package still has the potentially legally problematic name Mathematica. What should we do about that?

While GLPKSolverInterface has been renamed to GLPKMathProgInterface , the url file in the package still refers to GLPKSolverInterface git://github.com/carlobaldassi/GLPKSolverInterface.jl.git

cf: https://github.com/JuliaLang/METADATA.jl/blob/2f7c24b6c3efeabf68ce014aa6cf4bb30306d2dd/GLPKMathProgInterface/url

cc: @carlobaldassi

What do you guys think about splitting #482 into this and JuliaLang/julia#5302, then closing it?

I accidentally bumped Nettle to 0.1.4 for both Julia 0.3 and 0.2, but that change introduced new syntax that is only available in 0.3. I want to roll Nettle.jl back to 0.1.3 on the 0.2 branch of METADATA, what's the proper way to do this? I don't really want to create a 0.1.5 Nettle release on the 0.2 branch of METADATA that has an older gitsha, but I will if I must. Ideally, I'd like to just delete the 0.1.4 release.

The package has been and probably should continue to be a "toolkit for working with series data (including time series) in DataFrames and DataArrays"

There is a separate package that I'm working on called Series.jl that seeks to implement a specific type (SeriesPair). The new DataSeries package would require it and provide a path between Array{SeriesPair{T,V},1} and the DataFrames/Arrays framework.

JuliaStats/TimeSeries.jl#46

The thought here is that Series is more general than TimeSeries, and prepending Series with Data suggests some affiliation with the DataFrames/Arrays framework.

The only changes to the existing package besides the name change would be to REQUIRE Series and to include some conversion methods. The existing methods that are dispatched on DataFrames and DataArrays would remain unchanged.

The listing at METADATA.jl is pretty useless for browsing, because

• It doesn't link to the package pages (at least, not in a clickable way) or display the documentation for the packages.
• The number of packages is large enough, and the names obscure enough, that you really want to be able to browse by tags embedded in the packages, with some conventions for common tags, not to mention search them.

I wanted to create an issue for this, and METADATA.jl seemed like roughly the right place.

After the discussion #105 I want to rename Itostat.jl to SDE.jl. I will move the package on my repo, how to proceed here? Thank you.

He maintains Quaternions, I think we should give him access...

Here is what I got doing package update:

julia> Pkg.update()
Branch devel set up to track remote branch devel from origin.
Already up-to-date.
Already up-to-date.
Already up-to-date.
Already up-to-date.
Already up-to-date.
Already up-to-date.
ERROR: unknown dependency for HTTPClient: LibCURL
 in dependencies at pkg/metadata.jl:155
 in gather_repository_data at pkg.jl:274
 in _resolve at pkg.jl:341
 in anonymous at no file:625
 in cd at file.jl:25
 in cd_pkgdir at pkg.jl:42
 in update at pkg.jl:596

I installed neither HTTPClient nor LibCURL in my system.

Are there any special considerations I should be aware of before I do this?
JuliaQuant/FinancialBlotter.jl#16

Now that Pkg is sufficiently functional to create the necessary pull requests, I don't see a good reason any more to give package maintainers commit access. Maybe it's time to limit again (I'm thinking to people with commit access to JuliaLang/julia). Thoughts? @JuliaLang/packagers @JuliaLang/contributors