aameen-tulip / at-node-nix Goto Github PK

Functions sensitive to config should require the relevant arg instead.

This doesn't currently break anything but it makes writing overrides error prone and it defeats the purpose of it existing for "convenience".

The branch mk-src-ent has made a big push on this, and aims to create a formal spec for pkgEnt. This will help land on a common standard for package definitions to be shared in overlays across projects.

While pkgSet pipelines necessarily need to be more flexible, a set of interfaces implemented by different backends should help scaling small clusters centered around package locks and workspaces to be compatible with larger hierarchical sets with multiple versions like we have in flocoPackages.

addFlocoPackage[s] ( overlay generator ), getFlocoPackage ( lookup by exact ident+version ), resolveFlocoPackages ( filters by semver ), and similar interfaces can abstract a packument VInfo hierarchy vs. a keyed set. Definitions for these already exist for the keyed metaSet backend.

Certain NPM tarballs contain non-standard GZ directory entries with bogus permissions set.

If you try to unzip these naively you will create a directory owned by root, and then fail with a permissions error attempting to add files to the directory.

We deal with this issue in unpackSafe, and there's several other examples of dealing with these tarballs using tar options that ignore permission settings - however we cannot set flags for the builtin fetchers.

This issue only effects ~1/3,000 tarballs but it's a real PITA because it throws a wrench in some impure metadata collection routines.

[email protected] is an example. chalk and fsevents are also effected at certain versions.

The arg handlers and typeOfEntry routine are due for a revamp.

They predate the use of wrapped builtins which are a much more elegant solution to abstracting the fetch routines.

The typeOfEntry routine also predates builtins.fetchTree { type = "file"; ... } and alignment for handling needsUnpack would be cleaner than the current system.

This effort is already in progress under types/npm-lock.nix but integration with existing builders and metaEnt constructors needs to worked through and tested.

This may have been intentional since in theory we can collect that info when deserializing; but when reading back using metaSetFromSerial those values aren't being restored.

Either write it or read it back when deserializing.

Migration to typed fetched and pkgEnt:source should be reflected in pacote recipe, mostly because it is recommended as an example to learn from.

Alternatively ( preferred at the moment ) it makes more sense to cut flocoFetch and flocoUnpack from this build but still satisfy the fetched and pkgEnt:source records. From both a maintenance perspective and as an example to learn from - inlining the full abstractions is overkill.

Since libfetch was refactored I noticed that path type entries which use relative paths in locks are omitted from __serial.

I honestly can't remember if the old routines behaved this way, I think emitting __meta.lockDir might have been what allowed those to work; but in any case the __serial function for the fetchInfo object itself should be handling this.

You could avoid stashing cached metadata and retain purity by recording only the _rev fields from the NPM registry documents along with their narHash.

If you made these flake:inputs.myPackument.url = "https://registry.npmjs.org/foo?_rev=...";

Alternatively you can move stuff over but since you refactored libfetch you never rewrote the metaEntFrom* stuff.

There is an edge case in the current filtering that causes some packages to be preserved when they should be dropped.

This happens if a nested module's parent if filtered out, the nested module may not be dropped if it lacks a cpu and os field.
For example foo is the parent of bar in node_modules/foo/node_modules/bar, dropping foo should always cause bar to be dropped; but currently bar may be preserved if it lacks cpu or os fields.

Particularly since global install support was added, and new pjsUtil extensions have been created the integration with mkNmDirCmd has become wonky.

I think in practice ensuring that any installNodeModules routine accepts install prefixes as an arg is a good solution. See pjsSetDirs for an example.

Further I think it makes sense for global installs to be performed in a separate derivation. Doing so helps simplify branching logic and eliminates circular dependencies.

The Semver Range parser works in the vast majority of cases, but needs a bit of edge case coverage to be ready for use.

The following patterns fail to parse with parseVersionConstraint':

[x] "~0.1.2"
[x] "3.x", "3.X"
[ ] "1 - 2" -> "2.0.3"
[ ] "1 - 3" -> "3.0.1"
[ ] "2 - 3" -> "3.0.1"
[x] "^2.4.1 || ^3.0.0" = "3.3.3"
[x] "^3.0.0 || ^4.0.0" = "4.0.0"

This list is from a relatively large set of ~4,000 packages and includes all dependency descriptors that appear across them. It is not a "complete" list and I am sure there will be edge cases to deal with beyond this list.

https://github.com/aameen-tulip/at-node-nix/blob/main/lib/pkg-lock.nix#L212

This regex looks wrong, it's missing a / before ident.

Not 100% sure, but add test case.
This gets used in current V2 routines AFAIK.

Basically just a frontend for lib.libpkginfo, but you should handle workspaces.

Rather than probing source trees for registry packages we should refer to registry packuments first.

The hangup here is finding a way to tryFetchPackument, likely using the /search?text=@foo/bar endpoint to avoid making requests for private packages. Alternatively we could try to use the enty's fromType to only fetch for packages which are known to be registered.

This effort has already started under types/packument.nix and recent refactoring of libreg.

mkSourceTree, mkNmDirPlockV3, etc.

They're actually useful interfaces but they're too reliant on legacy.

Probably rewrite overlay.nix and just kill unconnected file.