divnix / std Goto Github PK

How would that look like?

This just doesn't correspond to the granularity requirement and not all outputs are derivations.

Push the definition closer to the derivations:

getPlatforms = d: if d?meta && d.meta?platforms
              then d.meta.platforms
              else throw "gotta put systems in place";
    smartPlatforms = lib.attrsets.mapAttrs (system: d:
        lib.filterAttrs (_: v: builtins.any (x: x == system) (getPlatforms v)) d
    );

/cc @tomberek

Within a same cell, it is awkward to accessor the outputs of that same cell with the cellname's prefix.

inputs.self.packages.<system>.cellname-mypackage
# could become
inputs.cell.packages.<system>.mypackage

inputs.cell thereby has only cell-local meaning.

This avoids much of the magic that a mypackage is all of a sudden accessored as cellname-mypackage, which feels very suspicious in practice.

At the same time, externally the cell-prefixing, which is required for good encapsulation of concerns is preserved.

https://clutch.sh/docs/about/what-is-clutch

This solves the problem of having upstream cells with different release cycles (in the same mono-repo), via harvesting.

{
  inputs.std.url = "...";
  inputs.other-cells.url = "...";
  outputs = inputs: inputs.std.growOn { /* ... */ }
    (inputs.std.harvest "foo" inputs.other-cells);
}

An interesting idea with potential also for Standard: https://github.com/nubank/ordnungsamt

A la sourcegraph-light.

• Cicero is the only open-source self-hostable CI/CD solution with a prime support for nix out there.

https://github.com/input-output-hk/cicero

b0828fd reads "guidance". An ADR log builds guidance.
Hence make an ADR Log.

Choose MADR, because why not and that nascent topic is just a mess (from that homepage).

• to honour the debug flag
• entice upstream PR fixes

mirror: NixOS/nix#6089

upstream is about to do away with defaultX anyway.

std does 2 level nesting:

• cell
  • organelle
    • target

nix does 1 level nesting (requiring prefixing):

• output
  • target

Make std style output shamelessly 2-level and avoid prefixing. Better mirrors the folder structure, anyway and keeps the mental model simpler.

/cc @tomberek

This is due, now:

std flow //cell/automation:build-a-house
[✔] build .003028233s
[x] fail .002209628s
[✔] failure .003738935s
[✔] test .002168770s

rel: #35

For bigger monorepos, we need a variation of tvl/depo/tools/magrathea that takes into account the celle/organelle/clade nomenclature.

I'm trying to conceptualize a patroni cell.

bitte-cells/patroni on  main [?] took 44m45s
❯ l -T
Permissions Size User      Date Modified Name
drwxrwxr-x     - blaggacao 25 Jan 18:05  .
drwxrwxr-x     - blaggacao 25 Jan 18:05  ├── apps
.rw-rw-r--  4.3k blaggacao 25 Jan 17:29  │  ├── backup-sidecar-entrypoint.sh
.rw-r--r--   725 blaggacao 25 Jan 17:28  │  └── patroni-entrypoint.sh
.rw-rw-r--   915 blaggacao 25 Jan 18:03  ├── apps.nix
drwxrwxr-x     - blaggacao 25 Jan 16:42  ├── nomad
.rw-rw-r--     0 blaggacao 25 Jan 16:42  │  ├── job.nix
.rw-rw-r--     0 blaggacao 25 Jan 16:42  │  └── taskgroups.nix
drwxrwxr-x     - blaggacao 25 Jan 18:05  ├── packages
.rw-r--r--  4.5k blaggacao 25 Jan 17:18  │  ├── clone-with-walg.py
.rw-r--r--  1.2k blaggacao 25 Jan 17:28  │  ├── patroni-callback.sh
.rw-r--r--   933 blaggacao 25 Jan 17:20  │  ├── restore-command.sh
.rw-r--r--  4.7k blaggacao 25 Jan 17:21  │  └── walg-restore.sh
.rw-rw-r--   964 blaggacao 25 Jan 18:04  ├── packages.nix
drwxrwxr-x     - blaggacao 25 Jan 16:41  └── profiles
.rw-rw-r--   103 blaggacao 26 Jan 14:28     ├── devshell.nix
.rw-rw-r--     0 blaggacao 25 Jan 16:41     ├── hydration.nix
.rw-rw-r--   458 blaggacao 26 Jan 15:11     └── nixos.nix

It has a bunch of specialties:

• It has specially typed functions, such as profiles (devshell, nixos, hyration =~ terranix config)
• It has one particularly specially typed function nomad that can render a job definition for patroni
• It outsources source code into subfolders for syntax highlighting, and generally improved IDE support.

Each cell should be consumed like so: github:input-output-hk/bitte-cells?dir=patroni in order to consume each cell according to it's individual release cycle, while still living in the same repo for access management and indexing/discovery purposes.

All the while, I might have a _shared cell with _shared/functions.nix that I still want to consume even if a cell is consumed via ?dir=<name> for it's individual release cycles' sake.

I will conduct a test if modern nix would allow me to inputs.shared = path:../.; in order to cater that need; if that works, this consideration could maybe find its way upstream.

• the one who overlays, use nixpkgs.extend in their cells

To bridge the gap between nix-experts and non-nix experts in a monorepo, we need an integration with just that can be used to gradually morph from an ad-hoc modus (under time pressure) into a rigorously reproducible world.

Document all the cells!

not neede anymore since #35

In parasitic adoption, we need to filter sources in order to avoid needless rebuilds, hence a good source filtering library is in order.

/cc @manveru https://github.com/input-output-hk/nix-inclusive

This shall be packaged right with std.

I don't know what the reason is at present, it may be caused by the filterattr process.

• https://github.com/GTrunSec/Coding-Dev-Env-With-NixFlake/blob/main/std/cells/test/lib.nix#L11
  • merging cell.library.mapAttrs' [infinite recursion encountered]
  • merging mapAttrs [works]

Reproduce code:

git clone https://github.com/GTrunSec/Coding-Dev-Env-With-NixFlake
cd Coding-Dev-Env-With-NixFlake/std
nix eval --json --option warn-dirty false .\#__std.x86_64-linux.test.lib

expected output:

{"foo_x":{"__std_cell":"test","__std_clade":"functions","__std_description":"n/a","__std_name":"foo_x","__std_organelle":"lib"},"foo_y":{"__std_cell":"test","__std_clade":"functions","__std_description":"n/a","__std_name":"foo_y","__std_organelle":"lib"}}% 

cc/ @jpverde

Something like:

{ inputs
, system
}: let
  nixpkgs = inputs.nixpkgs;
  makes =  inputs.makes + /src/args/agnostic.nix {
    inherit (system.host) system;
  } // { inherit inputs; }; #TODO: carefully de-system-space inputs here
  make = nixpkgs.lib.customization.callPackageWith makes;
in {
  createWallet = make ./createWallet/main.nix {};
}

@jpverde It would be nice if we could supply sources without having to move the fix-point of args, here: https://github.com/fluidattacks/makes/blob/1249f59eaf59c25c1afcc30f6852ee87cf653126/src/args/agnostic.nix#L113

... so that we can pass down nixpkgs.

Response: #27 (comment)

I did the cliche implement here. could you create a new repo for that or somehow? thanks

The readme is inconsistent with the code

I saw your @blaggacao work on bitte-cells that was pretty cool and practical such as _writers. Suddenly, one of the questions appeared in my mind, why not write a common lib for std orancells and especially like runner and writers..

From:

cli.nix
cli/main.scm

To:

cli/default.nix
cli/main.scm

Still ok:

cli.nix  # w/o a folder

x-compile is more like a 2% use case.

Rather implement a library function that makes it trivial to build cross-complied outputs within an organelle.

This also means the signature can change and { system } is no more required to pass down in the current way.

Signature change can happen together with #36

• devShell if devShell.nix
• devShells if devShells.nix

I think we can ship makes-essentials such as https://github.com/fluidattacks/makes/blob/main/src/args/make-template/builder.sh direcly as outputs.library.<system>.std-nameXY. (but only the essentials to limit the scope of "opinionating")

/cc @kamadorueda @tomberek wdyt?

(poor man's implementation of env substitution)

that let's you discover devshells more easily and enter them as an action.

cc @disassembler @nrdxp

It's easy to add a clade for this one: https://astro.github.io/microvm.nix/output-options.html

/cc @kreisys @manveru

Within the App Project epic:

• When we std.grow { cellsFrom = ./nix; } we call it the parasitic pattern -> developers perceive nix as a parasite.

• When we std.grow { cellsFrom = ./src; } we call it the symbiotic pattern -> developers embrace nix amidst their source code.

The parasitic pattern requires src = self + "/path/to/source".

To be shown: This makes usage of sources filter necessary to avoid cache collateral invalidations. Maybe use builtins.path more directly?

The symbiotic pattern can isolate sources with src = ./subfolder.

Otoh, the symbiotic pattern pushes down source code into subfolders. Acceptance?

• std ls | fzf | xargs std run
• std ls | fzf | xargs std build

We can provide an alias in the devshellProfile:

stdls run
stdls build

In order to be instantaneous, the devshell could cache std ls into PRJ_DATA on entry, where stdls then becomes:

cat $PRJ_DATA/stdls | fzf | xargs std

/cc @manveru

As an aproximate polyfill for hopeful:

inputs.X.patches

{
outputs = inputs': let
  inputs = inputs'.std.graft inputs' ./patches;
in inputs.std.grow { /* ... */ };
}

• "Grow cells on XY soil."
• Soil can have multiple layers:

{
  outputs = inputs: inputs.std.growOn { /* ... */ }
    # soil
    (inputs.std.harvest "cardano" inputs.bitte-cells1)  # top layer of soil
    (inputs.std.harvest "patroni" inputs.bitte-cells2)  # second layer of soil
  ;
}

we need a way to evolve the std API without too much downstream hassle

• should auto-lint certain patterns to enforce uniformity and stdization beyond the top-level args

Specifically it should enforce patterns on how to bring attributes from inputs and cell into scope (inherit-patterns).

Thx @kreisys for that idea!

The organelle definition at the flake.nix level in larger projects with largely non-aligned cell types is a cross-concern and doesn't feel right.

Rather it should be (optionally?) specific to each cell, while still being kept on the flake level for indexing purposes.

organelles.<cellname> = [];

• make use of yants for this

Basically what the title says. I can provide a simple repro if needed. 🐈

When borrowing ideas that are under a different license, we need this flexibility.

Currently inputs.cells is the actual stdOutput.

But that implies that it is flattened and cell-prefixed.

This is awkward:

{
inherit (inputs.cells) library;
writeText = library.<cellname>-writeText; # prefixing
}

Better would be:

{
inherit (inputs.cells.<cellname>.library) writeText;
}

Another use case (let's call it "collect functionality from cells") might indeed favor a prefixed model, hence we can also pass inputs.organelles with the prefixed outputs.

not needed any more

and double up #16 's md files for std info helptext rendering via https://github.com/charmbracelet/glamour

numtide/devshell is a nice onboarding trampoline for the more intricate entrypoints of just & magrathea.

I'd expect that an appropriate and concise devshell can help ergonomically reduce any contributing section by roughly 80% lah di dah.

This was a mistake (probably inspired by .#apps). Outputs are not only consumed by the CLI, but also in clasical nix and it gets quickly confusing if there is overloaded metadata present.

Instead only build up metadata within __std.

Some code represents just plain data. We need a clade for this, that doesn't take any (?) inputs.

Terminal needs reset after an action run.

The immediate and brutal use of execve might not be the best strategy.

Rather we should let tea to tea.Quit itself properly and only then execve