divnix / std Goto Github PK
View Code? Open in Web Editor NEWA DevOps framework for the SDLC with the power of Nix and Flakes. Good for keeping deadlines!
Home Page: https://std.divnix.com
A DevOps framework for the SDLC with the power of Nix and Flakes. Good for keeping deadlines!
Home Page: https://std.divnix.com
How would that look like?
This just doesn't correspond to the granularity requirement and not all outputs are derivations.
Push the definition closer to the derivations:
getPlatforms = d: if d?meta && d.meta?platforms
then d.meta.platforms
else throw "gotta put systems in place";
smartPlatforms = lib.attrsets.mapAttrs (system: d:
lib.filterAttrs (_: v: builtins.any (x: x == system) (getPlatforms v)) d
);
/cc @tomberek
Within a same cell, it is awkward to accessor the outputs of that same cell with the cellname's prefix.
inputs.self.packages.<system>.cellname-mypackage
# could become
inputs.cell.packages.<system>.mypackage
inputs.cell
thereby has only cell-local meaning.
This avoids much of the magic that a mypackage
is all of a sudden accessored as cellname-mypackage
, which feels very suspicious in practice.
At the same time, externally the cell-prefixing, which is required for good encapsulation of concerns is preserved.
This solves the problem of having upstream cells with different release cycles (in the same mono-repo), via harvesting.
{
inputs.std.url = "...";
inputs.other-cells.url = "...";
outputs = inputs: inputs.std.growOn { /* ... */ }
(inputs.std.harvest "foo" inputs.other-cells);
}
An interesting idea with potential also for Standard: https://github.com/nubank/ordnungsamt
A la sourcegraph-light.
# soil
{
devShells = inputs.std.harvest inputs.self ["automation" "devshells"];
}
debug
flagmirror: NixOS/nix#6089
upstream is about to do away with defaultX
anyway.
std
does 2 level nesting:
nix
does 1 level nesting (requiring prefixing):
Make std style output shamelessly 2-level and avoid prefixing. Better mirrors the folder structure, anyway and keeps the mental model simpler.
/cc @tomberek
This is due, now:
std flow //cell/automation:build-a-house
[✔] build .003028233s
[x] fail .002209628s
[✔] failure .003738935s
[✔] test .002168770s
rel: #35
For bigger monorepos, we need a variation of tvl/depo/tools/magrathea
that takes into account the celle/organelle/clade nomenclature.
I'm trying to conceptualize a patroni cell.
bitte-cells/patroni on main [?] took 44m45s
❯ l -T
Permissions Size User Date Modified Name
drwxrwxr-x - blaggacao 25 Jan 18:05 .
drwxrwxr-x - blaggacao 25 Jan 18:05 ├── apps
.rw-rw-r-- 4.3k blaggacao 25 Jan 17:29 │ ├── backup-sidecar-entrypoint.sh
.rw-r--r-- 725 blaggacao 25 Jan 17:28 │ └── patroni-entrypoint.sh
.rw-rw-r-- 915 blaggacao 25 Jan 18:03 ├── apps.nix
drwxrwxr-x - blaggacao 25 Jan 16:42 ├── nomad
.rw-rw-r-- 0 blaggacao 25 Jan 16:42 │ ├── job.nix
.rw-rw-r-- 0 blaggacao 25 Jan 16:42 │ └── taskgroups.nix
drwxrwxr-x - blaggacao 25 Jan 18:05 ├── packages
.rw-r--r-- 4.5k blaggacao 25 Jan 17:18 │ ├── clone-with-walg.py
.rw-r--r-- 1.2k blaggacao 25 Jan 17:28 │ ├── patroni-callback.sh
.rw-r--r-- 933 blaggacao 25 Jan 17:20 │ ├── restore-command.sh
.rw-r--r-- 4.7k blaggacao 25 Jan 17:21 │ └── walg-restore.sh
.rw-rw-r-- 964 blaggacao 25 Jan 18:04 ├── packages.nix
drwxrwxr-x - blaggacao 25 Jan 16:41 └── profiles
.rw-rw-r-- 103 blaggacao 26 Jan 14:28 ├── devshell.nix
.rw-rw-r-- 0 blaggacao 25 Jan 16:41 ├── hydration.nix
.rw-rw-r-- 458 blaggacao 26 Jan 15:11 └── nixos.nix
It has a bunch of specialties:
nomad
that can render a job definition for patroniEach cell should be consumed like so: github:input-output-hk/bitte-cells?dir=patroni
in order to consume each cell according to it's individual release cycle, while still living in the same repo for access management and indexing/discovery purposes.
All the while, I might have a _shared
cell with _shared/functions.nix
that I still want to consume even if a cell is consumed via ?dir=<name>
for it's individual release cycles' sake.
I will conduct a test if modern nix
would allow me to inputs.shared = path:../.;
in order to cater that need; if that works, this consideration could maybe find its way upstream.
nixpkgs.extend
in their cellsTo bridge the gap between nix-experts and non-nix experts in a monorepo, we need an integration with just
that can be used to gradually morph from an ad-hoc modus (under time pressure) into a rigorously reproducible world.
Document all the cells!
not neede anymore since #35
In parasitic adoption, we need to filter sources in order to avoid needless rebuilds, hence a good source filtering library is in order.
/cc @manveru https://github.com/input-output-hk/nix-inclusive
This shall be packaged right with std
.
I don't know what the reason is at present, it may be caused by the filterattr process.
Reproduce code:
git clone https://github.com/GTrunSec/Coding-Dev-Env-With-NixFlake
cd Coding-Dev-Env-With-NixFlake/std
nix eval --json --option warn-dirty false .\#__std.x86_64-linux.test.lib
expected output:
{"foo_x":{"__std_cell":"test","__std_clade":"functions","__std_description":"n/a","__std_name":"foo_x","__std_organelle":"lib"},"foo_y":{"__std_cell":"test","__std_clade":"functions","__std_description":"n/a","__std_name":"foo_y","__std_organelle":"lib"}}%
cc/ @jpverde
Something like:
{ inputs
, system
}: let
nixpkgs = inputs.nixpkgs;
makes = inputs.makes + /src/args/agnostic.nix {
inherit (system.host) system;
} // { inherit inputs; }; #TODO: carefully de-system-space inputs here
make = nixpkgs.lib.customization.callPackageWith makes;
in {
createWallet = make ./createWallet/main.nix {};
}
@jpverde It would be nice if we could supply sources without having to move the fix-point of args
, here: https://github.com/fluidattacks/makes/blob/1249f59eaf59c25c1afcc30f6852ee87cf653126/src/args/agnostic.nix#L113
... so that we can pass down nixpkgs
.
Response: #27 (comment)
I did the cliche implement here. could you create a new repo for that or somehow? thanks
The readme is inconsistent with the code
I saw your @blaggacao work on bitte-cells that was pretty cool and practical such as _writers. Suddenly, one of the questions appeared in my mind, why not write a common lib for std orancells and especially like runner
and writers.
.
From:
cli.nix
cli/main.scm
To:
cli/default.nix
cli/main.scm
Still ok:
cli.nix # w/o a folder
x-compile is more like a 2% use case.
Rather implement a library function that makes it trivial to build cross-complied outputs within an organelle.
This also means the signature can change and { system }
is no more required to pass down in the current way.
Signature change can happen together with #36
devShell
if devShell.nix
devShells
if devShells.nix
I think we can ship makes-essentials such as https://github.com/fluidattacks/makes/blob/main/src/args/make-template/builder.sh direcly as outputs.library.<system>.std-nameXY
. (but only the essentials to limit the scope of "opinionating")
/cc @kamadorueda @tomberek wdyt?
that let's you discover devshells more easily and enter them as an action.
It's easy to add a clade for this one: https://astro.github.io/microvm.nix/output-options.html
Within the App Project epic:
When we std.grow { cellsFrom = ./nix; }
we call it the parasitic pattern -> developers perceive nix as a parasite.
When we std.grow { cellsFrom = ./src; }
we call it the symbiotic pattern -> developers embrace nix amidst their source code.
The parasitic pattern requires src = self + "/path/to/source"
.
To be shown: This makes usage of sources filter necessary to avoid cache collateral invalidations. Maybe use builtins.path
more directly?
The symbiotic pattern can isolate sources with src = ./subfolder
.
Otoh, the symbiotic pattern pushes down source code into subfolders. Acceptance?
std ls | fzf | xargs std run
std ls | fzf | xargs std build
We can provide an alias in the devshellProfile
:
stdls run
stdls build
In order to be instantaneous, the devshell could cache std ls
into PRJ_DATA
on entry, where stdls
then becomes:
cat $PRJ_DATA/stdls | fzf | xargs std
/cc @manveru
As an aproximate polyfill for hopeful:
inputs.X.patches
{
outputs = inputs': let
inputs = inputs'.std.graft inputs' ./patches;
in inputs.std.grow { /* ... */ };
}
{
outputs = inputs: inputs.std.growOn { /* ... */ }
# soil
(inputs.std.harvest "cardano" inputs.bitte-cells1) # top layer of soil
(inputs.std.harvest "patroni" inputs.bitte-cells2) # second layer of soil
;
}
we need a way to evolve the std
API without too much downstream hassle
std
ization beyond the top-level argsSpecifically it should enforce patterns on how to bring attributes from inputs
and cell
into scope (inherit
-patterns).
Thx @kreisys for that idea!
The organelle definition at the flake.nix level in larger projects with largely non-aligned cell types is a cross-concern and doesn't feel right.
Rather it should be (optionally?) specific to each cell, while still being kept on the flake level for indexing purposes.
organelles.<cellname> = [];
Basically what the title says. I can provide a simple repro if needed. 🐈
When borrowing ideas that are under a different license, we need this flexibility.
Currently inputs.cells
is the actual stdOutput
.
But that implies that it is flattened and cell
-prefixed.
This is awkward:
{
inherit (inputs.cells) library;
writeText = library.<cellname>-writeText; # prefixing
}
Better would be:
{
inherit (inputs.cells.<cellname>.library) writeText;
}
Another use case (let's call it "collect functionality from cells") might indeed favor a prefixed model, hence we can also pass inputs.organelles
with the prefixed outputs.
not needed any more
and double up #16 's md
files for std info
helptext rendering via https://github.com/charmbracelet/glamour
numtide/devshell
is a nice onboarding trampoline for the more intricate entrypoints of just
& magrathea
.
I'd expect that an appropriate and concise devshell can help ergonomically reduce any contributing section by roughly 80% lah di dah.
This was a mistake (probably inspired by .#apps
). Outputs are not only consumed by the CLI, but also in clasical nix and it gets quickly confusing if there is overloaded metadata present.
Instead only build up metadata within __std
.
Some code represents just plain data. We need a clade for this, that doesn't take any (?) inputs.
Terminal needs reset
after an action run.
The immediate and brutal use of execve
might not be the best strategy.
Rather we should let tea
to tea.Quit
itself properly and only then execve
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.