grpc gem build failure about rails-nix HOT 8 CLOSED

I attempted using this overlay to fix the build:

final: super:
{
  grpc = super.grpc.overrideAttrs (old: {
    postPatch = ''
      substituteInPlace Makefile \
        --replace '-Wno-invalid-source-encoding' ""
    '' + lib.optionalString stdenv.isDarwin ''
      substituteInPlace src/ruby/ext/grpc/extconf.rb \
        --replace "ENV['AR'] = 'libtool'" ""
      substituteInPlace src/ruby/ext/grpc/extconf.rb \
      --replace "ENV['ARFLAGS'] = '-o'" ""

    '';
  });
}

and then in shell.nix

{ pkgs ? (import ./nixpkgs.nix {})}:

let
  newpkgs = pkgs.appendOverlays [
    (import ./overlay.nix)
  ];
  gems = newpkgs.bundlerEnv {
    name = "development";
    gemdir = ./.;
  };
in newpkgs.mkShell {
  buildInputs = [
    gems
    gems.wrappedRuby
  ];
}

This doesn't seem to perform the substitutions. What am I doing wrong?

from rails-nix.

Related PR to the actual failure: NixOS/nixpkgs#149901

I'm also curious - are there Darwin test runners for Nix? Would this build failure have been caught somehow by CI?

from rails-nix.

A lot of questions packed in that issue :).

I have to front-load with an explanation of why I'll be talking about gitaly. The grpc gem config is maintained mainly due to its use in gitaly.

I'm also curious - are there Darwin test runners for Nix? Would this build failure have been caught somehow by CI?

There is continuous integration for (almost) everything in Nixpkgs. Here's the link for gitaly for x86_64-darwin:

• https://hydra.nixos.org/job/nixpkgs/trunk/gitaly.x86_64-darwin/all

We can see that the version update here causes issues:

• First failure: https://hydra.nixos.org/build/165425510
• "change set" b05d20 to bd3256

I'll skip the how, it's hard and requires knowledge to really make sense, but we can assume the following PR is the cause of the "regression":

• NixOS/nixpkgs#156299
• version upgrade: https://github.com/NixOS/nixpkgs/pull/156299/files#diff-f2d42c7e377a08333eda6c5062d3eb734fd3e0db0e2ca82c010b033cbb80e951R26

On this PR, I can see that ofborg wasn't invoked to build gitaly specifically; it was for gitlab on x86_64-darwin, but it fails to evaluate, so nothing was built/tried to be built.

So to answer “are there Darwin test runners”, yes changes can be tested with some test infra, but getting everything tried-to-be-built is not necessarily trivial, and here gitlab not working on Darwin masked the slight regression.

from rails-nix.

Yeah. So what would we do to fix it? I've run into this in few projects already, as that gem is pretty common.

from rails-nix.

I attempted using this overlay to fix the build:

[...]

This doesn't seem to perform the substitutions. What am I doing wrong?

This is because you're patching the C library's derivation, and not the gem config.

• https://github.com/NixOS/nixpkgs/blob/3a7a5eed09415df114d4cabab99cb820d4f24c19/pkgs/development/libraries/grpc/default.nix

vs.

• https://github.com/NixOS/nixpkgs/blob/4d0e6f06a6ed0b17007c3d58d9be4dd77369656d/pkgs/development/ruby-modules/gem-config/default.nix#L293-L316

The solution to this would look more like the pg fix in this repo:

• rails-nix/default.nix

  Lines 10 to 11 in 16c65e5

  	defaultGemConfig = super.defaultGemConfig // {
  	pg = attrs: (super.defaultGemConfig.pg attrs) // {

If you look closely, the overlay changes the defaultGemConfig attribute, by using the update operator on defaultGemConfig to replace the pg attribute, which in turn uses the update operator to customize the postInstall attribute.

If you need the solution, I can provide it, but I figure with what you have already you should be good to complete the thought.

from rails-nix.

As far as solving, this is hard to do in a robust manner, but not harder to do locally than contributing upstream in Nixpkgs.

The main issue is that extconf.rb is changing across gem versions, but the gem config will technically be applied to any versions. It should be possible to fix by using attrs.version and lib.versionOlder or lib.versionAtLeast, and apply different fixes across versions.

An alternative could be to make the change "weirder", but more universal.

Changing ENV['AR']\s*= and ENV['ARFLAGS']\s*= both to ENV['XXX'] = would be functionally equivalent to removing the lines; the values would now be set to an environment variable that is not in use.

Another option could be to append to extconf.rb with the current values in the environment, but that becomes messy with escaping and such.

from rails-nix.

Nice - that worked. Someday I will get good at this!

from rails-nix.

Closing for now - a follow up can be done an an actual PR to Nixpkgs.

from rails-nix.