Crash during headless export for linux (nix build on gh actions)

[AsbjornOlling]

Tested versions

Tested using this latest godot stable version, (from latest nixpkgs-unstable):
Godot Engine v4.5.1.stable.nixpkgs (f62fdbd)

Nix is a build system that cares a lot about reproducibility, so the environment should be extremely similar across different machines.

However, I have only seen this issue occur when running this build on github actions runners (still within the same nix sandbox). Running the exact same nix build locally works just fine.

The build script is available here:

https://github.com/nobodywho-ooo/nobodywho/blob/4a79bad54b263af849c3ad185bba9d21aee344e1/nobodywho/godot/integration-test/default.nix

System information

Nix sandbox (both using nix on ubuntu in github actions runner and nixos) - Godot 4.5.1 from nixos-unstable branch of nixpkgs - Vulkan Forward+ - running in the nix sandbox on a github actions runner

Issue description

I'm running a headless debug export of a godot game for x86 linux, using the nix build system. The godot game includes a gdextension. It crashes with what looks like a memory error: "Program crashed with signal 11"

The build succeeds on my local machine, in an identical nix sandbox, so I have a suspicion that it could be a race condition. I expect that a github actions runner running the exact same nix build would have both fewer and slower cores than my local workstation.

The command I use to export is this (inside the nix sandbox):

godot4 --verbose --headless --export-debug "Linux" $out/game

It fails like this:

 nobodywho-example-game> ================================================================
nobodywho-example-game> handle_crash: Program crashed with signal 11
nobodywho-example-game> Engine version: Godot Engine v4.5.1.stable.nixpkgs (f62fdbde15035c5576dad93e586201f4d41ef0cb)
nobodywho-example-game> Dumping the backtrace. Please include this when reporting the bug on: https://github.com/godotengine/godot/issues
nobodywho-example-game> [1] /nix/store/rcp9sdrrq8sfxkm5zdykglx7hd2gzbfy-glibc-2.40-66/lib/libc.so.6(+0x419c0) [0x7ffff2a419c0] (??:0)
nobodywho-example-game> [2] /nix/store/9np7aw0b9zwf65y24180pkv1qkd76rzn-godot-4.5.1-stable/bin/godot4(+0x1a6d6de) [0x555556fc16de] (??:?)
nobodywho-example-game> [3] /nix/store/9np7aw0b9zwf65y24180pkv1qkd76rzn-godot-4.5.1-stable/bin/godot4(+0x1a47a8c) [0x555556f9ba8c] (??:?)
nobodywho-example-game> [4] /nix/store/9np7aw0b9zwf65y24180pkv1qkd76rzn-godot-4.5.1-stable/bin/godot4(+0x1a51bf5) [0x555556fa5bf5] (??:?)
nobodywho-example-game> [5] /nix/store/9np7aw0b9zwf65y24180pkv1qkd76rzn-godot-4.5.1-stable/bin/godot4(+0x5089f31) [0x55555a5ddf31] (??:?)
nobodywho-example-game> [6] /nix/store/9np7aw0b9zwf65y24180pkv1qkd76rzn-godot-4.5.1-stable/bin/godot4(+0x4d6989f) [0x55555a2bd89f] (??:?)
nobodywho-example-game> [7] /nix/store/9np7aw0b9zwf65y24180pkv1qkd76rzn-godot-4.5.1-stable/bin/godot4(main+0x1aac) [0x555555c417ec] (??:?)
nobodywho-example-game> [8] /nix/store/rcp9sdrrq8sfxkm5zdykglx7hd2gzbfy-glibc-2.40-66/lib/libc.so.6(+0x2a4d8) [0x7ffff2a2a4d8] (??:0)
nobodywho-example-game> [9] /nix/store/rcp9sdrrq8sfxkm5zdykglx7hd2gzbfy-glibc-2.40-66/lib/libc.so.6(__libc_start_main+0x8b) [0x7ffff2a2a59b] (??:0)
nobodywho-example-game> [10] /nix/store/9np7aw0b9zwf65y24180pkv1qkd76rzn-godot-4.5.1-stable/bin/godot4(+0x72fa05) [0x555555c83a05] (??:?)
nobodywho-example-game> -- END OF C++ BACKTRACE --
nobodywho-example-game> ================================================================
nobodywho-example-game> /nix/store/k5z8rkvj1fn1nrf05l5a86aflc5s1p6m-stdenv-linux/setup: line 1783: 37656 Aborted                    (core dumped) /nix/store/9np7aw0b9zwf65y24180pkv1qkd76rzn-godot-4.5.1-stable/bin/godot4 --verbose --headless --export-debug "Linux" $out/game
error: Cannot build '/nix/store/6rsnlqhi1x031s24wwn7mbpw2zkj467r-nobodywho-example-game.drv'.
       Reason: builder failed with exit code 134.
       Output paths:
         /nix/store/1gpx9p22adqsigmc52hzr0c5jbkws9an-nobodywho-example-game
error: build of '/nix/store/6rsnlqhi1x031s24wwn7mbpw2zkj467r-nobodywho-example-game.drv', '/nix/store/vky5aw69sjfrjr2nl9szn10iw4fbk0bd-checkgame.drv' failed

Steps to reproduce

Run nix build --extra-experimental-features "nix-command flakes" -L 'github:nobodywho-ooo/nobodywho/4a79bad54b263af849c3ad185bba9d21aee344e1#checks.x86_64-linux.default' on a github actions machine with nix - (or possibly any sufficiently slow machine)

Or, if you have docker and not nix:

docker run -it nixos/nix nix build --extra-experimental-features "nix-command flakes" -L 'github:nobodywho-ooo/nobodywho/4a79bad54b263af849c3ad185bba9d21aee344e1#checks.x86_64-linux.default'

But do note that I see this issue far more frequently on a github actions machine. It's difficult for me to reproduce on my local, powerful machine.

Minimal reproduction project (MRP)

The project is already a very small reproduction project. It is the integration test for my gdextension.

See the code here:
https://github.com/nobodywho-ooo/nobodywho/tree/4a79bad54b263af849c3ad185bba9d21aee344e1/nobodywho/godot/integration-test

Note that I can't reliably reproduce the issue on my own local machine: only on github actions runners (still inside the nix build sandbox)

[Zed-z]

I've encountered the same issue on GitHub Actions (runners using Ubuntu and Arch Linux containers), while building locally worked perfectly.

Although I found that despite this issue, the export itself still successfully generates output files (it crashes after savepack).

A (bad) workaround I found for now is to add || true after the godot export command. If there are no other errors, artifacts can still be successfully copied.

[AsbjornOlling]

Wow. It works.

Never would I have expected such a thing as a non-critical segfault. Damn.

Thanks for the tip, @Zed-z !