8.8 KiB
title | description | series | tags |
---|---|---|---|
Building gomobile Using Nix | Harder than I thought it would be! | nebula | tech |
When I last left off with the nebula project I wanted to nix-ify the build process for Cryptic's mobile_nebula fork. While I've made progress on the overall build, one particular bit of it really held me up, so I'm writing about that part here. I'll finish the full build at a later time.
gomobile
gomobile is a toolkit for the go programming language to allow for
running go code on Android and iOS devices. mobile_nebula
uses gomobile
to
build a simple wrapper around the nebula client that the mobile app can then
hook into.
This means that in order to nix-ify the entire mobile_nebula
project I first
need to nix-ify gomobile
, and since there isn't (at time of writing) an
existing package for gomobile
in the nixpkgs repo, I had to roll my own.
I started with a simple buildGoModule
nix expression:
pkgs.buildGoModule {
pname = "gomobile";
version = "unstable-2020-12-17";
src = pkgs.fetchFromGitHub {
owner = "golang";
repo = "mobile";
rev = "e6ae53a27f4fd7cfa2943f2ae47b96cba8eb01c9";
sha256 = "03dzis3xkj0abcm4k95w2zd4l9ygn0rhkj56bzxbcpwa7idqhd62";
};
vendorSha256 = "1n1338vqkc1n8cy94501n7jn3qbr28q9d9zxnq2b4rxsqjfc9l94";
}
The basic idea here is that buildGoModule
will acquire a specific revision of
the gomobile
source code from github, then attempt to build it. However,
gomobile
is a special beast in that it requires a number of C/C++ libraries in
order to be built. I discovered this upon running this expression, when I
received this error:
./work.h:12:10: fatal error: GLES3/gl3.h: No such file or directory
12 | #include <GLES3/gl3.h> // install on Ubuntu with: sudo apt-get install libegl1-mesa-dev libgles2-mesa-dev libx11-dev
This stumped me for a bit, as I couldn't figure out a) the "right" place to
source the GLES3
header file from, and b) how to properly hook that into the
buildGoModule
expression. My initial attempts involved trying to include
versions of the header file from my androidsdk
nix package which I had already
gotten (mostly) working, but the version which ships there appears to expect to
be using clang. cgo
(go's compiler which is used for C/C++ interop) only
supports gcc, so that strategy failed.
I didn't like having to import the header file from androidsdk
anyway, as it
meant that my gomobile
would only work within the context of the
mobile_nebula
project, rather than being a standalone utility.
nix-index
At this point I flailed around some more trying to figure out where to get this
header file from. Eventually I stumbled on the nix-index project,
which implements something similar to the locate
utility on linux: you give it
a file pattern, and it searches your active nix channels for any packages which
provide a file matching that pattern.
Since nix is amazing it's not actually necessary to install nix-index
, I
simply start up a shell with the package available using nix-shell -p nix-index
. On first run I needed to populate the index by running the
nix-index
command, which took some time, but after that finding packages which
provide the file I need is as easy as:
> nix-shell -p nix-index
[nix-shell:/tmp]$ nix-locate GLES3/gl3.h
(zulip.out) 82,674 r /nix/store/wbfw7w2ixdp317wip77d4ji834v1k1b9-libglvnd-1.3.2-dev/include/GLES3/gl3.h
libglvnd.dev 82,674 r /nix/store/pghxzmnmxdcarg5bj3js9csz0h85g08m-libglvnd-1.3.2-dev/include/GLES3/gl3.h
emscripten.out 82,666 r /nix/store/x3c4y2h5rn1jawybk48r6glzs1jl029s-emscripten-2.0.1/share/emscripten/system/include/GLES3/gl3.h
So my mystery file is provided by a few packages, but libglvnd.dev
stood out
to me as it's also the pacman package which provides the same file in my real
operating system:
> yay -Qo /usr/include/GLES3/gl3.h
/usr/include/GLES3/gl3.h is owned by libglvnd 1.3.2-1
This gave me some confidence that this was the right track.
cgo
My next fight was with cgo
itself. Go's build process provides a few different
entry points for C/C++ compiler/linker flags, including both environment
variables and command-line arguments. But I wasn't using go build
directly,
instead I was working through nix's buildGoModule
wrapper. This added a huge
layer of confusion as all of nixpkgs is pretty terribly documented, so you
really have to just divine behavior from the source
(good luck).
After lots of debugging (hint: NIX_DEBUG=1
) I determined that all which is
actually needed is to set the CGO_CFLAGS
variable within the buildGoModule
arguments. This would translate to the CGO_CFLAGS
environment variable being
set during all internal commands, and whatever go build
commands get used
would pick up my compiler flags from that.
My new nix expression looked like this:
pkgs.buildGoModule {
pname = "gomobile";
version = "unstable-2020-12-17";
src = pkgs.fetchFromGitHub {
owner = "golang";
repo = "mobile";
rev = "e6ae53a27f4fd7cfa2943f2ae47b96cba8eb01c9";
sha256 = "03dzis3xkj0abcm4k95w2zd4l9ygn0rhkj56bzxbcpwa7idqhd62";
};
vendorSha256 = "1n1338vqkc1n8cy94501n7jn3qbr28q9d9zxnq2b4rxsqjfc9l94";
CGO_CFLAGS = [
"-I ${pkgs.libglvnd.dev}/include"
];
}
Running this produced a new error. Progress! The new error was:
/nix/store/p792j5f44l3f0xi7ai5jllwnxqwnka88-binutils-2.31.1/bin/ld: cannot find -lGLESv2
collect2: error: ld returned 1 exit status
So pretty similar to the previous issue, but this time the linker wasn't finding
a library file rather than the compiler not finding a header file. Once again I
used nix-index
's nix-locate
command to find that this library file is
provided by the libglvnd
package (as opposed to libglvnd.dev
, which provided
the header file).
Adding libglvnd
to the CGO_CFLAGS
did not work, as it turns out that flags
for the linker cgo
uses get passed in via CGO_LDFLAGS
(makes sense). After
adding this new variable I got yet another error; this time X11/Xlib.h
was not
able to be found. I repeated the process of nix-locate
/add to CGO_*FLAGS
a
few more times until all dependencies were accounted for. The new nix expression
looked like this:
pkgs.buildGoModule {
pname = "gomobile";
version = "unstable-2020-12-17";
src = pkgs.fetchFromGitHub {
owner = "golang";
repo = "mobile";
rev = "e6ae53a27f4fd7cfa2943f2ae47b96cba8eb01c9";
sha256 = "03dzis3xkj0abcm4k95w2zd4l9ygn0rhkj56bzxbcpwa7idqhd62";
};
vendorSha256 = "1n1338vqkc1n8cy94501n7jn3qbr28q9d9zxnq2b4rxsqjfc9l94";
CGO_CFLAGS = [
"-I ${pkgs.libglvnd.dev}/include"
"-I ${pkgs.xlibs.libX11.dev}/include"
"-I ${pkgs.xlibs.xorgproto}/include"
"-I ${pkgs.openal}/include"
];
CGO_LDFLAGS = [
"-L ${pkgs.libglvnd}/lib"
"-L ${pkgs.xlibs.libX11}/lib"
"-L ${pkgs.openal}/lib"
];
}
Tests
The CGO_*FLAGS
variables took care of all compiler/linker errors, but there
was one issue left: buildGoModule
apparently runs the project's tests after
the build phase. gomobile
's tests were actually mostly passing, but some
failed due to trying to copy files around, which nix was having none of. After
some more buildGoModule source divination I found that
if I passed an empty checkPhase
argument it would skip the check phase, and
therefore skip running these tests.
Fin!
The final nix expression looks like so:
pkgs.buildGoModule {
pname = "gomobile";
version = "unstable-2020-12-17";
src = pkgs.fetchFromGitHub {
owner = "golang";
repo = "mobile";
rev = "e6ae53a27f4fd7cfa2943f2ae47b96cba8eb01c9";
sha256 = "03dzis3xkj0abcm4k95w2zd4l9ygn0rhkj56bzxbcpwa7idqhd62";
};
vendorSha256 = "1n1338vqkc1n8cy94501n7jn3qbr28q9d9zxnq2b4rxsqjfc9l94";
CGO_CFLAGS = [
"-I ${pkgs.libglvnd.dev}/include"
"-I ${pkgs.xlibs.libX11.dev}/include"
"-I ${pkgs.xlibs.xorgproto}/include"
"-I ${pkgs.openal}/include"
];
CGO_LDFLAGS = [
"-L ${pkgs.libglvnd}/lib"
"-L ${pkgs.xlibs.libX11}/lib"
"-L ${pkgs.openal}/lib"
];
checkPhase = "";
}
Once I complete the nix-ification of mobile_nebula
I'll submit a PR to the
nixpkgs upstream with this, so that others can have gomobile
available as
well!