update build script for lmbench

[robinyuan1002]

There are mainly three scripts. One bootstrap for libtirpc, one bootstrap for lmbench and a shell script for compiling lmbench tests.

[m-hemmings]

To align with the proposed build contract style from issue 10 here are my suggestions:

1. Create a makefile that performs the make actions you need (all and clean)
2. Set up the folder structure similar to this:

lmbench/
├── Makefile
├── compile_lmbench.sh
├── out/
│   ├── lat_syscall.cwasm
│   └── syscall.cwasm
└── sysroot_overlay/
    ├── include/
    │   └── wasm32-wasi/
    │       ├── netconfig.h
    │       └── rpc/
    │           ├── svc.h
    │           └── clnt.h
    └── lib/
        └── wasm32-wasi/
            └── libc.a

3. Change the scripts to not modify the sysroot or files therein directly, but instead place them into the sysroot_overlay folder and allow the tooling script to manage merging. It will call the make all and then handle merging the files into the appropriate places following the folder structure relative to sysroot that you put in that folder (example above)

[m-hemmings]

Since the script depends on several specific tools (clang, wasm-opt, wasmtime, and a specific SYSROOT), it might be a good idea to move all the path exports and environment variable setups to the top of the script using fallbacks like this:

export CLANG="${CLANG:-/home/lind/lind-wasm/clang+llvm-16.0.4-x86_64-linux-gnu-ubuntu-22.04/bin/clang}"
export CXX="${CXX:-/home/lind/lind-wasm/clang+llvm-16.0.4-x86_64-linux-gnu-ubuntu-22.04/bin/clang++}"
export AR="${AR:-/home/lind/lind-wasm/clang+llvm-16.0.4-x86_64-linux-gnu-ubuntu-22.04/bin/llvm-ar}"
export LD="${LD:-/home/lind/lind-wasm/clang+llvm-16.0.4-x86_64-linux-gnu-ubuntu-22.04/bin/wasm-ld}"
export RANLIB="${RANLIB:-/home/lind/lind-wasm/clang+llvm-16.0.4-x86_64-linux-gnu-ubuntu-22.04/bin/llvm-ranlib}"

export WASM_OPT="${WASM_OPT:-/home/lind/lind-wasm/tools/binaryen/bin/wasm-opt}"
export WASMTIME="${WASMTIME:-/home/lind/lind-wasm/src/wasmtime/target/debug/wasmtime}"

export SYSROOT="${SYSROOT:-/home/lind/lind-wasm/src/glibc/sysroot}"

export CFLAGS="${CFLAGS:---sysroot=$SYSROOT -g -O2}"
export CXXFLAGS="${CXXFLAGS:---sysroot=$SYSROOT -g -O2}"
export LDFLAGS="${LDFLAGS:---sysroot=$SYSROOT -Wl,--import-memory,--export-memory,--max-memory=67108864,--export=__stack_pointer,--export=__stack_low}"

This way you could modify them at the time you call the compile without having to edit the script, like this:
WASMTIME=$HOME/.cargo/bin/wasmtime ./compile_lmbench.sh

And if you later want to move it to the Makefile to adhere to the build contract style we talked about in Issue 10

[rennergade]

@m-hemmings so basically we have to build libtirpc.a with wasm-clang, then combine that lib with the sysroot into a new sysroot and then build lmbench.

lmbench should build somewhat properly on it's own once that sysroot is created and has its own Makefile/output folder for binaries once that step is completed.

[rennergade]

@m-hemmings this is where the sysroot stuff comes into play, I like your overlay idea but i'm not sure runbin and I completely understand it

[m-hemmings]

@rennergade The idea should be that the app does all it needs to in its own context and then the tooling script moves what needs to be moved. So I wonder, could this merge be done in the app's own folders and then have the merged file be placed in the sysroot_overlay folder with the same folder structure it would need if it went to sysroot? or is it required that it be done in situ in the actual sysroot?
The original idea was that the app would not modify the actual sysroot at all, but we can revisit that if it doesn't fit

[rennergade]

@robinyuan1002 can we not call the lmbench Makefile to build it? not the toplevel one but the one in src?

[robinyuan1002]

yes we can, but I think I can create a new Makefile just for compiling tests.

[rennergade]

If the one provided works why would we want to add a different one?

[rennergade]

@stupendoussuperpowers can you take a look at just using the Makefile in lmbench/src instead of us creating a new script

[robinyuan1002]

If we want to create a new Makefile should we delete the old one in the lmbench directory?

[m-hemmings]

My original idea for build contracts and application installations was like this, as briefly touched on above, but I think I can be clearer than I was:

Per-app:

produce artifacts in an app-owned overlay (sysroot_overlay/). Headers and small compat libs only. No sudo. No writes to $SYSROOT. No libc surgery.

Top-level: a single orchestrator:

(Makefile target or script in the repo root) Assembles a scratch sysroot for the build by layering the base sysroot + app overlays. That orchestrator is the only thing allowed to copy into a sysroot tree.
something like, but obviously not exactly, like this:

1. for each app in $(APPS):
    # if necessary, call bootstrap:
    make app bootstrap
    # call the build for the app here.
    # compiles to app/out + app/sysroot_overlay
    make -C app build
2. merge all app/sysroot_overlay trees into a scratch sysroot
3. link or test using that scratch sysroot.  If libc.a needs to be rebuilt, it should happen here once after all the merges are complete

[m-hemmings]

did we not update this to clang18?

[robinyuan1002]

I not very sure about this, @rennergade should we use clang18 or clang16 here

[rennergade]

It definitely should be 18 but we probably shouldn't hardcode that into these. For now put it at 18 but @m-hemmings can we think of a better option

[m-hemmings]

This should be clang 18, shouldn't it?