use of xeus-cpp support from xeus-python

[maedoc]

hi, this is very cool, and I was wondering if it would be remotely feasible to use the xeus-cpp compiler to generate wasm modules which can be used from xeus-python via e.g. cffi?

[vgvassilev]

One approach we are exploring is connecting both via https://github.com/compiler-research/cppyy

There is still work that needs to be done.

cc: @aaronj0

[anutosh491]

Hey @maedoc

Thanks a lot for raising this. I think you can use xeus-cpp as a runtime compiler to generate wasm modules which you can then call from xeus-python.

I played around with this and this is what I could do. Here's a simple example

Prerequisities : You need a Jupyterlite instance with kernel support for both xeus-cpp and xeus-python. This is the offical link Jupyter provides where you can achieves this : https://jupyter.org/try-jupyter/lab/

Step 1: Open a C++ kernel . Any input_line_X technically gives you an incr_module_X+1.wasm. So processing cell 1 would give you incr_module_2.wasm

Step 2 : The wasm modules are being stored in the tmp (temporary) dir. Can probably use it from there would be better to shift it to the file system on the left (in the drive dir, so that you can see it being present)

Possibly something like this (can see the wasm binary in the file system on the left)

Step 3: Open a python kernel and fetch cffi through a mamba magic (fetches stuff at runtime for you)

Step 4: Load the wasm binary and make use of it

[anutosh491]

P.S : Even I wasn't sure ffi's dlopen maps correctly with the dynamic loading in wasm but I think it does.

Let me know if this something you were looking for. Thanks for the report.

[maedoc]

thanks for the reply! that's pretty much what I was after: the ability to prototype cxx kernels and python. this looks great, I'm going to try it out.

An auxiliary question: how to influence the optimization flags used to compile cxx?

[anutosh491]

An auxiliary question: how to influence the optimization flags used to compile cxx?

Hmmm, for that you might want to build the xeus-cpp-lite kernel locally and extend it as per your use case. Our kernels (let's say the C++20 one) currently is extended only to allowing exception handling and simd in wasm

xeus-cpp/share/jupyter/kernels/xcpp20/wasm_kernel.json.in

Lines 1 to 8 in 64e0919

	{
	"display_name": "C++20",
	"argv": [
	"@XEUS_CPP_KERNELSPEC_PATH@xcpp",
	"-resource-dir", "/lib/clang/@CPPINTEROP_LLVM_VERSION_MAJOR@",
	"-std=c++20", "-mllvm", "-enable-emscripten-cxx-exceptions",
	"-mllvm", "-enable-emscripten-sjlj", "-msimd128"
	],

As can be seen in the console eventually

If you want to change anything in how the incoming c++ code is being compiled to wasm, you need to add relevant flags in the kernel.json and build the kernel accordingly for the use case.

[maedoc]

this is really great! thanks for being generous with your time answering the questions.

I have one last one question then I'll stop 😅 : I'm still trying to understand jupyterlite architecture, and on a regular jupyter install, I could, from Python, generate some C++ source, subprocess.Popen the C++ compiler to generate the shared object, and so on. Is there here a way to invoke this kernel @XEUS_CPP_KERNELSPEC_PATH@xcpp programmatically to build the wasm?