Add TCI Support

[Naville]

Title.

Per discussion with @wtdcode

[Naville]

Note:
This was tested with -DUNICORN_INTERPRETER=ON -DBUILD_SHARED_LIBS=OFF -DUNICORN_ARCH=arm on:

Darwin Kernel Version 24.0.0: Tue Sep 24 23:39:07 PDT 2024; root:xnu-11215.1.12~1/RELEASE_ARM64_T6000 arm64

and iOS.
Only one arch is selected due to the linkage issue we discussed privately in email, that I still don't know why and how to fix

[aquynh]

Can you add a doc on how to use this as a framework?

…

On Mon, Nov 4, 2024, 10:26 Zhang ***@***.***> wrote: Title. Per discussion with @wtdcode <https://github.com/wtdcode> ------------------------------ You can view, comment on, or merge this pull request online at: #2050 Commit Summary - 57ce881 <57ce881> [qemu]Copy tci from upstream qemu of corresponding version - 771e1a7 <771e1a7> [qemu][uc]Define interpreter mode in build options - 9d20ee8 <9d20ee8> [qemu]In interpreter mode, disable allocation of _EXEC memory - 0d9f826 <0d9f826> [qemu]It compiles! File Changes (9 files <https://github.com/unicorn-engine/unicorn/pull/2050/files>) - *M* CMakeLists.txt <https://github.com/unicorn-engine/unicorn/pull/2050/files#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20a> (16) - *M* qemu/accel/tcg/translate-all.c <https://github.com/unicorn-engine/unicorn/pull/2050/files#diff-842456abe9564ae1e7d75ab8f322be6c27ca3c512e445a18e5898dea68ad9799> (6) - *M* qemu/configure <https://github.com/unicorn-engine/unicorn/pull/2050/files#diff-b756ac6e890984aafb9c8dbc2634906a2ac05b99818ee6b87f06863ba10a6179> (17) - *M* qemu/include/tcg/tcg.h <https://github.com/unicorn-engine/unicorn/pull/2050/files#diff-6ca102b9f51cd6308d5eb05777a4dfd67725e4265ce483d9dbe3400a44bf7dab> (1) - *M* qemu/tcg/tcg.c <https://github.com/unicorn-engine/unicorn/pull/2050/files#diff-730325e5d49d67286bf55f7870b5d49a69257a19c9791e166de1690ad0a4f6e7> (6) - *A* qemu/tcg/tci.c <https://github.com/unicorn-engine/unicorn/pull/2050/files#diff-0e5f53b874ae207c307972c1c85bed7c0d09578708acbaa45a625cc011212d1f> (1275) - *A* qemu/tcg/tci/README <https://github.com/unicorn-engine/unicorn/pull/2050/files#diff-037ab352d2d0a3f231b2ccf854ff594f2f909247558633e924167beb88636819> (130) - *A* qemu/tcg/tci/tcg-target.h <https://github.com/unicorn-engine/unicorn/pull/2050/files#diff-761b26202cf915d2bfe6c6a4e93fac3b10827658a8bba60987f5340b3bf7fecb> (213) - *A* qemu/tcg/tci/tcg-target.inc.c <https://github.com/unicorn-engine/unicorn/pull/2050/files#diff-12ee6faf7e35a522873d0642ce2e0466a01932046366a76fe3a744292cb0651f> (896) Patch Links: - https://github.com/unicorn-engine/unicorn/pull/2050.patch - https://github.com/unicorn-engine/unicorn/pull/2050.diff — Reply to this email directly, view it on GitHub <#2050>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNQNYFRSOG3ZQWXVATDXZTZ63LPHAVCNFSM6AAAAABRDK2NS6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGYZTCNRYGA3DGOA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Naville]

Can you add a doc on how to use this as a framework?

not sure what you meant by "as a framework", but in my test sample I just compiled and linked everything as a static library

[wtdcode]

Can you add a doc on how to use this as a framework?

This is still work-in-progress and I need to fix the tests failure before really adding flags, samples, docs etc.

@Naville Could you rebase this against dev branch?

For more context, TCI is an interpreter backend, i.e. doesn't do any JIT and thus is suitable for running on systems without writable & executable pages, specifically iOS. More context is included in #1695

[Naville]

Can you add a doc on how to use this as a framework?

This is still work-in-progress and I need to fix the tests failure before really adding flags, samples, docs etc.

@Naville Could you rebase this against dev branch?

For more context, TCI is an interpreter backend, i.e. doesn't do any JIT and thus is suitable for running on systems without writable & executable pages, specifically iOS. More context is included in #1695

done

[Naville]

Note: Once you're done, lmk so I could upstream patches I've provided for downstream package manager: vcpkg

[petabyt]

The tci code that handles INDEX_op_call will be problematic because it passes a fixed number of arguments to helper functions. Passing more arguments than a function accepts is undefined behavior in C. See:
.

unicorn/qemu/tcg/tci.c

Line 518 in 7f77dce

case INDEX_op_call:

I tweaked the build system to make this branch compile in emscripten. It compiles and runs, but it crashes on a basic example (function signature mismatch) because it passed too many arguments to a helper function.

The unicorn.js solution is to use an adapter generator:
https://github.com/AlexAltea/unicorn.js/blob/master/build.py
I tried merging the adapter generator into this branch but it ended up being harder than it looks.

Upstream qemu uses a different solution based on libffi. I assume we don't want to introduce another dependency.

[Naville]

Good catch!

Upstream qemu uses a different solution based on libffi. I assume we don't want to introduce another dependency.

This was copy-pasted from a previous release from qemu with VERY minimum modifications, I assume the upstream knows they broke something in subsequent releases

I assume we don't want to introduce another dependency.

I figure the most viable option here is to left this as-is and update the tcg copy with the baseline used by unicorn?

The tci code that handles INDEX_op_call will be problematic because it passes a fixed number of arguments to helper functions. Passing more arguments than a function accepts is undefined behavior in C. See: .

unicorn/qemu/tcg/tci.c

Line 518 in 7f77dce

case INDEX_op_call:

I tweaked the build system to make this branch compile in emscripten. It compiles and runs, but it crashes on a basic example (function signature mismatch) because it passed too many arguments to a helper function.

The unicorn.js solution is to use an adapter generator: https://github.com/AlexAltea/unicorn.js/blob/master/build.py I tried merging the adapter generator into this branch but it ended up being harder than it looks.

Upstream qemu uses a different solution based on libffi. I assume we don't want to introduce another dependency.

[petabyt]

I get a segfault doing a memory write in RISC-V:
Tested against this feature/tci branch. Not in emscripten.

#include <stdlib.h>
#include <stdio.h>
#include <unicorn/unicorn.h>

int main(void) {
	uc_engine *uc;
	uc_err err;

    // sw x1, 0(x0)
	char export[] = {0x23, 0x20, 0x10, 0x00, };

	err = uc_open(UC_ARCH_RISCV, UC_MODE_RISCV64, &uc);
    if (err) {
        printf("Failed on uc_open() with error returned: %u (%s)\n", err,
               uc_strerror(err));
        return -1;
    }

	uc_mem_map(uc, 0x0, 0x30000, UC_PROT_ALL);
	uc_mem_write(uc, 0x0, export, sizeof(export));

    err = uc_emu_start(uc, 0x0, 0x0 + 4, 0, 0x0);
    if (err) {
        printf("Failed on uc_emu_start() with error returned: %u\n", err);
    }

    uint64_t x0 = 0;
    uc_reg_read(uc, UC_RISCV_REG_X0, &x0);
    printf("x0: %lx\n", x0);
	return 0;
}

==128069== Warning: set address range perms: large range [0x4fac000, 0x44fac000) (defined)
==128069== Invalid read of size 8
==128069==    at 0x193BAD: tlb_index (cpu_ldst.h:106)
==128069==    by 0x198B6F: store_helper (cputlb.c:2089)
==128069==    by 0x199CE6: helper_le_stl_mmu_aarch64 (cputlb.c:2389)
==128069==    by 0x1AF335: tcg_qemu_tb_exec (tci.c:1245)
==128069==    by 0x3DCA3B: cpu_tb_exec (cpu-exec.c:60)
==128069==    by 0x3DD78B: cpu_loop_exec_tb (cpu-exec.c:506)
==128069==    by 0x3DD9DB: cpu_exec_riscv64 (cpu-exec.c:608)
==128069==    by 0x3A8450: tcg_cpu_exec (cpus.c:97)
==128069==    by 0x3A871B: resume_all_vcpus_riscv64 (cpus.c:216)
==128069==    by 0x3A87B6: vm_start_riscv64 (cpus.c:235)
==128069==    by 0x14EC32: uc_emu_start (uc.c:1101)
==128069==    by 0x14C9CF: main (main.c:22)
==128069==  Address 0x297 is not stack'd, malloc'd or (recently) free'd

Why is helper_le_stl_mmu_aarch64 being called? I've selected RISC-V...

[petabyt]

This patch to symbols.sh fixes it:

diff --git a/symbols.sh b/symbols.sh
index 29600c7e..ee49cca4 100755
--- a/symbols.sh
+++ b/symbols.sh
@@ -4,6 +4,8 @@ CMD_PATH=$(realpath $0)
 SOURCE_DIR=$(dirname ${CMD_PATH})
 
 COMMON_SYMBOLS="
+tci_tb_ptr \
+tcg_qemu_tb_exec \
 unicorn_fill_tlb \
 reg_read \
 reg_write \

Once it's run the headers will be redefined so each arch will call it's own tci runner.

[Naville]

@petabyt I no longer work on this anymore, feel free to take over my existing work

[petabyt]

@wtdcode Let me know if you need help fixing this PR, I have a branch partially working for my project here: staging...petabyt:unicorn:tci-emscripten

Fixed some other issues too.

[wtdcode]

@wtdcode Let me know if you need help fixing this PR, I have a branch partially working for my project here: staging...petabyt:unicorn:tci-emscripten

Fixed some other issues too.

Cool, will have a look.