ptl: mmu: Enable module data processing in userspace #10305

softwarecki · 2025-10-14T20:32:24Z

Add helper functions user_add_memory and user_remove_memory that allows to add/remove memory regions from the memory domain. The purpose of these functions is to round addresses appropriately for the memory domain.
Extend the module_adapter_new_ext function with an additional user_ctx parameter. This enables the caller to explicitly set the userspace context field within the processing module structure during module adapter initialization.
Introduce initial support for userspace proxy logic. This includes configuring the module's memory domain to ensure proper access rights and creating intermediary functions that forward calls to the module's interface.
Introduce support for loading userspace modules. Allow the DP thread to process audio data in userspace.

Add helper functions user_add_memory and user_remove_memory that allows to add/remove memory regions from the memory domain. The purpose of these functions is to round addresses appropriately for the memory domain. Signed-off-by: Jaroslaw Stelter <[email protected]> Signed-off-by: Adrian Warecki <[email protected]>

Introduce initial support for userspace proxy logic. This includes configuring the module's memory domain to ensure proper access rights and creating intermediary functions that forward calls to the module's interface. Signed-off-by: Jaroslaw Stelter <[email protected]> Signed-off-by: Adrian Warecki <[email protected]>

Extend the module_adapter_new_ext function with an additional user_ctx parameter. This enables the caller to explicitly set the userspace context field within the processing module structure during module adapter initialization. Signed-off-by: Adrian Warecki <[email protected]>

Introduce support for loading userspace modules. Allow the DP thread to process audio data in userspace. Signed-off-by: Jaroslaw Stelter <[email protected]> Signed-off-by: Adrian Warecki <[email protected]>

Copilot

Pull Request Overview

This PR implements userspace module data processing support in the PTL MMU component. It introduces helper functions for memory domain management, extends module adapter initialization, and provides userspace proxy functionality to enable audio data processing in userspace modules.

Add memory domain helper functions for adding/removing memory regions with proper alignment
Extend module adapter initialization with userspace context parameter
Implement userspace proxy module with complete interface forwarding for secure module execution

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
zephyr/lib/userspace_helper.c	Implements `user_add_memory` and `user_remove_memory` functions for memory domain management
zephyr/include/rtos/userspace_helper.h	Adds function declarations for memory domain helper functions
uuid-registry.txt	Registers UUID for userspace_proxy module
src/library_manager/lib_manager.c	Integrates userspace proxy creation and module registration with userspace support
src/include/sof/audio/module_adapter/module/generic.h	Updates module_adapter_new_ext signature to include userspace context
src/include/sof/audio/module_adapter/library/userspace_proxy.h	Defines userspace proxy interface and context structure
src/audio/module_adapter/module_adapter.c	Updates module adapter to use userspace-aware memory allocation
src/audio/module_adapter/module/modules.c	Adds APP_TASK_DATA annotation to processing module interface
src/audio/module_adapter/library/userspace_proxy.c	Implements complete userspace proxy module with interface forwarding
src/audio/module_adapter/CMakeLists.txt	Adds conditional compilation for userspace proxy

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-14T20:33:15Z

zephyr/lib/userspace_helper.c

+	int ret;
+
+	/* Define parameters for user_partition */
+	k_mem_region_align(&addr_aligned, &size_aligned, (uintptr_t)addr, size, HOST_PAGE_SIZE);


Redundant cast to uintptr_t - the parameter addr is already of type uintptr_t.

Suggested change

k_mem_region_align(&addr_aligned, &size_aligned, (uintptr_t)addr, size, HOST_PAGE_SIZE);

k_mem_region_align(&addr_aligned, &size_aligned, addr, size, HOST_PAGE_SIZE);

ditto - lets keep this in pages so that clients are forced to consider what's in userspace and whats not.

src/library_manager/lib_manager.c

src/audio/module_adapter/library/userspace_proxy.c

src/include/sof/audio/module_adapter/module/generic.h

src/audio/module_adapter/module_adapter.c

src/audio/module_adapter/library/userspace_proxy.c

lyakh · 2025-10-15T07:40:40Z

src/audio/module_adapter/library/userspace_proxy.c

+			   const struct sof_man_module *manifest, system_agent_start_fn start_fn,
+			   uintptr_t entry_point, uint32_t module_id, uint32_t instance_id,
+			   uint32_t core_id, uint32_t log_handle, byte_array_t *mod_cfg,
+			   const void **agent_interface, const struct module_interface **ops)


that looks like way too many parameters to me

I agree that this function takes too many parameters, but I see that lib_manager_start_agent looks similar. Maybe later it would be worth giving them a little makeover.

lyakh · 2025-10-15T07:55:43Z

src/audio/module_adapter/module_adapter.c

+#endif /* CONFIG_IPC_MAJOR_4 */
+
+	module_driver_heap_free(drv->user_heap, mod);
+	module_driver_heap_free(drv->user_heap, dev);


this and a similar hunk below are unrelated bug-fixes, also fixed in 0087192

lyakh · 2025-10-15T08:38:05Z

@softwarecki also a general request: could you please put your implementation under #if CONFIG_... using a new Kconfig option, e.g. the one I'm adding in 3e5f62c

lgirdwood

@softwarecki we are going to have several options around userspace and will need to split the CONFIG_USERSPACE into more meaningful configs e.g. we could have

CONFIG_USERSPACE code needed by all userspace.
CONFIG_USERSPACE_DP_PROXY code needed by this PR
CONFIG_USERSPACE_APP code needed by running infra and LL in userspace.

This way it should mean easier partitioning of the code.

One other thing is we will need more abstraction around memory alloc and free so that the modules allocate from the right place depending on the CONFIG.

lgirdwood · 2025-10-19T10:18:31Z

zephyr/lib/userspace_helper.c

 	return k_mem_domain_add_thread(comp_dom, thread_id);
 }

+int user_add_memory(struct k_mem_domain *domain, uintptr_t addr, size_t size, uint32_t attr)


it probably better if we fail if size or addr are not page aligned, that way we force the client users to think about whats in the domain (rather than silently growing the domain to include the whole page).

lgirdwood · 2025-10-19T10:22:40Z

zephyr/lib/userspace_helper.c

+	int ret;
+
+	/* Define parameters for user_partition */
+	k_mem_region_align(&addr_aligned, &size_aligned, (uintptr_t)addr, size, HOST_PAGE_SIZE);


ditto - lets keep this in pages so that clients are forced to consider what's in userspace and whats not.

src/audio/module_adapter/library/userspace_proxy.c

lgirdwood · 2025-10-19T10:38:17Z

src/audio/module_adapter/module_adapter.c

-	rfree(mod->stream_params);
-	rfree(mod);
-	rfree(dev);
+	module_driver_heap_free(drv->user_heap, mod->stream_params);
+	module_driver_heap_free(drv->user_heap, mod);
+	module_driver_heap_free(drv->user_heap, dev);


Fwiw, I have a change coming here that will mean some abstraction. Yes we will still be able to assign the pages to the user driver domain, but the method to allocate/free will be different as pipeline/DP modules will have virtual continuous pages preallocated for them (and the driver user heap would reside in these pages).

@softwarecki ok, we are still keeping the free() so this should merge in with minimal update now.

lgirdwood · 2025-10-19T10:40:35Z

src/library_manager/lib_manager.c

 	/* Intel modules expects DW size here */
 	mod_cfg.size = args->size >> 2;

+#if CONFIG_USERSPACE


We need to abstract this config a little better as we will have more userspace Kconfig options.

tmleman

Just minor comments and questions.

src/audio/module_adapter/library/userspace_proxy.c

tmleman · 2025-10-22T08:00:33Z

src/audio/module_adapter/library/userspace_proxy.c

+			   const struct sof_man_module *manifest, system_agent_start_fn start_fn,
+			   uintptr_t entry_point, uint32_t module_id, uint32_t instance_id,
+			   uint32_t core_id, uint32_t log_handle, byte_array_t *mod_cfg,
+			   const void **agent_interface, const struct module_interface **ops)


I agree that this function takes too many parameters, but I see that lib_manager_start_agent looks similar. Maybe later it would be worth giving them a little makeover.

tmleman · 2025-10-22T08:29:54Z

zephyr/lib/userspace_helper.c

+	ret = k_mem_domain_add_partition(domain, &part);
+	/* -EINVAL means that given page is already in the domain */
+	/* Not an error case for us. */
+	if (ret == -EINVAL)
+		return 0;


How do we know that's the case here? This error can be returned in many other cases by this function.

tmleman

You'll get my approval once you fix the issues reported by checkpatch and clarify that one small issue below.

tmleman · 2025-10-22T11:46:28Z

src/library_manager/lib_manager.c

-		rfree(drv);
-		rfree(new_drv_info);
+		module_driver_heap_free(drv_heap, drv);
+		module_driver_heap_free(drv_heap, new_drv_info);
+		module_driver_heap_remove(drv_heap);


new_drv_info is from rzalloc.

jxstelter and others added 4 commits October 14, 2025 21:49

ptl: mmu: Enable module data processing in userspace

fc52897

Introduce support for loading userspace modules. Allow the DP thread to process audio data in userspace. Signed-off-by: Jaroslaw Stelter <[email protected]> Signed-off-by: Adrian Warecki <[email protected]>

softwarecki requested review from abonislawski, Copilot, jxstelter, serhiy-katsyuba-intel and tmleman October 14, 2025 20:32

softwarecki requested review from dbaluta, iuliana-prodan, kv2019i, lbetlej, lgirdwood, lyakh, mmaka1, plbossart and ranj063 as code owners October 14, 2025 20:32

Copilot AI reviewed Oct 14, 2025

View reviewed changes

lyakh reviewed Oct 15, 2025

View reviewed changes

lgirdwood reviewed Oct 19, 2025

View reviewed changes

tmleman reviewed Oct 22, 2025

View reviewed changes

tmleman requested changes Oct 22, 2025

View reviewed changes

	k_mem_region_align(&addr_aligned, &size_aligned, (uintptr_t)addr, size, HOST_PAGE_SIZE);
	k_mem_region_align(&addr_aligned, &size_aligned, addr, size, HOST_PAGE_SIZE);

ptl: mmu: Enable module data processing in userspace #10305

Are you sure you want to change the base?

ptl: mmu: Enable module data processing in userspace #10305

Conversation

softwarecki commented Oct 14, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lyakh commented Oct 15, 2025

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmleman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmleman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants