-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
Bug: rx.asset(shared=True) symlink creation triggers Granian hot reload crash loop after directory rename
Describe the bug
When renaming a Python module directory that contains rx.asset(path, shared=True) declarations, stale symlinks persist in assets/external/ pointing to the old directory path. On the next reflex run, rx.asset() creates new symlinks for the renamed path, which triggers Granian's hot reload file watcher. This causes an infinite worker crash loop:
[WARNING] Killing worker-0 after it refused to gracefully stop
The worker respawns, re-imports the app, rx.asset() runs again, symlinks are re-evaluated, Granian detects file activity, kills the worker, and the cycle repeats indefinitely. The app never becomes usable in development mode. Production mode (reflex run --env prod) is unaffected since it disables hot reload.
To Reproduce
-
Create a component using
rx.asset("my_file.js", shared=True)in a directory, e.g.,components/my_component_v1/ -
Run
reflex run— works fine. Symlinks are created inassets/external/…/my_component_v1/ -
Rename the directory:
my_component_v1/→my_component/ -
Run
reflex runagain -
Result: Infinite
[WARNING] Killing worker-0 after it refused to gracefully stopcrash loop
Root Cause
The issue involves an interaction between rx.asset() (in reflex/assets.py) and Granian's hot reload watcher:
-
rx.asset(shared=True)creates symlinks during import time (lines 84–95 ofassets.py), not during a dedicated compilation phase. This means symlink creation happens inside the Granian worker process. -
Symlinks are created inside
assets/external/, which is withinPath.cwd()— the directory monitored by Granian's reload watcher (viareload_pathsinrun_granian_backend). -
Stale symlinks are never cleaned up. When a source directory is renamed or deleted, the old symlinks in
assets/external/become broken but remain on disk. On the next run,rx.asset()creates new symlinks for the new path, writing to the watched directory. -
Granian detects the file writes and triggers a reload, which kills the current worker and spawns a new one. The new worker re-imports the app,
rx.asset()runs again, and the cycle repeats.
The workers_kill_timeout=2 (in run_granian_backend) is shorter than the typical app import time for large projects (~4–5 seconds), so the worker is killed before it even finishes importing — guaranteeing it never becomes ready.
Workaround
Manually delete the stale symlink directory:
# Find broken symlinks
find assets/external -xtype l
# Delete the stale directory
rm -rf assets/external/<module_path_to_old_directory>
Suggested Fix
Any one (or combination) of these would prevent the issue:
Option A: Clean up stale symlinks on startup (recommended)
Add a cleanup step before compilation that removes broken symlinks in assets/external/:
# In reflex startup/compile phase
external_dir = Path.cwd() / constants.Dirs.APP_ASSETS / constants.Dirs.EXTERNAL_APP_ASSETS
if external_dir.exists():
for symlink in external_dir.rglob("*"):
if symlink.is_symlink() and not symlink.resolve().exists():
symlink.unlink()
# Also remove empty directories left behind
for dirpath in sorted(external_dir.rglob("*"), reverse=True):
if dirpath.is_dir() and not any(dirpath.iterdir()):
dirpath.rmdir()Option B: Exclude assets/external/ from Granian reload paths
Since rx.asset() writes to assets/external/ during import (inside the worker), this directory should not trigger reloads. Add it to HOTRELOAD_IGNORE_PATTERNS or exclude it from reload_paths.
Option C: Defer symlink creation outside the worker process
Move the symlink creation from import time to a pre-worker compilation phase, so file writes don't occur inside the Granian-watched worker process.
Related Issues
-
Granian killing workers #5308 — Granian killing workers: Same symptom (
[WARNING] Killing worker-0 after it refused to gracefully stop) caused by.sqlitedatabase file writes triggering hot reload. -
ignore certain file formats from granian hot reload #5326 — Ignore certain file formats from granian hot reload: Fix for Granian killing workers #5308 that added
.db,.sqlite, etc. toHOTRELOAD_IGNORE_EXTENSIONS. This fix addressed database files but did not account forrx.asset()symlink writes inassets/external/.
Specifics
-
Python Version: 3.12
-
Reflex Version: 0.8.27
-
Granian: (bundled with Reflex)
-
OS: Debian/Ubuntu (WSL2)
Additional Context
The rx.asset() function in assets.py only creates symlinks when not backend_only and not dst_file.exists(). On a clean run with no stale symlinks, this is a no-op and causes no issues. The problem only manifests after renaming or deleting a directory that previously had rx.asset(shared=True) declarations, because the old symlinks become broken and new ones must be created at the new path.