Add alembic for versioning of db tables #919
Conversation
deep1401
commented
Nov 21, 2025
deep1401
commented
- Handle existing tables
- Handle old existing user schema
…s are handled in init migration
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
| op.create_index(op.f("ix_config_key"), "config", ["key"], unique=True) | ||
|
|
||
| # Plugin table | ||
| if not table_exists("plugins"): |
There was a problem hiding this comment.
I am going to make a PR to remove plugins and training_templates from our code base. Since we're doing this as a sort of clean start let's not even bother putting them in the initial migration?
There was a problem hiding this comment.
I could just remove the definition directly in this PR itself and have the initial migration just drop those tables if they exist
There was a problem hiding this comment.
Yeah I think remove the definition. I'm not sure if it's necessary to drop the table or not. I guess there's no point in letting them hang around and possibly cause problems later.
Maybe in the future we remove all of the "old schema" cleanup.
There was a problem hiding this comment.
Okay will cleanup the definition and just drop those.
I can wait and let you confirm for workflow and workflowruns before doing those two as well
| op.create_index(op.f("ix_training_template_updated_at"), "training_template", ["updated_at"], unique=False) | ||
|
|
||
| # Workflow table | ||
| if not table_exists("workflows"): |
There was a problem hiding this comment.
Workflows and Workflow Runs are a little trickier. I'd love to eliminate them from the initial migration as well but I don't know with absolute certainty that we never hit the tables. Maybe I will try to clean that up today as well.
There was a problem hiding this comment.
I added a function in the env.py of alembic to not track workflow and workflow_runs tables so we might not have to purge things for those
|
Tested this with the old user schema and it worked (deleted old table and migrated). |
| output = f.read() | ||
| return output | ||
| return "" | ||
| # @router.get("/{job_id}/output") |
There was a problem hiding this comment.
Is this not used anymore? I was trying to figure out what was going on with the calls to training template in here!
If we do remove this it is causing ruff errors as a head's up!
There was a problem hiding this comment.
It was only being used in ViewOutputModal under Training but we dont use that modal anymore since we switched to ViewOutputModalStreaming so I deleted that modal too
There was a problem hiding this comment.
OK. I think we should just delete this then instead of commenting out. We can always go look it up in git history if we need it.
api/transformerlab/routers/train.py
Outdated
| datasets = configObject["dataset_name"] | ||
| await db.create_training_template(name, description, type, datasets, config) | ||
| return {"message": "OK"} | ||
| # @router.post("/template/create") |
There was a problem hiding this comment.
Let's just delete this. No point leaving most of the file commented out. :)
| output = f.read() | ||
| return output | ||
| return "" | ||
| # @router.get("/{job_id}/output") |
There was a problem hiding this comment.
OK. I think we should just delete this then instead of commenting out. We can always go look it up in git history if we need it.
|
Made the requested change now |
