Fix bulk_create not setting M2M on FK-related objects#583
Fix bulk_create not setting M2M on FK-related objects#583benaduo wants to merge 2 commits intomodel-bakers:mainfrom
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughAdds a second post-processing pass in bulk_create to apply many-to-many relations specified via single-level double-underscore kwargs on foreign-key-related objects (e.g., Changes
Sequence DiagramsequenceDiagram
participant User
participant Baker as baker.bulk_create
participant ORM as Django ORM
participant Through as Through Model
User->>Baker: bulk_create(entries, home__dogs=[dog], _bulk_create=True)
Baker->>Baker: prepare entries, persist FK fields
Baker->>ORM: bulk_create(prepared_entries)
ORM-->>Baker: created objects
Baker->>Baker: existing per-entry M2M handling
Baker->>Baker: parse single-level `fk__m2m` kwargs
loop per created object with fk__m2m
Baker->>Baker: validate FK is FK/OneToOne and m2m exists
Baker->>Through: set m2m on fk_obj (if through.auto_created)
Through-->>Baker: relation established
end
Baker-->>User: return created entries
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip You can enable review details to help with troubleshooting, context usage and more.Enable the |
When baker.make() is called with _bulk_create=True and a M2M field is specified via double-underscore FK lookup syntax (e.g. home__dogs=[dog]), the M2M relationship was never applied to the saved FK object. Root cause: during the prepare() phase, baker correctly stores M2M values in the sub-baker's m2m_dict, but _handle_m2m() is gated behind `if _commit:` and therefore never executes in the bulk_create path. Subsequently, _save_related_objs() only saves FK rows and makes no M2M calls. The two existing M2M loops in bulk_create() scan only the top-level model's direct and reverse M2M fields, neither of which covers the fk__m2m kwarg pattern. The net result is that home.dogs.set([dog]) is never called, leaving the M2M relation empty in the database. Fix: after all entries have been bulk-created and FK objects saved, inspect the kwargs dict for double-underscore patterns of the form fk_field__m2m_field. For each pattern where the prefix resolves to a ForeignKey or OneToOneField on the current model and the suffix resolves to a ManyToManyField on the related model, call .set() on the FK object for every created entry. Changes: - model_bakery/baker.py: add FK-nested M2M handling loop at the end of bulk_create(), after the existing reverse-relation M2M loop - tests/generic/models.py: add HomeOwner model with ForeignKey to Home, providing the minimal fixture required to reproduce the reported scenario - tests/test_baker.py: add test_create_through_foreign_key_field to TestCreateM2MWhenBulkCreate, reproducing the exact scenario from the issue (baker.make(HomeOwner, home__dogs=[dog], _quantity=10, _bulk_create=True)) Fixes model-bakers#490 Author: Benjamin Aduo (@benaduo)
When baker.make() is called with _bulk_create=True and a M2M field is specified via double-underscore FK lookup syntax (e.g. home__dogs=[dog]), the M2M relationship was never applied to the saved FK object. Root cause: during the prepare() phase, baker correctly stores M2M values in the sub-baker's m2m_dict, but _handle_m2m() is gated behind `if _commit:` and therefore never executes in the bulk_create path. Subsequently, _save_related_objs() only saves FK rows and makes no M2M calls. The two existing M2M loops in bulk_create() scan only the top-level model's direct and reverse M2M fields, neither of which covers the fk__m2m kwarg pattern. The net result is that home.dogs.set([dog]) is never called, leaving the M2M relation empty in the database. Fix: after all entries have been bulk-created and FK objects saved, inspect the kwargs dict for double-underscore patterns of the form fk_field__m2m_field. For each pattern where the prefix resolves to a ForeignKey or OneToOneField on the current model and the suffix resolves to a ManyToManyField on the related model, call .set() on the FK object for every created entry. Only auto-created through models are supported; fields using a custom through= model are skipped since they require explicit through instance creation with extra field values baker cannot infer. FieldDoesNotExist is caught specifically (not bare Exception) so that unexpected errors still surface rather than being silently swallowed. Changes: - model_bakery/baker.py: import FieldDoesNotExist from django.core.exceptions; add FK-nested M2M handling loop at the end of bulk_create() with specific exception handling and auto-created through model guard - tests/generic/models.py: add HomeOwner model with ForeignKey to Home, providing the minimal fixture required to reproduce the reported scenario - tests/test_baker.py: add test_create_through_foreign_key_field to TestCreateM2MWhenBulkCreate, reproducing the exact scenario from the issue (baker.make(HomeOwner, home__dogs=[dog], _quantity=10, _bulk_create=True)) Fixes model-bakers#490 Author: Benjamin Aduo (@benaduo)
e0919e0 to
5574857
Compare
amureki
left a comment
There was a problem hiding this comment.
Hey @benaduo !
Thank you for opening this PR and fighting against linters and typing checks. I understand that this experience could be improved and will try to address things there.
Regarding the change, very well done with handling all the different cases and scenarios here, good guardrails!
I noted a couple of things we could improve before moving forward.
| h1, h2 = models.HomeOwner.objects.all()[:2] | ||
| assert list(h1.home.dogs.all()) == list(h2.home.dogs.all()) == [dog] |
There was a problem hiding this comment.
What stops us from doing a full assert here?
| h1, h2 = models.HomeOwner.objects.all()[:2] | |
| assert list(h1.home.dogs.all()) == list(h2.home.dogs.all()) == [dog] | |
| for owner in models.HomeOwner.objects.all(): | |
| assert list(owner.home.dogs.all()) == [dog] |
| for entry in created_entries: | ||
| fk_obj = getattr(entry, fk_field_name, None) | ||
| if fk_obj is not None: | ||
| getattr(fk_obj, m2m_field_name).set(kwarg_value) |
There was a problem hiding this comment.
How about we consider optimizing this via bulk_create?
This would be less readable than a simple set(); however, we clearly reduce the number of queries used.
| for entry in created_entries: | |
| fk_obj = getattr(entry, fk_field_name, None) | |
| if fk_obj is not None: | |
| getattr(fk_obj, m2m_field_name).set(kwarg_value) | |
| through_rows = [] | |
| for entry in created_entries: | |
| fk_obj = getattr(entry, fk_field_name, None) | |
| if fk_obj is not None: | |
| through_rows.extend( | |
| through_model( | |
| **{ | |
| related_m2m.m2m_field_name(): fk_obj, | |
| related_m2m.m2m_reverse_field_name(): obj, | |
| } | |
| ) | |
| for obj in kwarg_value | |
| ) | |
| if through_rows: | |
| through_model.objects.bulk_create(through_rows) |
We can potentially extend the test with self.assertNumQueries to see its stats.
| # set many-to-many relations on FK-related objects specified via double-underscore | ||
| # syntax (e.g. home__dogs=[dog]). During prepare() the M2M values are stored in | ||
| # the sub-baker's m2m_dict but _handle_m2m() is never called because commit=False. | ||
| # _save_related_objs() only persists FK objects without touching M2M, so we must | ||
| # apply them here after all entries have been bulk-created and FK objects saved. |
There was a problem hiding this comment.
Could we revise this comment?
I appreciate you going the extra mile to document things, but it's hard to read here. :(
| # set many-to-many relations on FK-related objects specified via double-underscore | |
| # syntax (e.g. home__dogs=[dog]). During prepare() the M2M values are stored in | |
| # the sub-baker's m2m_dict but _handle_m2m() is never called because commit=False. | |
| # _save_related_objs() only persists FK objects without touching M2M, so we must | |
| # apply them here after all entries have been bulk-created and FK objects saved. | |
| # set M2M on FK-related objects (e.g. `home__dogs=[dog]`) | |
| # `_handle_m2m()` is skipped during `prepare()` since `commit=False`, | |
| # and `_save_related_objs()` only persists FK rows without touching M2M. |
Something like this is more readable?
Summary
baker.make()with_bulk_create=Truenot applying M2M relationshipswhen the M2M field is on a FK-related object, specified via double-underscore
syntax (e.g.
home__dogs=[dog])HomeOwnertest model and a regression test reproducing the exactscenario from the issue
Motivation
When a M2M value is passed as
fk_field__m2m_field=[...], baker stores thevalue in the sub-baker's
m2m_dictduringprepare(), but_handle_m2m()isgated behind
if _commit:and never executes in the bulk_create path._save_related_objs()only saves FK rows without touching M2M, and the twoexisting M2M loops in
bulk_create()only scan the top-level model's ownfields — neither handles the
fk__m2mkwarg pattern. The result is thathome.dogs.set([dog])is never called and the M2M relation stays empty.Changes
model_bakery/baker.py: importFieldDoesNotExistfromdjango.core.exceptions; add FK-nested M2M handling loop at the end ofbulk_create()— parses kwargs forfk_field__m2m_fieldpatterns, validatesfield types, guards against custom through models, and calls
.set()on theFK object for each bulk-created entry
tests/generic/models.py: addHomeOwnermodel withForeignKeytoHometests/test_baker.py: addtest_create_through_foreign_key_fieldtoTestCreateM2MWhenBulkCreateTesting
pytest tests/test_baker.py::TestCreateM2MWhenBulkCreate -vAuthor
Benjamin Aduo (@benaduo)
Summary by CodeRabbit
New Features
Tests