RFC: What if `StripeObject` didn't inherit from `dict`?

[xavdid-stripe]

Background

The objects you get back from the Stripe API all share a common base class: StripeObject (source). It holds the raw API response and has helpers for reading, writing, and updating data using a convenient dot notation: obj.name.whatever.

For many years, StripeObject has inherited from dict, which means you get dict-like behavior "for free": len(obj) shows how many keys are in the StripeObject, for key in obj iterates through those keys, etc.

While implementing all of the dict methods confers certain advantages when working with maps of data, there are also drawbacks. Most notably, any API response fields that share a name with a built-in method are inaccessible with dot notation:

o = Subscription() # an API response
o.whatever # works
o.whatever.sub_property # also fine
o.items # builtin method of `dict`
o.items[0] # error, method isn't subscriptable
o['items'] # list of StripeObjects, unintuitive

Over the years, we've gotten many issues filed about this counterintuitive behavior. But, we're hesitant to make breaking changes for behavior that's existed for so long. To better understand the implications of these changes, we're soliciting developer feedback to make sure we're serving your needs well.

Proposed Solution

Our proposal is that StripeObject would not inherit from anything. As a result, it would lose its dict-like behavior (len(obj), obj.items(), {**obj}, json.dumps(obj) etc) but retain property access via dot notation. Most documented examples don't take advantage (or even mention) that StripeObjects are dictionaries, so we're hoping to not lose much functionality here.

In the interest of minimizing the impact of this change, we'd add a StripeObject.as_dict() (name not final) method to return the underlying dictionary and enable all the behavior from before. As a result, the migration to keep existing behavior would be clear and relatively simple: call that method when you need dict-specific functionality.

Impact

As a result of no longer being a dict, any. This includes (but is not limited to):

1. Passing the properties of a StripeObject to a function using some_func(**stripe_obj)
2. Dumping a StripeObject to json: json.dumps(stripe_obj) would need the default=vars kwarg
3. isintance(obj, dict) checks would change behavior, if you have any of those

Alternative Solution

We have an existing mechanism to separate the name of the class property from the API value. We already use this for tax.Registration.CountryOptions.Is (docs) because tax.registration.country_options.is is a SyntaxError; we useis_ instead.

We could do the same thing for subscription.items. .items would continue to be dict.items() and .items_ would be the list of SubscriptionItems. The type annotations would indicate this and the existing obj['items'] approach would continue to work.

We're not thrilled with this because it doesn't match the data you get from the API and it's an extra thing to think about and remember. But, in a world where most users write code with rich typing support, it may not be a big deal. Plus, this wouldn't be a breaking change, which is useful for reducing general churn and toil.

Our Questions for You:

1. What code, if any, do you have that takes advantage of the fact that StripeObject is a dict?
2. If you called some_obj.as_dict() and made modifications to that dictionary, would you expect those changes to be reflected in some_obj?. Does your answer change if the method is called to_dict() instead?
3. Would you prefer that we fixed the items bug using the alternative solution mentioned above and kept the current dict inheritance? Why or why not?

Feel free to provide any other suggestions, comments, or use cases that you think can be helpful. And as always, thank you for helping us make the Python SDK the best it can be!

See Also

• Subscription Items are not retrievable with items-attribute on Subscription object #297
• Are stripe objects dicts? #494
• Subscription Items Not Accessible #504
• There doesn't seem to be a way of robustly relying on library's types for API responses #1344

[Benoss]

I think not extending dict is the way to go. Maybe you should support both behaviors by having a DictClient returning the current behavior and another one maybe using Pydantic models instead.

This would help a lot with autocomplete and data validation via Static analyse but also at runtime before it even calls the API.

This would also probably help to build a mock client for unit testing

[brendanator]

I'm in favour of @Benoss suggestion of having Pydantic models

Another issue that it would solve is that some fields marked as Optional actually might not even be available at all. e.g. Customer.deleted is an optional field

class Customer:
    ...

    deleted: Optional[Literal[True]]
    """
    Always true for a deleted object
    """

But I use if customer.deleted: when the customer is not deleted I get

  File "stripe_hook.py", line 91, in _handle_subscription_upserted
    await self._update_database_subscription(subscription.id, subscription)
  File "stripe_hook.py", line 194, in _update_database_subscription
    if customer.deleted:
  File "$HOME/.local/share/virtualenvs/core-bRkH1-KB/lib/python3.11/site-packages/stripe/_stripe_object.py", line 172, in __getattr__
    raise AttributeError(*err.args) from err
AttributeError: deleted

Failures like this decrease my confidence that my code will just work. I do not know which other fields might go wrong.

---

I strongly support a single breaking change to migrate to using Pydantic models. I expect this would resolve all these issues in a single swipe.

A lot of behaviour is built in to pydantic that supports the use-cases you mention:

Passing the properties of a StripeObject to a function using some_func(**stripe_obj)

Can use some_func(**stripe_obj.model_dump())

Dumping a StripeObject to json

stripe_obj.model_dump_json()

If you called some_obj.as_dict() and made modifications to that dictionary, would you expect those changes to be reflected in some_obj?

I wouldn't expect this to work, but Pydantic can do something better. Dump a dict, make changes, then validate that they are correct:

customer_dict = customer.model_dump()
customer_dict["deleted"] = True
new_customer = Customer.model_validate(**customer_dict)

[mdepinet]

Thank you for working on this!

1. None. I didn't realize StripeObject inherited from dict until I got a TypeError for items.
2. No, but I do think to_dict makes that more obvious.
3. As long as pyright correctly predicts types, I don't have a strong preference. I'd slightly prefer the primary proposal, but only because we're currently not using dict capabilities anyway.

[adambirds]

I hadn't realised there was a seperate thread for the comments and originally posted my comments on #1344. I got the issue after moving to Basil and having to access items for current_period_start. Having to access items by subscription["items"] is causing all my types underneath to be incorrect as of right now making confidence in my code harder.

After reading the above and the various other comments here are my observations:

1. Wasn't aware StripeObject inherited from dict until I had the issue accessing .items.
2. I don't think changes to the dict should change some_obj. Usually you would do something like this:

some_obj = stripe.susbcription.Retrieve("xxxxxxxxx")

dict_obj = some_obj.as_dict() 

or

dict_obj_2 = some_obj.to_dict()

In conclusion whether it is to_dict() or as_dict() they would be seperate objects to the original object stored in a seperate veriable.

3. I would prefer the dot notation to work where possible, especially when thats how accessing the rest of library works. Using .items_ whilst more beneficial than the current situation, just seems unnatural and would end up probably causing a lot of confusion when developers (especially new ones) look at the API docs and see both ["items"]and .items_ which wouldn't match all the other API clients.

I would however also back @Benoss's suggestion of one massive breaking change to using pydantic models instead. This would be a lot more reliable, trust worthy and gives a lot fo great features builtin.

[rminderhoud]

1. What code, if any, do you have that takes advantage of the fact that StripeObject is a dict?

Conveniently serialize stripe event to backend to send to worker process

2. If you called some_obj.as_dict() and made modifications to that dictionary, would you expect those changes to be reflected in some_obj?. Does your answer change if the method is called to_dict() instead?

No, I consider this functionality a one-way serialization step. getattr and setattr can be used in case for need for dynamic key name changes.

3. Would you prefer that we fixed the items bug using the alternative solution mentioned above and kept the current dict inheritance? Why or why not?

Prefer explicit methods for converting to dict vs. dict inheritance.