Skip to content

Runtime Mode does not seem to keep Optional attribute to Recursive Fields #107

@Levak

Description

@Levak

Describe the bug

Using protobuf_to_pydantic in runtime mode (i.e. using msg_to_pydantic_model) yields a class that seems to trigger a validation error when optional message fields are missing. I have not tested with plugin mode as I need a specific version of protobuf.

Dependencies

  • Python 3.8.2
  • protobuf_to_pydantic 0.3.3.1 (not the "all" version, because of my protobuf requirement to be 3.20.3)
  • protobuf 3.20.3
  • pydantic 2.10.6
  • quart-schema 0.20.0

Protobuf File Content

// ------------------ Case 1 -----------------------

message Node {
  optional uint64 id = 1;
  optional Node next = 2;
}
message Result {
  optional int32 err_code = 1;
  repeated Node nodes = 2;
}

// -------------------- Case 2 ---------------------
message ListNode {
  optional uint64 id = 1;
  //optional ListNode next = 2;  // <==== Will cause RecursionError
}

message ResultList {
  optional int32 err_code = 1;
  optional ListNode list = 2;
}

// ------------------ Case 3 -----------------------
message TreeChild {
  optional uint64 id = 1;
  repeated TreeNode elt = 2;
}

message TreeNode {
  optional uint64 value = 1;
  optional TreeChild left = 2;
  optional TreeChild right = 3;
}

message ResultTree {
  optional int32 err_code = 1;
  optional TreeNode tree = 2;
}

Use

from quart import Quart
from quart_schema import (
  QuartSchema, RequestSchemaValidationError, 
  validate_querystring, validate_request, validate_response
)
from protobuf_to_pydantic import msg_to_pydantic_model
import master_pb2
from google.protobuf.json_format import MessageToDict

app = Quart(__name__)
QuartSchema(app)

@app.errorhandler(RequestSchemaValidationError)
async def handle_request_validation_error(error):
  return { "errors": str(error.validation_error) }, 400

## Output 1
@app.route('/get')
@validate_response(msg_to_pydantic_model(master_pb2.Result))
async def get():
  msg = master_pb2.Result()
  node = msg.nodes.add()
  node.id = 54
  return MessageToDict(msg, preserving_proto_field_name=True)

## Output 2
@app.route('/get_list')
@validate_response(msg_to_pydantic_model(master_pb2.ResultList))
async def get_list():
  msg = master_pb2.ResultList()
  return MessageToDict(msg, preserving_proto_field_name=True)

## Output 3
@app.route('/get_tree')
@validate_response(msg_to_pydantic_model(master_pb2.ResultTree))
async def get_tree():
  msg = master_pb2.ResultTree()
  child = msg.tree.left
  child.id = 42
  return MessageToDict(msg, preserving_proto_field_name=True)

Output 1

quart_schema.validation.ResponseSchemaValidationError: 1 validation error for Result
nodes.0.next
  Field required [type=missing, input_value={'id': '54'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing

Output 2

  File "site-packages/protobuf_to_pydantic/get_message_option/from_message_option/base.py", line 106, in get_message_option_dict_from_desc
    message_option_dict["nested"][field.message_type.name] = self.get_message_option_dict_from_desc(
  [Previous line repeated 967 more times]
  File "site-packages/protobuf_to_pydantic/get_message_option/from_message_option/base.py", line 100, in get_message_option_dict_from_desc
    field_info_dict = gen_field_info_dict_from_field_desc(type_name, field.full_name, field, self.protobuf_pkg)
  File "site-packages/protobuf_to_pydantic/field_info_rule/protobuf_option_to_field_info/desc.py", line 101, in gen_field_info_dict_from_field_desc
    if isinstance(field, FieldDescriptor):
  File "site-packages/google/protobuf/descriptor.py", line 66, in __instancecheck__
    if super(DescriptorMetaclass, cls).__instancecheck__(obj):
RecursionError: maximum recursion depth exceeded while calling a Python object

Output 3

quart_schema.validation.ResponseSchemaValidationError: `TreeChild` is not fully defined; you should define `TreeNode`, then call `TreeChild.model_rebuild()`.

Expected behavior

  • Output 1: { "err_code": 0, "nodes": { "id": 42 } }
  • Output 2: { "err_code": 0 }
  • Output 3: { "err_code": 0, "tree": { "left": { "id": 42 } } }

Additional context

I have found this issue because of Case 3. While writing a minimal working example for this Github issue I found the Cases 1 and 2. The real world case I am using is closer to Case 3.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions