Add structured field and rule paths to Violation

[ghost]

This PR introduces a new structured field path format, and uses it to provide a structured path to the field and rule of a violation.

• The new message buf.validate.FieldPathElement is added.
  • It describes a single path segment, e.g. equivalent to a string like repeated_field[1]
  • Both the text name and field number of the field is provided; this allows the field path to be rendered into a string trivially without the need for descriptor lookups, and will work for e.g. unknown fields. (Example: A new field is marked required; old clients can still print the field path, even if they do not have the new field in their schema.)
  • It also contains the kind of field, to make it possible to interpret unknown field values.
  • Finally, it contains a subscript oneof. This contains either a repeated field index or a map key. This is needed because maps in protobuf are unordered. There are multiple map key entries, one for each distinctly encoded valid kind of map key.
• The new message buf.validate.FieldPath is added. It just contains a repeated field of buf.validate.FieldPathElement
  • It would be possible to just have repeated buf.validate.FieldPathElement anywhere a path is needed to save a level of pointer chasing, but it is inconvenient for certain uses, e.g. comparing paths with proto.Equal.
• Two new buf.validate.Violation fields are added: field and rule, both of type buf.validate.FieldPath. The old field_path field is left for now, but deprecated.
• The conformance tests are updated to match the expectations.

Note that there are a number of very subtle edge cases:

• In one specific case, field paths point to oneofs. In this case, the last element of the fieldpath will contain only the field name, set to the name of the oneof. The field number, field type and subscript fields will all be unset. This is only intended to be used for display purposes.
• Only field constraints will output rule paths, because it is a relative path to the FieldConstraints message. (In other cases, constraint_id is always sufficient anyways, but we can change this behavior later.)
• Custom constraints will not contain rule paths, since they don't have a corresponding rule field. (Predefined constraints will contain rule paths, of course.)

Implementations:

• Implement structured field and rule paths, add field and rule values to ValidationErrors protovalidate-go#154
• Implement structured field and rule paths for violations protovalidate-python#217
• Add rule and field value to violations protovalidate-python#224
• Implement structured field and rule paths for violations protovalidate-cc#63
• Add field and rule value to violations protovalidate-cc#64
• Implement structured field and rule paths for violations protovalidate-java#213
• Add field and rule value to violations protovalidate-java#215

[github-actions]

The latest Buf updates on your PR. Results from workflow Buf CI / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	✅ passed	Nov 25, 2024, 5:23 PM

[ghost]

I finished adding rule_path to all violation cases in the conformance test suite, but unfortunately doing so did reveal a mistaken assumption I've been making: I've been thinking that we can just follow the field path and find the rule on its options, but, somewhat obviously in retrospect, the field the violation is on might not be the field that has the rule. Particularly in the repeated.items, map.keys and map.values cases. It might be possible to still work backwards by parsing the rule path and using it to back track from the field path...

[ghost]

JH realized that our idea for handling the unknown map key types was not quite enough so I made a slight tweak: now FieldPathElement doesn't have a special case for sint* types and instead the map key and value protobuf kinds are conveyed directly. This is a little simpler anyways since it closer mirrors what you would want for both the reflection and unknown wire data cases. Adding map value type isn't strictly necessary, but it brings paths referring to primitives inside map values to parity with paths referring to primitives inside messages and repeated fields: it will be possible to display them even if the field is unknown. Seems like it's worth it for 2 bytes of wire data.

[Alfus]

repeated?

[ghost]

FieldPathElement is meant to be logically equivalent to a path segment, e.g. repeated[1] or map["key"]. The subscript contains either a repeated field index or a map key. If this wording is confusing, I'm happy to update the documentation.

WDYT about this? (would "list index" be better?)

Suggested change

	// `subscript` contains a repeated index or map key, if this path element nests into a repeated or map field.
	// `subscript` contains a repeated field index or map key, if this path element nests into a repeated or map field.