-
Notifications
You must be signed in to change notification settings - Fork 159
Open
Description
Hi, I believe i discovered a bug in table-pivot.
To reproduce, building off of table_pivot test:
from frictionless import Pipeline, Resource, steps
data = [
{"region": "east", "gender": "boy", "style": "tee", "units": 12},
{"region": "east", "gender": "boy", "style": "golf", "units": 14},
{"region": "east", "gender": "boy", "style": "fancy", "units": 7},
{"region": "east", "gender": "girl", "style": "tee", "units": 3},
{"region": "east", "gender": "girl", "style": "golf", "units": 8},
{"region": "east", "gender": "girl", "style": "fancy", "units": 18},
{"region": "west", "gender": "boy", "style": "tee", "units": 12},
{"region": "west", "gender": "boy", "style": "golf", "units": 15},
]
source = Resource(data=data)
pipeline = Pipeline(
steps=[
steps.table_normalize(),
steps.table_pivot(f1="region", f2="gender", f3="units", aggfun=sum),
]
)
# This is OK
target = source.transform(pipeline)
print(target.to_view())
json_descriptor = """
{ "steps":
[
{"type": "table-normalize"},
{
"type": "table-pivot",
"f1": "region",
"f2": "gender",
"f3": "units",
"aggfun": "sum"
}
]
}"""
source2 = Resource(data=data)
pipeline2 = Pipeline.from_descriptor(json.loads(json_descriptor))
# This fails
target2 = source2.transform(pipeline2)
# This also fails
pipeline2 = Pipeline.from_descriptor(json.loads(pipeline.to_json()))This fails with,
frictionless.exception.FrictionlessException: [step-error] Step is not valid: "table_pivot" raises "[source-error] The data source has not supported or has inconsistent contents: 'str' object is not callable "
In table_pivot aggfun is passed directly to petl, there's no check and conversion for it the parameter is a string. I would have also thought Pipeline.from_descriptor would contain the logic but it fails to export to json as well.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels