Skip to content

[csv-loader] Support for custom dynamicTyping option #50

@marceloverdijk

Description

@marceloverdijk

When loading the CSV the papaparse dynamicTyping seems always be set to true, and cannot be overridden:

https://github.com/ascorbic/astro-loaders/blob/main/packages/csv/src/csv-loader.ts#L55C1-L60C8

    const csvStream = Papa.parse(Papa.NODE_STREAM_INPUT, {
      dynamicTyping: true,
      ...parserOptions,
      header: true,
      transformHeader: transformHeader === false ? undefined : transformHeader,
    });

parserOptions?: Omit<
    Papa.ParseConfig,
    "header" | "dynamicTyping" | "transformHeader" | "step" | "complete"
  >;

(same for header and transformHeader, step and complete).

The issue is I have a CSV like:

"name","category","priority"
"foo","A1",1
"bar","A2",2 
"xyz","1",1 

Where category is a string (all values are quoted as well),
but when parsing "1" it still converts it to a number (I believe because of dynamicTyping = true) which result in a terminal error:

AstroError [InvalidContentEntryDataError]: **persons → xyz** data does not match collection schema.
**category**: Expected type `"string"`, received "number"

I think in this case I should configure the csv loader like:

const persons = defineCollection({
  loader: csvLoader({
    fileName: 'data/persons.csv',
    transformHeader: false,
    idField: 'name',
    parserOptions: {
      dynamicTyping: (field) => field !== 'category',
    },
  }),
  schema: z.object({
    name: z.string(),
    category: z.string(),
    priority: z.number().int(),
  }),
});

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions