Skip to content

QuickParquetWrite

Cinchoo edited this page Jun 24, 2020 · 2 revisions

Writing Parquet file

To generate Parquet file, use the ChoParquetWriter component to generate it. Sample below shows how to create Parquet file (Emp.parquet)

1,Mark
2,Jason

Write list of objects

List<ExpandoObject> objs = new List<ExpandoObject>();
dynamic rec1 = new ExpandoObject();
rec1.Id = 1;
rec1.Name = "Mark";
objs.Add(rec1);
 
dynamic rec2 = new ExpandoObject();
rec2.Id = 2;
rec2.Name = "Jason";
objs.Add(rec2);
 
using (var parser = new ChoParquetWriter("Emp.parquet"))
{
    parser.Write(objs);
}

Write each object

using (var parser = new ChoParquetWriter("Emp.parquet"))
{
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 1;
    rec1.Name = "Mark";
    parser.Write(item);

    dynamic rec1 = new ExpandoObject();
    rec1.Id = 2;
    rec1.Name = "Jason";
    parser.Write(item);
}

Write using POCO object

public class Employee
{
    public int Id { get; set; }
    public string Name { get; set; }
}
List<Employee> objs = new List<Employee>();
objs.Add(new Employee() { Id = 1, Name = "Tom" });
objs.Add(new Employee() { Id = 2, Name = "Mark" });

using (var parser = new ChoParquetWriter<Employee>("Emp.parquet"))
{
    parser.Write(objs);
}

Please visit below article for detailed walk-through of Parquet writer

Cinchoo ETL - Parquet Writer

Clone this wiki locally