Please Share your Product Ideas with us!

All ideas are welcome. Just because the Idea doesn't make it into the product immediately does not make it a bad idea.

Parquet and AVRO Format for AWS S3

Main advantages for customers:

  • Reduces IO operations.

  • Fetches specific columns that you need to access.

  • It consumes less space.

  • Support type-specific encoding.

Currently Syniti puts data in CSV. We've faced following issues because of this

  • extra columns in .mir file than ref. Reading data from text delimited files which do not support schema evolution becomes a tedious task.

  • Unrecognized characters in columns making parsing erroneous.

  • Adding of \r column in the column name itself because of text file creation and handling of such characters by different libraries.

  • Emily Williams
  • Feb 24 2023
  • Attach files