Please Share your Product Ideas with us!

hi Team,

Currently Syniti puts data in CSV. We've faced following issues because of this

1. Extra columns in .mir file than ref. Reading data from text delimited files which do not support schema evolution becomes a tedious task

2. Unrecognized characters in columns making parsing erroneous.

3. Adding of \r column in the column name itself because of text file creation and handling of such characters by different libraries

The solution to it would be to use Parquet as the target data format for Syniti output.