frendsplatform / frends.csv Goto Github PK

Update to 30.0.1 so it match Frends.Community.Apache

Converting XML to CSV is common integration scenario and this is also promised at project description: "...as well as creating CSV and fixed-width files from JSON and XML inputs."

Issues:

• JSON input fails with properties that have null as value
• JSON input removes decimals from double 0.0 (0.0 --> 0)
• string values with whitespaces are inside quotes in result.
• Date time fields are changed to US type dd/MM/yyyy ((02/05/2019 00:00:00)
  example:
  [{
  "double":0.0,
  "thisFails": null,
  "iGetDoubleQuotes" : " i am claustrophobic "
  "DateTime" : "2018-11-22T10:30:55"
  }]
  CSV result (when null value removed):
  double;iGetDoubleQuotes;DateTime
  0;" i am claustrophobic ";11/22/2018 00:00:00

When dealing with malformed CSV (see example below), where some rows are not complete currently the task throws an exception about missing field values. This is easily recoverable problem and our underlying library has a config option for dealing with this - MissingFieldFound delegate for handling this case.

To be done: add a boolean option to CSV Parse task called TreatMissingFieldsAsNulls and property configure the parser. In output we should produce all the missing fields as null values.

NB! Default value should remain false to keep the backwards compatibility!

header1,    header2,    header3
value1,     value2,    value3
value1,     value2,    value3
value1,     value2,    value3
value1,     value2,    value3
value1,     value2,    value3
value1,     value2,    value3
value1,     value2
value1,     value2,    value3
value1,     value2,    value3

Numbers Are used in column names if source csv does not have headers which gives errors like:
Name cannot begin with the '0' character, hexadecimal value 0x30

Columns could be named for example Column1,Column2 etc instead of 0,1,2

The option 'Trim output' works incorrectly. When setting 'Trim output' to true, whitespace is not trimmed from the values. Setting it to false causes whitespace to be trimmed. This seems backwards and is also a regression from version 0.8.11.

If we return an object instead of JToken we can access the Json with '.' notation e.g.
#result.ToJson()[0].user
instead of
#result.ToJson()[0]["user"]

I have this csv I need to parse and it has a single quotation mark in single value on one row like this:
value;foobar 5x5";value

I get this error

Is is possible to add a option to the task to ignore quotes?

Frends.Community.Apache uses a newer version of CsvHelper (27.1.1 vs 12.2.2).

The version used by Frends.Csv could be updated

ToXml should either Trim header values when creating elements, or replace white spaces with '_' or some user defined option.

String.Replace( ) method here only identifies headers containing space characters. There could be plenty of other whitespace characters, which go unidentified now.

What about header column containing zero characters? It might be handy to replace those, too.

Example for csv parse