Ollama is inserting "commas" as punctuation in it's CSV output about autocrew HOT 2 CLOSED

I switched to using ", (double quote followed by comma) as the delimiter always, and getting the LLM to always double quote each value in the CSV. Then strip the double quotes from each value, which leaves you with clean values, unless a value itself contains ",. This will always be a problem unless the LLM can properly escape the delimiters and double quotes inside each value.

from autocrew.

In the original script, the parse_csv_data function splits the lines of the CSV data using the delimiter (, in this case). However, this approach fails when Ollama inserts "," within the quoted areas, as it incorrectly splits the values.

To handle this scenario, we can use the csv module provided by Python. This module has a reader function that can parse CSV data while taking into account the quoting rules.

In the modified parse_csv_data function, we import the csv module and create a csv_data object using the csv.reader function. We pass the response string and the delimiter (, in this case) to the reader function.

Next, we convert the csv_data object to a list of lines using list(csv_data). This allows us to access individual lines of the CSV data.

We then process each line of the CSV data. For each line, we iterate over the values and the corresponding header indices. We use the header indices to map the values to the correct header names.

To handle the double quotes within the values, we use the strip('"') method to remove the leading and trailing quotes from each value.

Finally, we append the agent data to the agents_data list and return it.

With these modifications, the parse_csv_data function will correctly handle the scenario where Ollama inserts "," within the quoted areas in the CSV output.

from autocrew.