This repo is the demo transaction csv handler.

csv file maybe have a lot additional spaces. So I use csv_parser/clean to clean all space inside. Function handle_transaction_file reads whole file to the string, so when the file is huge. It might be cause some problem.

Ideas

maybe read file line by line, make it like a stream. Except the header, each line is a record, which can used inside src/client. Save memory.

maybe use BufReader for csv file? Maybe not, because BufReader used in the situation small and repeated read calls to the same file.

but, if we have a network socket (tcp connection) of our csv data, we can use it (I guess).

update on 5/27/2022

test_read_stream in src/csv_parser.rs use BufReader run the TcpStream for reading each line of csv tcp stream (without headers). So we can use stream rather than read a big file, beside, stream reading can fit concurrency easily.