strtokm function about bikedata HOT 6 CLOSED

Using boost::replace_all(line, "\"", "") should allow a reversion to strtok. Since we can pass multiple (single) delimiters to strtok, it should work for the last token as well.

I'll have a go and see how I get on.

from bikedata.

I remember now why I initially didn't just use strtok with replace_all. It was because I wasn't sure if any of the data would contain strings with commas (in an address for instance). The Citibike data doesn't seem to have this issue but it is something to keep in mind for other systems.

from bikedata.

One alternative if we want to use strtok together with boost::replace_all would be to use boost::replace_all(line, "\",\"", "?") where instead of the question mark we use an infrequently used character.

from bikedata.

nah, we've still got comma separators, so can't stick anything but black spaces in (with strok). The good news is that all data thus far from global bike systems is ultimately stored in strict .csv format, so comma's are used exclusively as field delimiters and never as punctuation. That means we can definitely rely on comma separators at all times.

i didn't document my previous attempts, but recall that everything worked fine simply removing double quotes up until the line ending, but I couldn't get strok to properly return the final token. The rm_dos_end function was intended to solve it, but still didn't somehow.

from bikedata.

In that case I'll go ahead and try removing any speech marks and use strtok.

from bikedata.

Closing this because strtokm is utlimately still needed because there are cities (such as London) in which station names do have commas, yet are delimited by double quotes. All such cases use strtokm, while all others revert to std::strtok where possible.

from bikedata.