spread causes system to run out of memory about tidyr HOT 4 CLOSED

Just a quick note that I'm having memory issues with spread(..., drop=FALSE). If I use spread(..., drop=TRUE) then everything works out fine, the process takes just a few seconds, and the result is of size 0.2Mb.

My input dataset is 0.4MB, has 6000 rows, and 11 variables. This is the result of a filter on a dataset which is of size 200Mb. When running with spread(..., drop=FALSE), the rsession memory expands to over 20Gb.

Unfortunately I can't provide the exact dataset, but if there is anything I can provide to help, I'll be happy to do so.

from tidyr.

I have not. It might be possible to replace the vectorised R code with optimised C++ code that would need less memory.

from tidyr.

How many unique values are there in the variables that you are spreading? It is easy to create very very large data frames with spread.

from tidyr.

There are some numeric variables with a few thousand unique values, but isn't spread just going to make a variable for each key? Also, by virtue of spread(..., drop=TRUE) working fine, the only variables remaining to spread only have one value: NA.

from tidyr.