Provide rbind solution that can add list element names as a variable in the output about tidyr HOT 15 CLOSED

So ... in case anyone else finds this in their search for adding a new column with bind_rows (or rbind_list or do.call(rbind, ...) and return a new column without inelegant convulsions... I'll finish out the example to demonstrate:

By

dplyr::bind_rows() should do the equivalent in the next version

What I gather is that (using dplyr 0.5.0) the argument .id can be used to specify the new column name, so following from the example #22 (comment)

bind_rows(my_list, .id = 'species')

returns

     species Sepal.Length Sepal.Width Petal.Length Petal.Width
1     setosa          5.1         3.5          1.4         0.2
2     setosa          4.9         3.0          1.4         0.2
3 versicolor          7.0         3.2          4.7         1.4
4 versicolor          6.4         3.2          4.5         1.5
5  virginica          6.3         3.3          6.0         2.5
6  virginica          5.8         2.7          5.1         1.9

P.S. thanks again for all of the great work you and your team are doing!

from tidyr.

Thank -- yes unnest() does it!

@jennybc can you show in code how this works?

I tried this with throws an error:

> my_list %>% unnest
Error in UseMethod("unnest_") : 
  no applicable method for 'unnest_' applied to an object of class "list"

from tidyr.

The built in method that works for me was to loop through the list and add the item name as a constant column.

for (i in 1:length(mylist)) {
  mylist[[i]]$newvar <- as.numeric(names(mylist[i]))
}

After that you can apply the usual do.call as following:
mylist<- do.call(rbind,mylist)

from tidyr.

No documentation yet, but unnest() should do now what you want.

I'm still struggling if unnest() should work on both lists and columns of data frames that are lists. That might be confusing, but it's basically the same behaviour albeit with slightly different output.

from tidyr.

Thank -- yes unnest() does it!

As a happy user of the grammar that runs through your packages, the words this task evokes for me are rbind and/or gather. Maybe unnest will seem more natural when I've worked with it more.

Morally, this operation seems like gathering variables into key-value pairs, with really different mechanics. Instead of different levels of a factor represented as separate variables in a data.frame, we've got them represented as separate data.frames.

from tidyr.

I think it'll be more obvious why it's called unnest() once I sketch out the other pieces, and when you see what nest() does - they're fundamentally about lists of data frames and vectors, where spread() and gather() deal with columns.

from tidyr.

I have every faith that it will be the most natural thing when you're done. :)

from tidyr.

This is what I would imagine to be the outcome of do.call( rbind.fill, df_list) but that's not what the code actually uses. It's an S3 method that seems to be first an rbind operation for the unmatched columns followed by (v)applying append column-wise matched on the basis of names. I wasn't really sure how the append_df would succeed, since it looked just like base::append with some attribute management (to handle Dates, datetimes, and factors presumably) but there is no append.data.frame. I was expecting some lapply(list, append_df), but it appears to be succeeding nonetheless, so probably it's just my confusion.

from tidyr.

FWIW, I've removed this experimental method because I'm now pretty sure it's a bad idea (and dplyr::bind_rows() should do the equivalent in the next version)

from tidyr.

Hmmm, but maybe unnest() needs to handle the case where a column of the data frame is a list of data frames:

data_frame(
  x = c(1, 2),
  y = c(3, 4),
  z = list(data_frame(a = 1), data_frame(a = 1:3))
)

(and those data frames could contain lists-columns themselves, but I think you'd need a second unnest to handle that)

I'm not certain whether or not this is useful (it might crop up as an alternative way of handling relational data), but it is interesting.

from tidyr.

Oh that's #58

from tidyr.

Is there a solution for the cases when rbind_rows will not accomplish this task if the constituent lists are of different length? In these cases I usually appeal to do.call(rbind, df) but there is obviously no .id argument to go along with that solution...

from tidyr.

@d8aninja Do you want to make a little example that shows exactly what you mean (I'm not entirely sure) and ask it over in the tidyverse section of community.rstudio.com? This question is a good fit.

from tidyr.

@jennybc will do, thanks!

from tidyr.

from tidyr.