disk.frame-fannie-mae-example's Introduction
disk.frame-fannie-mae-example's People
disk.frame-fannie-mae-example's Issues
Issues with Fannie Mae SF loan data
I personally work on this data for research purposes. Interestingly, I found you showed a few lines of codes on this database at the R 2020 conference. But, since you only showed a part of them and I couldn't find the rest anywhere, I wonder would you like to help me with a few challenges that I've encountered.
-
I totally get the idea of Lazy Functions. Therefore, after edit data from a disk.frame, the result needs to be collected to actually process all the functions. However, in my case, the data is still too big to be loaded into memory even after the changes. I want to directly change the data under the disk.frame. So far, what I did is write_disk.frame() at the end of each block. But it does not always work. For (summarise ..) function, the data remain unchanged if I write without (collect). So, is there an easy way to get around with this?
-
After (summarise a=min(b)), I want to perform a left join between two disk.frames. So I ran: X %>%
left_join(Y, by = c("LOAN_ID" = "LOAN_ID", "a" = "b"),merge_by_chunk_id = TRUE) . However, I encountered an error, showing: Error: Join columns must be present in data.
I have checked the colnames of X, those two columns do exist. I am not sure why this happens then. -
Delete Function: I found that even I delete (x.df) the disk space is not freed. Although I could not run things on x again, it still exists in the environment and I need to rm(x.df) again. So what's delete() actually for?
Thank you for reading such a long message. It was a great presentation at the R 2020.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.