Git Product home page Git Product logo

Comments (1)

k-segiet avatar k-segiet commented on August 30, 2024

Adam's answer:

Thank you for reaching out!

I wrote up some example code to illustrate how this can be done. This takes advantage of the option include_id_reference_dict in BipartitePandas. Unfortunately this means that the data cleaning must be done manually, but it's just a few extra lines of code.

To run this on your own code, you can replace sim_data with your own data, and delete the line that takes the subset of i < 100.

Also note that I used some options I added after this issue was raised on the github, which makes it so it only generates the fixed effects and doesn't estimate the variance/covariances.

Best,
Adam

import bipartitepandas as bpd
import pytwoway as tw
import pandas as pd

#### Simulate data
sim_data = bpd.SimBipartite({'nk': 50, 'num_time': 2, 'num_ind': 1000}).sim_network()
#### Manually clean data
bdf = bpd.BipartiteLong(sim_data, include_id_reference_dict=True) # Set include_id_reference_dict=True to save original ids
#### Subset of data so largest connected set is subset of all firms
bdf = bdf[bdf['i'] < 100]
bdf = bdf.clean_data()
bdf.gen_m()

#### Create TwoWay object
tw_net = tw.TwoWay(bdf.original_ids()) # bdf.original_ids() creates a dataframe with columns that give the original ids
#### Skip data cleaning step in TwoWay object, but mark data as clean
tw_net.clean = True

fe_params = {
'ncore': 1, # Number of cores to use
'batch': 1, # Batch size to send in parallel
'ndraw_pii': 50, # Number of draws to use in approximation for leverages
'levfile': '', # File to load precomputed leverages
'ndraw_tr': 5, # Number of draws to use in approximation for traces
'he': False, # If True, compute heteroskedastic correction
'out': 'res_fe.json', # Outputfile where results are saved
'statsonly': False, # If True, return only basic statistics
'feonly': True, # If True, compute only fixed effects and not variances
'Q': 'cov(alpha, psi)' # Which Q matrix to consider. Options include 'cov(alpha, psi)' and 'cov(psi_t, psi_{t+1})'
}

#### Since we set 'feonly': True, we just run the estimator normally and it only estimates the fixed effects to save time
tw_net.fit_fe(fe_params)

#### Now look at the data
new_data = tw_net.data

I would also recommend setting the following for better performance:

bdf = bdf.clean_data({'data_validity': False})

But also be careful that this isn't designed to work if you are manipulating the data or reformatting the data (for instance from long to event study, etc.) after data cleaning, so you should verify it is working properly in your case before committing to using it.

from pytwoway.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.