Adding update / replace to fmerge about ftools HOT 9 OPEN

Yes. It might involve a bit of Mata work, but I'm a bit pressed on time for the next month or so, so can't promise any update.

For reference (in case you want to try, or maybe for future me), I think it would involve changing line 381:

	// Check that variables don't exist yet
	msg = "{err}merge:  variable %s already exists in master dataset\n"
	for (i=1; i<=cols(deck); i++) {
		var = deck[i]
		if (_st_varindex(var) != .) {
			printf(msg, var)
			exit(108)
		}
	}

Instead of raising an error if the variable exists, when the -update- option is on, you would have to create a tempvar (st_tempvar()?) and then replace row i of varnames_num.

Then, after the Mata code finished running, something like replace original_var = tempvar if mi(original_var)

from ftools.

from ftools.

mmerge has some really useful options (e.g. unmatched [unmatched observations to keep - none, both, master, using], umatch [for the case that variables in using are named different to master], uname [add a stuf to variables in using])

Is there any intent to add additional features to fmerge?

from ftools.

Hi Alan,

Can you explain a bit more what these options do? I installed mmerge from SSC but I'm not entirely sure of what unmatched() does that merge's keep() doesn't.

Regarding umatch() it can actually be done through the join command. I actually wrote fmerge as a wrapper to join (which has a more familiar syntax for me). For instance, suppose you have a panel of consumers (where t is the year identifier) and want to add some macro data from a dataset (where year is the year identifier)

With merge, you do:

rename t year
merge m:1 year using "annual_data", keepusing(gdp inflation)
rename year t

With join, you do:

join gdp inflation, from("annual_data") by(t=year)

(Note how the join syntax looks more like the collapse() one, and is more explicit about which variables get added)

from ftools.

Hi Sergio,

Ignore my message. You're completely correct - it's all possible with the other options. (the main advantage to mmerge is that its a bit more verbose in it's reporting)

I wasn't aware of join... I think i'll be looking into that a bit more - i have some quite large datasets which take merge/mmerge a long time to combine...

Thanks!!

from ftools.

@aghaynes mentioned one which, as far as I can see, join doesn't do and could be potentially useful. The uname option allows adding a stub to the variable names of using data. This makes it easy to distinguish which variables were pre-existing and which are new, maybe for comparison.

from ftools.

Agree, that should be useful and simple to implement. That said, uname() doesn't seem like an esy-to-remember option, so maybe stub(), prefix() or sth like that?

from ftools.

Yes, I agree. Either of these seem fine. stub seems to be commonly used, but I'd say prefix is more intuitive if you've never heard of it.

from ftools.

For what it's worth prefix is the pattern used by frget in Stata 16

from ftools.