wayfair / pylift Goto Github PK
View Code? Open in Web Editor NEWUplift modeling package.
Home Page: http://pylift.readthedocs.io
License: BSD 2-Clause "Simplified" License
Uplift modeling package.
Home Page: http://pylift.readthedocs.io
License: BSD 2-Clause "Simplified" License
up.transformed_y_train # The predicted uplift.
This is not the predicted uplift, but just the transformed outcome.
We have some EDA functions, up.NIV
and up.NWOE
, but no docs on them.
Docs should specify this, or this should be fixed: if a (-1,1) array is given, a plot is still produced, but it's wrong.
Thanks for the great package. I got a model on a set of features, a conversion column (0, 1), and a treatment indicator (0, 1). I am interested in plotting a decile of predicted vs actual treatment effect for hold-out set.
I am struggling to understand how I can calculate the predicted and the actual treatment effect based on the model. Any pointers?
From my understanding, the actual average treatment effect (ATE) can be calculated by the difference in mean outcomes of treatment - control. How can I get predicted average treatment effect? My hunch is that I can take the average of the uplift score per decile. Is that correct?
Thanks in advance!
New to transformed outcome method but found the package super interesting!
How does the method work with imbalanced outcome data though? In many marketing use cases, the proportion of customers who buy tend to be very low, so transformed y^{*} will be 0 most of the time. How does that affect the model?
Can we possibly take a resampling approach for training data? However, if we sample a, say 1:1 balanced outcome dataset, how should we estimate the treatment policy p?
Would love to hear your thoughts on if pylift
can consider such data sets.
How to show custom labels instead of showing "Fraction of Data"
counts_dict = {}
y = df_new[col_treatment]
trt = df_new[col_outcome]
for feat in feats:
bin_feat = str(feat) + '_bin'
counts1_t1 = df_new[(y == 1) & (trt == 1)][[feat, bin_feat]].groupby(bin_feat).count().rename(
columns={feat: 'counts_y1t1'})
counts1_t0 = df_new[(y == 1) & (trt == 0)][[feat, bin_feat]].groupby(bin_feat).count().rename(
columns={feat: 'counts_y1t0'})
counts0_t1 = df_new[(y == 0) & (trt == 1)][[feat, bin_feat]].groupby(bin_feat).count().rename(
columns={feat: 'counts_y0t1'})
counts0_t0 = df_new[(y == 0) & (trt == 0)][[feat, bin_feat]].groupby(bin_feat).count().rename(
columns={feat: 'counts_y0t0'})
# creating a dataframe with all of these results
counts_dict[feat] = pd.concat([counts1_t1, counts1_t0, counts0_t1, counts0_t0], axis=1).fillna(
0) + 1 # replace any empty slots with zeros (and add 1 to everything)
should be
counts_dict = {}
y = df_new[col_outcome]
trt = df_new[col_treatment]
for feat in feats:
bin_feat = str(feat) + '_bin'
counts1_t1 = df_new[(y == 1) & (trt == 1)][[feat, bin_feat]].groupby(bin_feat).count().rename(
columns={feat: 'counts_y1t1'})
counts1_t0 = df_new[(y == 1) & (trt == 0)][[feat, bin_feat]].groupby(bin_feat).count().rename(
columns={feat: 'counts_y1t0'})
counts0_t1 = df_new[(y == 0) & (trt == 1)][[feat, bin_feat]].groupby(bin_feat).count().rename(
columns={feat: 'counts_y0t1'})
counts0_t0 = df_new[(y == 0) & (trt == 0)][[feat, bin_feat]].groupby(bin_feat).count().rename(
columns={feat: 'counts_y0t0'})
# creating a dataframe with all of these results
counts_dict[feat] = pd.concat([counts1_t1, counts1_t0, counts0_t1, counts0_t0], axis=1).fillna(
0) + 1 # replace any empty slots with zeros (and add 1 to everything)
Hi,
I was hoping you guys could please help me!
My dataset is around 270k rows with around 60 variables, its a 60mb file, when I try to call NIV, I run into memory error. To tackle this, I have ran NIV on a sample of around 180k max with success, I then refer to the sample's NIV dictionary and select all variables with a NIV higher than 0.03 for example. I then select these variables from my 270k dataset and build my model on this.
However, my cumulative gains plot always ends up with a negative correlation and cgains line plotted below the random selection line.
My guess is this problem is as a result of two possible things:
Possible solutions/questions I have:
Are there any other solutions you guys would recommend? Apologies - I am fairly new to machine learning in general so still learning alot!
Many thanks for your help in advance, its much appreciated!
Khaashif
I am using the Pylift module in an AWS-EC2 linux instance with the code below and getting 2 different errors
up = TransformedOutcome(df_fil, col_treatment='Treatment',col_outcome='Outcome',col_policy='prop_scores', stratify=df_fil['Treatment'],sklearn_model = XGBClassifier)
param_grid = {#'estimator': XGBClassifier(), 'param_grid': {'max_depth': range(1,8,1) 'learning_rate':[x/100 for x in range(1,12,4)], 'colsample_bytree':[x/10 for x in range(3,10,1)], 'min_child_weight':range(1,6,1), 'scale_pos_weight':[x/10 for x in range(12,18,1)], },'n_jobs' : -1}
up.grid_search(**param_grid,cv=2)
Getting the following error while using the above code
TerminatedWorkerError Traceback (most recent call last)
in
----> 1 up.grid_search(**param_grid,cv=2)
~/anaconda3/lib/python3.7/site-packages/pylift/methods/base.py in grid_search(self, **kwargs)
337 self.grid_search_params.update(kwargs)
338 self.grid_search_ = GridSearchCV(**self.grid_search_params)
--> 339 self.grid_search_.fit(self.x_train, self.transformed_y_train)
340 return self.grid_search_
341
~/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/search.py in fit(self, X, y, groups, **fit_params)
685 return results
686
--> 687 self.run_search(evaluate_candidates)
688
689 # For multi-metric evaluation, store the best_index, best_params and
~/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_search.py in _run_search(self, evaluate_candidates)
1146 def _run_search(self, evaluate_candidates):
1147 """Search all candidates in param_grid"""
-> 1148 evaluate_candidates(ParameterGrid(self.param_grid))
1149
1150
~/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_search.py in evaluate_candidates(candidate_params)
664 for parameters, (train, test)
665 in product(candidate_params,
--> 666 cv.split(X, y, groups)))
667
668 if len(out) < 1:
~/anaconda3/lib/python3.7/site-packages/joblib/parallel.py in call(self, iterable)
932
933 with self._backend.retrieval_context():
--> 934 self.retrieve()
935 # Make sure that we get a last message telling us we are done
936 elapsed_time = time.time() - self._start_time
~/anaconda3/lib/python3.7/site-packages/joblib/parallel.py in retrieve(self)
831 try:
832 if getattr(self._backend, 'supports_timeout', False):
--> 833 self._output.extend(job.get(timeout=self.timeout))
834 else:
835 self._output.extend(job.get())
~/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py in wrap_future_result(future, timeout)
519 AsyncResults.get from multiprocessing."""
520 try:
--> 521 return future.result(timeout=timeout)
522 except LokyTimeoutError:
523 raise TimeoutError()
~/anaconda3/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)
430 raise CancelledError()
431 elif self._state == FINISHED:
--> 432 return self.__get_result()
433 else:
434 raise TimeoutError()
~/anaconda3/lib/python3.7/concurrent/futures/_base.py in __get_result(self)
382 def __get_result(self):
383 if self._exception:
--> 384 raise self._exception
385 else:
386 return self._result
TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker. The exit codes of the workers are {SIGABRT(-6)}`
When I remove n_jobs=-1 from the param_grid i.e with the code below
param_grid = {#'estimator': XGBClassifier(), 'param_grid': {'max_depth': range(1,8,1) 'learning_rate':[x/100 for x in range(1,12,4)], 'colsample_bytree':[x/10 for x in range(3,10,1)], 'min_child_weight':range(1,6,1), 'scale_pos_weight':[x/10 for x in range(12,18,1)], }}
up.grid_search(**param_grid,cv=2)
I am getting the following error
terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc
I am using Jupyter notebook to run the above mentioned code, I know it can't be a memory error, cause I am having ample of memory i.e. 64 GB with 8 cores and using Python3.7 anaconda distribution
Are there any descriptions in the docs for scoring new data?
Hi,
I have a question. In your documentation you claim that qini curve is defined as:
nt,1 - nc,1 * Nt / Nc. But your implemented function looks like: nt1o1/Nt - nt0o1/Nc.
It looks like there is an indifference between your documentation and code.
Thanks for your help!
Thank you for publishing this! I was not sure I entirely followed from the documentation, does the class essentially:
Otherwise, what i was curious about, once transformed, is the model fit exactly the same as the base regressor (meaning no change to the fit or objective function)?
Also I was curious, is the same method applicable when the target is not binary but is perhaps something like revenue?
This module runs well on python 3.6x. & 3.7.
in terms of treatment and control group split , under what circumstances is adjusted qini recommended for model evaluation, compared to qini?
If I have a control group which is ~2% of the overall data compared to ~98% treatment group (i.e., p=0.98), should the training data be balance such that it will have a 50/50 split between control and treatment?
Otherwise, the negative Inverse probability weight Multiplier for the control group (-1/(1-p) = -50) will be much larger than the positive Inverse probability weight Multiplier for the treatment group (1/p = 1.02).
I have experiment data with multiple treatment groups. How do I include this model ?
Hello, I've been looking at the uplift model recently. I have the following questions about pylift model. Please reply.
Feature screening: how do I need to screen features? How many features are more suitable for molding?
Model evaluation: which index of the model is used to evaluate the effect, and how many general indexes can be considered as the model can be used?
Model export and model loading, up model. save_ Model ('. / AA. Pkl') export the model like this? What about the way to load the model?
Some good functionalities of the package are not sufficiently documented on https://pylift.readthedocs.io/, for example
TransformedOutcome
object with a tuple of custom training and testing dataframes (very neat flexibility) is not documentedHello,
I've setup my model via TransformedOutcome, then wanted to check all the NIV features. However, when I use NIV (dict, or plot) all the included features are empty. The rest of the steps work and I get plots further down the line, but the NIV being empty leads me to believe they are incorrect.
up = TransformedOutcome(df, col_treatment='Response', col_outcome='TotalRevenueFoodItems',random_state=4, stratify=y)
up.NIV_dict
Thank you.
I have a few questions related to working with continuous target variable, please let me know if this is not the right place for my questions because they donβt directly relate to the actual code.
Based on t Athey, S., & Imbens, G. W. (2015) paper, the transformed outcome (Y^) is the
Conditional Average Treatment Effect (CART), aka the uplift, for a given Xi. Therefore, if the original outcome column, Y, is continues (e.g., net sale, earning, etc.), then transforming it to Y^, fitting a regression model (e.g., Linear regression, Boosted Decision Tree Regression, etc.), and predicting for a given Xi, the model output can be interpreted as the uplift for a given Xi in the original scale (e.g., uplift in net sale for a given Xi, uplift in earning for a given Xi, etc.). Is this correct?
Now if we decide to apply a data transformation to the original continues outcome column (Only for the samples corresponding with response because the continuous values for non-response are equal to zero), fit a regression model and then predict for a given Xi, does it mean that we have to apply the inverse data transformation for the model output/prediction for it to represent the uplift in the original scale?
In general, can positive predictions be explained as positive uplift compared to negative predictions which indicates negative uplift? or should the model prediction be only perceived as a score/ranking index?
I would recommend improving the documentations regarding continuous outcome and adding an example.
Description of the scoring_method and scoring_cutoff exists in the docstrings of derivatives.py but not in base.py.
In addition the default values for scoring_method is inconsistent in both files. I will create a pull Request once I have a chance.
See PR #32
hello, I have two questions and hope that you can help me.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
This repository currently has no open or pending branches.
requirements.txt
matplotlib >=2.1.0
numpy >=1.13.3
scikit-learn >=0.19.1
scipy >=1.0.0
seaborn >=0.7.1
xgboost >=0.6a2
Hi, I have two questions.
Question 1: Considering the following scenario:
I have a dataset where my treatment is given to 94% of the total base so the control is just 6%. Is it the right to approach to use this imbalanced data into the model with specifying the p=0.94 in the TransformedOutcome method?
Or if a sampling is needed, how to deal with the validation data? Should that also be sampled?
Question 2:
I am using the trained model on a hold out data and I have the uplift scores. But I am getting a huge chunk of customers getting exactly the same uplift score. Any thoughts on why this occurs?
Please advice.
π This repository is not currently configured for Renovate. This issue proposes the steps necessary to add Renovate to this project!
π‘ Not familiar with Renovate, or are confused about what advantages it holds over GitHub's Dependabot? Learn more here!
renovate[bot]
account has already auto-filed a Configure Renovate
PR against this repository, feel free to reference the proposed changes in your own Pull Request. If you are contributing to this project as a Hacktoberfest participant, you must file your own PR in order to get credit for your contribution!.github/workflows/lint.yml
). See here for an example..github/dependabot.yml
file. See here for an example.renovate[bot]
account has already auto-filed a Configure Renovate
PR in this repository, I confirm that I will create a separate PR under my own GitHub account, using the initial PR as inspiration.A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.