The auditor from modeloriented

Update NEWS.md

Update NEWS.md right after all plots are changed.

audit() doesn't work for xgboost

List of diagnostic plots with `plot.modelAudit`

Wouldn't it be cool to handle a vector of diagnostics plots by the plot.modelAudit function.
I.e. if the type argument is longer than a single element then list of plots is returned (more or less like in the plot.lm function)

add orcid to description

travis build error - different version of hnp package

plotTestGQ - draw geom_smooth for two groups separately

p-values in the plots of relevant tests

and statistics

Reach at least 90% coverage

dragons data is outdated

If dragons are used in examples, then link to DALEX2 and use the newest version
if dragons are not used in examples, then do we really need them here?

Possibility of choosing of positive level in classification models audit

Add tests (scores) descriptions

delete weird fitted line from autocorrelation plot

vignette `Introduction into model audit`

delete last circle from Model Ranking Radar polt

plotPrediction - some options to add

To have the y axis show residuals instead of the actual values (y-y_predicted)
Add some smoothed line over the scatter plot (geom_smooth with method = 'loess' would do the job)

R packages related to regression model validation and visualization

Add extra features to selected plot functions

position of score in plotAutocorrelation, plotHalfNormal & plotScaleLocation
different hue of smooth and points on selected plots

Add tests

min 80% coverage

Additional vignette with detailed description of pipelines

A problem with `plotACF`

I am getting strange error


library("titanic")
titanic <- titanic_train[,c("Survived", "Pclass", "Sex", "Age", "SibSp", "Parch", "Fare", "Embarked")]
titanic$Survived <- factor(titanic$Survived)
titanic$Sex <- factor(titanic$Sex)
titanic$Embarked <- factor(titanic$Embarked)
titanic <- na.omit(titanic)
titanic <- titanic[titanic$Embarked != "",]
titanic$Embarked <- factor(titanic$Embarked)
head(titanic)

library("randomForest")
rf_model <- randomForest(Survived ~ .,  data = titanic)
rf_model



library("DALEX2")
predict_fuction <- function(m,x) predict(m, x, type = "prob")[,2]
rf_explain <- explain(rf_model, data = titanic[,-1],
                      y = titanic$Survived == "1", label = "RF",
                      predict_function = predict_fuction)

library(auditor)
rf_audit <- audit(rf_explain)
auditor::plotACF(rf_audit)

Error in `$<-.data.frame`(`*tmp*`, "index", value = c("1", "2", "3", "4",  : 
  replacement has 712 rows, data has 1000

Enter a frame number, or 0 to exit   

1: auditor::plotACF(rf_audit)
2: modelResiduals(object, variable)
3: orderResidualsDF(object, variable, is.df = TRUE)
4: `$<-`(`*tmp*`, "index", value = c("1", "2", "3", "4", "5", "7", "8", "9", "10", "11", "12", "1
5: `$<-.data.frame`(`*tmp*`, "index", value = c("1", "2", "3", "4", "5", "7", "8", "9", "10", "11

convert data to data.frame internally

I've been using auditor with data that were stored in a tibble and this resulted in weird error in plotResidualDensity function. Converting "data" argument to "audit" function solved the issue. Same problem occured in other packages like pdp i guess. I can give more details if you need them.

auditor::audit.explainer()

the integration of auditor and DALEX

use the breakdown data in the vignette

Two models - one plot

Very nice cheatsheet.
What about the possibility of plotting two models in the same chart. Then it will be easier to compare models.

And/or what about lots inspired by ROC curves or lift curves. In most cases they are used for binary classification, but maybe they can be extended to other glm models as well.

halfnorm plot with envelope

https://www.jstatsoft.org/article/view/v081i10

Fix LIFT

function plotLIFT gives different values than plotLift from lift package

audit not always works for factor response variables

Cleanup in arguments of plotting functions

While installing from CRAN version 2.1 is installed

Hi,
it might be not Auditor related error, but I'm not sure, so I will post it here.

I'm installing auditor in following way:
install.packages("auditor")

After installing and running:
library(auditor)
I have following environment:

> sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.5 LTS

Matrix products: default
BLAS: /home/damian/miniconda3/envs/jakbadacdane.pl/lib/R/lib/libRblas.so
LAPACK: /home/damian/miniconda3/envs/jakbadacdane.pl/lib/R/lib/libRlapack.so

locale:
[1] en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RevoUtils_11.0.1     RevoUtilsMath_11.0.0

loaded via a namespace (and not attached):
 [1] gtools_3.8.1       zoo_1.8-3          tidyselect_0.2.4   purrr_0.2.5       
 [5] lattice_0.20-35    haven_1.1.2        carData_3.0-1      colorspace_1.3-2  
 [9] yaml_2.2.0         rlang_0.2.1        pillar_1.3.0       foreign_0.8-71    
[13] glue_1.3.0         RColorBrewer_1.1-2 TTR_0.23-3         readxl_1.1.0      
[17] bindrcpp_0.2.2     factoextra_1.0.5   bindr_0.1.1        plyr_1.8.4        
[21] quantmod_0.4-13    munsell_0.5.0      gtable_0.2.0       cellranger_1.1.0  
[25] zip_1.0.0          caTools_1.17.1.1   tseries_0.10-45    rio_0.5.10        
[29] GGally_1.4.0       forcats_0.3.0      curl_3.2           auditor_0.2.1     
[33] fdrtool_1.2.15     xts_0.11-0         Rcpp_0.12.18       KernSmooth_2.23-15
[37] ROCR_1.0-7         scales_0.5.0       gdata_2.18.0       abind_1.4-5       
[41] gplots_3.0.1       ggplot2_3.0.0      hms_0.4.2          openxlsx_4.1.0    
[45] dplyr_0.7.6        ggrepel_0.8.0      grid_3.5.1         quadprog_1.5-5    
[49] tools_3.5.1        bitops_1.0-6       magrittr_1.5       lazyeval_0.2.1    
[53] tibble_1.4.2       crayon_1.3.4       car_3.0-0          pkgconfig_2.0.1   
[57] MASS_7.3-50        data.table_1.11.4  hnp_1.2-6          assertthat_0.2.0  
[61] reshape_0.8.7      rstudioapi_0.7     plotROC_2.2.1      R6_2.2.2          
[65] rpart_4.1-13       compiler_3.5.1

I believe I should have auditor_3.0.1 or something similar (based on CRAN). What I am doing wrong?

adapt functions to pipeline

audit %>% plot
audit %>% score

plotLIFT - add two legends

Function to calculate length of arrows in `plotModelPCA`

New theme_drwhy for D3 plots

Overlying responses from different models on single plot

Residuals plot
Scale location + GQ score values

data consistency

i accidentally provided wrong y argument (from outside the data.frame specified in data argument), is there any way there could be a check if the data are consistent (y + data and maybe model)?

Examples with DALEX not DALEX2

Since DALEX2 is in sunset phase, use DALEX in vignettes

problem with isFALSE for the CRAN version of auditor

I've got following error

> auditor::plotResidual(audit_rf)
Error in isFALSE(is.df) : could not find function "isFALSE"

devtools::install_github("tidyverse/broom", force=TRUE, ref = "3df7a2d")

modeloriented / auditor Goto Github PK

auditor's People

Contributors

Stargazers

Watchers

Forkers

auditor's Issues

Recommend Projects

Recommend Topics

Recommend Org