Cross-validation

Methods for measuring the performance of a predictive model on sets of test data in Bradley-Terry model from psychotree, Generalized Linear and Generalized Nonlinear models from gnm, and Plackett-Luce model from PlackettLuce

crossvalidation(formula, data, k = 10, folds = NULL, seed = NULL, ...)

# S3 method for class 'bttree'
AIC(object, newdata = NULL, ...)

# S3 method for class 'bttree'
deviance(object, newdata = NULL, ...)

# S3 method for class 'pltree'
deviance(object, newdata = NULL, ...)

# S3 method for class 'gnm'
AIC(object, newdata = NULL, ...)

# S3 method for class 'gnm'
deviance(object, newdata = NULL, ...)

Arguments

formula

an object of class "formula" (or one that can be coerced to that class): a symbolic description of the model to be fitted, of the form \(y ~ x1 + ... + xn\)

data

a data frame (or object coercible by as.data.frame to a data frame) containing the variables in the model

k

an integer for the number of bins in the cross-validation

folds

an optional vector or list of vectors specifying the k-folds in the cross-validation

seed

integer, the seed for random number generation. If NULL (the default), gosset will set the seed randomly

...

additional arguments passed the methods of the chosen model

object

a model object

newdata

a data.frame with test data

Value

an object of class gosset_cv with the cross-validation goodness-of-fit estimates, which are:

AIC

Akaike Information Criterion

deviance

Model deviance

logLik

Log-Likelihood

MaxLik

Maximum likelihood pseudo R-squared

CraggUhler

Cragg and Uhler's pseudo R-squared

McFadden

McFadden pseudo R-squared

kendallTau

the Kendall correlation coefficient

References

Elder J. F. (2003). Journal of Computational and Graphical Statistics, 12(4), 853–864. doi:10.1198/1061860032733

James G., et al. (2013). doi:10.1007/978-1-4614-7138-7

Whitlock M. C. (2005). Journal of Evolutionary Biology, 18(5), 1368–1373. doi:10.1111/j.1420-9101.2005.00917.x

See also

bttree, gnm, pltree

Other model selection functions: btpermute()

Author

Kauê de Sousa, Jacob van Etten and David Brown

Examples

# \donttest{
# Generalized Linear Models
if (require("gnm")) {
data("airquality")

cv = crossvalidation(Temp ~ Wind + Solar.R,
                      data = airquality,
                      k = 3,
                      seed = 999,
                      family = poisson())
}                    
# Plackett-Luce Model
if(require("PlackettLuce")) {
# beans data from PlackettLuce
data("beans", package = "PlackettLuce")

G = rank_tricot(data = beans,
                 items = c(1:3),
                 input = c(4:5),
                 additional.rank = beans[c(6:8)],
                 group = TRUE)

beans = cbind(G, beans)

# take seasons as bins
k = length(unique(beans$season))
folds = as.integer(as.factor(beans$season))

cv = crossvalidation(G ~ maxTN,
                      data = beans,
                      k = k,
                      folds = folds,
                      minsize = 100)
}
# }