imbalanced.rfsrc.RdFits a random forest for a two-class outcome using the random forest quantile classifier (RFQ), balanced random forest (BRF), or standard classification. RFQ changes the probability threshold used to assign class labels; BRF changes the sampling used to grow the trees. The splitting rule, performance measure, and variable importance can be specified separately.
Companion helpers summarize performance, select a probability threshold, or extract the class-frequency threshold from response labels.
# S3 method for class 'rfsrc'
imbalanced(formula, data, ntree = 3000,
method = c("rfq", "brf", "standard"), splitrule = "auc",
perf.type = NULL, block.size = NULL, fast = FALSE,
ratio = NULL, ...)
get.imbalanced.performance(obj, prob = NULL, threshold = NULL,
confusion = FALSE, robust = FALSE)
get.imbalanced.optimize(obj, prob = NULL, newdata = NULL,
measure = c("gmean", "F1", "F1mod", "F1modgmean"),
ngrid = 1000, plot.it = TRUE)
get.rfq.threshold(y)
# S3 method for class 'imbalanced.performance'
print(x, digits = 4,
show.confusion = TRUE, ...)A model formula with one factor response having two levels and one or more predictors.
A data frame containing the response and predictors. Data-frame subclasses are converted to a plain data frame.
Number of trees to grow. The default is 3000.
Classification method. "rfq", the default, uses
the RFQ decision rule of O'Brien and Ishwaran (2019).
"brf" uses class-weighted sampling with replacement and a
reduced tree sample size. "standard" uses ordinary random
forest classification without enabling the RFQ rule. See Details.
Splitting rule passed to the forest grow function.
The default is "auc". Other choices include "gini"
and "entropy"; see rfsrc for supported
classification rules. This setting is separate from
perf.type.
Performance measure used by the forest and its
variable importance calculations. The default is "gmean"
for RFQ and BRF, and "default" (misclassification error) for
standard classification. Choices accepted by this wrapper are
"none", "default", "standard",
"misclass", "brier", "gmean", and the legacy
spelling "g.mean". The latter is converted to
"gmean". Use "none" to omit forest performance
calculations.
Block size passed to the selected forest grow
function for performance and VIMP calculations; see
rfsrc. This wrapper passes NULL explicitly
when the argument is not set. Specify the same block size when
comparing VIMP across methods.
Logical. Use rfsrc.fast for the RFQ or
standard method when TRUE. The BRF branch always calls
rfsrc and does not use this argument.
Optional, experimental majority-class undersampling for the RFQ and standard methods. Supply a number between zero and one. The majority class is sampled without replacement, while the minority class is bootstrapped with replacement. Small requested fractions may be increased by the sampling helper. This is a majority-class sampling fraction, rather than a requested final minority prevalence. Ignored for BRF. See Details for OOB support.
For the performance and optimization helpers, a grow or
prediction object with a single two-class outcome and observed
responses, or a two-level response factor when prob is
supplied.
Optional probabilities aligned with the responses in
obj. A numeric vector or one-column matrix or data frame
contains minority-class probabilities. A two-column matrix or data
frame contains both class probabilities; named columns are matched
to factor levels, and unnamed columns follow factor-level order.
NULL uses the object's predictions as described in Details.
Cutoff for minority-class probabilities in
get.imbalanced.performance. NULL, the default, uses
the minority proportion among the observed evaluation responses.
"auto" uses that prevalence for an RFQ object and \(0.5\)
otherwise. "bayes" always uses \(0.5\). A single finite
numeric value specifies the cutoff directly; values outside
\([0,1]\) are clipped to that interval. With direct response and
probability inputs, "auto" uses \(0.5\) because no fitted
classifier is supplied.
Return a list including the confusion matrix instead of the default named vector of performance measures?
Add one to the diagonal counts when calculating the four classification rates and their derived summaries? See Details.
Optional threshold-selection data for
get.imbalanced.optimize, supplied with a grow object and
prob = NULL. Must contain the predictors and observed
response. NULL uses predictions already in obj.
Ignored when prob is supplied.
Performance measure to maximize during threshold
selection: "gmean" (the default), "F1",
"F1mod", or "F1modgmean". See Details for definitions.
Number of equally spaced thresholds from zero to one, including both endpoints. Must be an integer of at least two.
Draw the four threshold-search diagnostic panels?
Class-label vector for get.rfq.threshold, typically a
two-level factor. Missing responses are omitted when counting classes.
Additional arguments passed to rfsrc or,
when selected, rfsrc.fast. Examples include
mtry, nodesize, nsplit, importance, and
na.action. BRF sets its own case.wt, sampsize,
and samptype. When ratio is supplied, the wrapper
constructs samp and sets bootstrap = "by.user".
Additional arguments to the print method are currently unused.
For the print method, the result of
get.imbalanced.performance.
Number of decimal places used in the performance display.
Print model and expected random-reference
confusion tables when a confusion matrix is present in x?
Obtain that matrix with confusion = TRUE in
get.imbalanced.performance. This option does not recompute it.
Let \(\pi\) be the training proportion of the minority class and
\(\widehat p(x)\) its predicted probability. RFQ assigns
an observation to the minority class when
\(\widehat p(x) \geq \pi\). The class-frequency
threshold can identify minority observations whose predicted
probability is below one half. The standard method instead uses
ordinary class assignment based on the largest predicted probability.
RFQ here is a classification rule, distinct from the continuous-response
quantile estimation performed by quantreg.
BRF calls the grow function with replacement sampling, a tree sample
size of twice the smaller observed class count, and case weights that
give the two classes equal total sampling weight. The wrapper does
not construct separate fixed-size samples from each class. Both
fast and ratio are ignored in this branch.
G-mean is the geometric mean of minority sensitivity and majority
specificity, \(\sqrt{\mathrm{sensitivity}\,
\mathrm{specificity}}\). Larger
values indicate better balance between the two class-specific rates.
Misclassification and Brier scores are error measures for which smaller
values are better. Changing perf.type selects the forest
performance measure; it does not select the RFQ decision rule.
Request permutation VIMP explicitly with
importance = "permute". For comparisons, use a common
perf.type, splitting rule, and block.size, unless the
comparison is intended to study those settings. Increase ntree
as needed to assess the stability of OOB performance and VIMP.
With ratio = NULL, the RFQ and standard branches pass
missing-data handling to the selected grow function. BRF and calls
using ratio first apply complete-case omission to the supplied
data frame. This omission currently includes columns that are not
selected by the formula and occurs before the grow function handles
na.action.
With ratio supplied, each tree uses a bootstrap sample of the
minority observations and a without-replacement sample of majority
observations. At ratio = 1, every majority observation is
in-bag for every tree, so majority-class OOB performance is unavailable.
A single minority observation is also always in-bag in this sampling
scheme. Use a separate evaluation sample when OOB support is absent.
get.imbalanced.performance calculates performance from
observed responses and predicted probabilities.
Supply a grow or prediction object, or
a two-level response factor with prob. With
prob = NULL, the helper uses predicted.oob when
present and not entirely missing; otherwise it uses
predicted. When only some OOB predictions are missing,
those entries remain missing.
A numeric vector or one-column matrix or data frame supplies minority-class probabilities; no column name is required. Two-column inputs supply both class probabilities; named columns are matched to factor levels, and unnamed columns follow factor-level order. Rows must align with the responses.
All observed evaluation responses determine class counts, the minority class, and its proportion, before excluding unavailable predictions. The first factor level is the minority when counts tie.
Finite probabilities below zero are set
to zero, and those above one are set to one. Finite two-column
rows must sum to one within numerical tolerance. Scoring excludes
rows with missing responses or nonfinite probabilities; both
columns must be finite for two-column inputs. Unavailable scores
are NA.
An observation is assigned to the
minority class when its predicted minority-class probability is
at least the threshold, and to the majority class otherwise.
The threshold choices are:
NULLThe evaluation minority proportion for every method (default).
"auto"The evaluation minority proportion when
forest$rfq is TRUE; \(0.5\) otherwise,
including direct response/probability inputs.
"bayes"\(0.5\) for every method.
A single finite cutoff. Values below zero are set to zero, and values above one are set to one.
For RFQ test predictions, NULL and "auto" use
evaluation prevalence, not training prevalence. The RFQ flag,
rather than splitrule or perf.type, determines
the "auto" choice. Changing the threshold preserves
the evaluation minority definition and the fitted forest.
The summary includes sensitivity
(sens), specificity (spec), precision (prec),
negative predictive value (npv), G-mean, misclass,
brier, brier.norm, auc, logloss, and
pr.auc. For complementary binary probabilities,
brier is the mean squared error of the minority probability;
brier.norm is four times this value. Log loss excludes
infinite losses without clipping zero probabilities.
F1 is the harmonic mean of precision and
sensitivity; F1mod is the harmonic mean of all four
classification rates. F1gmean averages F1 and G-mean;
F1modgmean averages
F1mod and G-mean. A harmonic mean is zero if any component rate
is zero and all are defined; it is NA if any required
rate is undefined.
robust = TRUE adds one to each
diagonal confusion count when calculating the four rates and
their derived summaries. Returned confusion counts,
misclassification error, and probability-based scores are unchanged.
confusion = TRUE includes the
confusion matrix: rows are observed classes and columns are
predicted classes, with 0 denoting the majority and
1 the minority.
The performance summary has a "rand" attribute containing
reference metrics and expected confusion counts. This reference uses
a score \(U\) uniformly distributed on \([0,1]\), independent of
class, and predicts the minority class when \(U\) reaches the chosen
threshold. For threshold \(t\), its sensitivity is \(1-t\),
specificity is \(t\), and ROC AUC is \(1/2\). The separately
returned PR-AUC reference is the proportion of positive observations
among the rows scored by the PR helper. Its unnormalized Brier, normalized Brier,
and log-loss references are \(1/3\), \(4/3\), and \(1\),
respectively. These differ from a predictor that always returns
probability \(1/2\).
Nonlinear rate summaries are computed from the reference rates (or
smoothed expected counts under robust = TRUE); they are not
finite-sample expectations of those nonlinear statistics.
The performance print method compares the model and reference with
Delta and percentage Gain, oriented so that positive
values indicate improvement. The fitted-forest printer has its own
probability reference; it need not use the same baseline.
get.imbalanced.optimize searches ngrid equally spaced
thresholds from zero to one, using G-mean, F1, F1mod, or F1modgmean as
selected by measure. It uses the first grid point attaining
the largest available value. It returns one row of performance
metrics at the selected threshold and does not change the forest or
its stored class predictions. plot.it = FALSE suppresses its
four diagnostic panels. ngrid must be an integer of at least
two. When the chosen measure is unavailable at every grid point, the
function stops with an explanatory error. A valid optimum of zero is
retained. The caller's graphics settings are restored after plotting.
A grow object ordinarily supplies OOB probabilities for this search.
Supplying newdata searches using that data's observed responses
and predictions, making it a threshold-selection sample. For an
independent assessment of the chosen threshold, apply it unchanged
to a separate test sample. The examples select a threshold from
training OOB predictions and carry it forward to the test data.
Printing a performance result displays two metric tables. The first contains classification rates and their F1 and G-mean combinations; the second contains overall performance, including G-mean, AUC, PR-AUC, misclassification, Brier scores, and log loss. The same G-mean row appears in both tables; F1-related combinations appear only in the first.
Delta and percentage Gain compare the model with its
random-score reference, with positive values indicating improvement.
Larger rates, G-mean, and AUC values are better; smaller errors and
losses are better. digits controls the display precision,
and show.confusion = FALSE omits the optional confusion tables.
These settings do not change the stored performance values.
imbalanced returns an object from the selected forest grow
function. Usual components include the class-probability matrices
predicted
and predicted.oob, class assignments class and
class.oob, and requested performance and importance values.
Availability depends on the selected grow interface and sampling
options; see rfsrc and rfsrc.fast.
get.imbalanced.performanceA named numeric vector of
class counts, imbalance ratio, resolved numeric threshold, and
performance measures. With confusion = TRUE, a named list also
includes confusion, the confusion matrix with a
class.error column. Both forms have class
"imbalanced.performance" and a "rand" attribute
containing the random-reference metrics and expected confusion
counts.
get.imbalanced.optimizeA one-row data frame of
performance measures at the selected threshold. Extract the
cutoff from the threshold column; the row name is its
grid index.
get.rfq.thresholdThe smaller class count divided by
the total observed class count, as a numeric scalar. Returns
NULL unless the class-frequency table has exactly two
entries. No forest or probability predictions are required.
The performance and optimization helpers return NULL when
the evaluation response is not a two-level factor.
print.imbalanced.performance displays the summaries and returns
x invisibly.
Chen, C., Liaw, A. and Breiman, L. (2004). Using random forest to learn imbalanced data. University of California, Berkeley, Technical Report 110.
Kubat, M., Holte, R. and Matwin, S. (1997). Learning when negative examples abound. Machine Learning, ECML-97: 146-153.
O'Brien, R. and Ishwaran, H. (2019). A random forests quantile classifier for class imbalanced data. Pattern Recognition, 90, 232-249.
rfsrc, rfsrc.fast,
get.imbalanced.performance, subsample
# \donttest{
## ------------------------------------------------------------
## RFQ, BRF, and standard classification
## ------------------------------------------------------------
data(breast, package = "randomForestSRC")
dta <- na.omit(breast)
f <- status ~ .
set.seed(19)
rfq <- imbalanced(f, dta, ntree = 300)
print(rfq)
print(get.imbalanced.performance(rfq, confusion = TRUE))
## Extract the class-frequency threshold from the observed responses.
print(get.rfq.threshold(rfq$yvar))
brf <- imbalanced(f, dta, ntree = 300, method = "brf")
standard <- imbalanced(f, dta, ntree = 300, method = "standard")
## Compare all three using the helper's default evaluation rule.
metrics <- c("sens", "spec", "gmean", "misclass", "brier", "auc")
comparison <- rbind(
RFQ = as.numeric(get.imbalanced.performance(rfq)[metrics]),
BRF = as.numeric(get.imbalanced.performance(brf)[metrics]),
Standard = as.numeric(get.imbalanced.performance(standard)[metrics])
)
colnames(comparison) <- metrics
print(comparison)
## Supply a cutoff explicitly to inspect another probability rule.
print(get.imbalanced.performance(standard, threshold = .5))
## Let the saved classifier setting select prevalence or 0.5.
print(get.imbalanced.performance(rfq, threshold = "auto"))
print(get.imbalanced.performance(standard, threshold = "auto"))
print(get.imbalanced.performance(brf, threshold = "auto"))
## Request the 0.5 Bayes cutoff explicitly, including for RFQ.
print(get.imbalanced.performance(rfq, threshold = "bayes"))
## Vectors, one-column objects, and named two-column matrices.
class.frequency <- table(rfq$yvar)
minority <- names(class.frequency)[which.min(class.frequency)]
p <- rfq$predicted.oob[, minority]
print(get.imbalanced.performance(rfq$yvar, p))
print(get.imbalanced.performance(rfq$yvar, data.frame(p = p)))
print(get.imbalanced.performance(
rfq$yvar, rfq$predicted.oob[, rev(levels(rfq$yvar)), drop = FALSE]
))
## ------------------------------------------------------------
## Train/test assessment with a training-selected threshold
## ------------------------------------------------------------
set.seed(23)
trn.id <- sort(unlist(lapply(
split(seq_len(nrow(dta)), dta$status),
function(idx) idx[sample.int(length(idx),
size = floor(2 * length(idx) / 3))]
)))
trn <- dta[trn.id, , drop = FALSE]
tst <- dta[-trn.id, , drop = FALSE]
print(rbind(training = table(trn$status), test = table(tst$status)))
fit <- imbalanced(f, trn, ntree = 300)
selected <- get.imbalanced.optimize(fit, measure = "gmean",
ngrid = 501, plot.it = FALSE)
print(selected)
threshold.selected <- selected$threshold
pred <- predict(fit, newdata = tst)
print(get.imbalanced.performance(pred))
print(get.imbalanced.performance(pred, threshold = threshold.selected))
## Predicting labels for new rows does not require their outcomes.
class.frequency <- table(trn$status)
minority <- names(class.frequency)[which.min(class.frequency)]
majority <- setdiff(levels(trn$status), minority)
pred.x <- predict(fit, newdata = tst[, fit$xvar.names, drop = FALSE])
selected.class <- factor(
ifelse(pred.x$predicted[, minority] >= threshold.selected,
minority, majority),
levels = levels(trn$status)
)
print(head(selected.class))
## ------------------------------------------------------------
## Explicit permutation VIMP and blocking
## ------------------------------------------------------------
rfq.vimp <- imbalanced(f, dta, ntree = 300,
importance = "permute", block.size = 1)
rfq.block <- imbalanced(f, dta, ntree = 300,
importance = "permute", block.size = 10)
print(cbind(unblocked = rfq.vimp$importance[, 1],
blocked = rfq.block$importance[, 1]))
## Small B is for illustration; increase it for final inference.
smp <- subsample(rfq.block, B = 25, verbose = FALSE)
print(extract.subsample(smp)$var.sel.Z)
plot.subsample(smp, cex.axis = .7, alpha = .5)
## ------------------------------------------------------------
## Optional sampling and grow-interface choices
## ------------------------------------------------------------
rfq.sample <- imbalanced(f, dta, ntree = 300, ratio = .25)
print(get.imbalanced.performance(rfq.sample))
rfq.fast <- imbalanced(f, dta, ntree = 300, fast = TRUE)
print(get.imbalanced.performance(rfq.fast))
# }