Metadata
- Author: sebastianraschka.com
- Title: Creating Confidence Intervals for Machine Learning Classifiers
- Reference: https://sebastianraschka.com/blog/2022/confidence-intervals-for-ml.html
- Category: #article
Page Notes
Highlights
-
As a side-note, we can say that the difference of two measurements is statistically significant if confidence intervals do not overlap. However we cannot say that results are not statistically significant if confidence intervals overlap. — Updated on 2022-06-04 18:33:45 — Group: #Public
-
This is done by taking multiple samples with replacement from a single random sample. The equation is as follows: ACCbootavg=1b∑bj=1ACCboot,j, — Updated on 2022-06-04 19:06:01 — Group: #Public
-
Note that 200 is usually recommended as the minimum number of bootstrap rounds (see “Introduction to the Bootstrap” book). — Updated on 2022-06-04 19:06:26 — Group: #Public
-
However, suppose we don’t tune our model on the training set. In that case, we can use the whole dataset and report the averaged bootstrap accuracy ACCbootavgACCbootavg\text{ACC}_{\text{bootavg}} as your model performance estimate instead of using an independent test set. — Updated on 2022-06-04 19:08:00 — Group: #Public