Exact two-sample Kolmogorov-Smirnov test

Regular test

HypothesisTests.ExactOneSampleKSTest — Type

ExactOneSampleKSTest(uv::AbstractUncertainValue,
    d::UnivariateDistribution, n::Int = 1000) -> ExactOneSampleKSTest

Perform a one-sample exact Kolmogorov–Smirnov test of the null hypothesis that a draw of n realisations of the uncertain value uv comes from the distribution d against the alternative hypothesis that the sample is not drawn from d.

source

Example

We'll test whether the uncertain value uv = UncertainValue(Gamma, 2, 4) comes from the theoretical distribution Gamma(2, 4). Of course, we expect the test to confirm this, because we're using the exact same distribution.

uv = UncertainValue(Gamma, 2, 4)

# Perform the Kolgomorov-Smirnov test by drawing 1000 samples from the
# uncertain value.
ExactOneSampleKSTest(uv, Gamma(2, 4), 1000)

That gives the following output:

Exact one sample Kolmogorov-Smirnov test
----------------------------------------
Population details:
    parameter of interest:   Supremum of CDF differences
    value under h_0:         0.0
    point estimate:          0.0228345021301449

Test summary:
    outcome with 95% confidence: fail to reject h_0
    two-sided p-value:           0.6655

Details:
    number of observations:   1000

As expected, the test can't reject the hypothesis that the uncertain value uv comes from the theoretical distribution Gamma(2, 4), precisely because it does.

Pooled test

UncertainData.UncertainStatistics.ExactOneSampleKSTestPooled — Function

ExactOneSampleKSTestPooled(ud::UncertainDataset,
    d::UnivariateDistribution, n::Int = 1000) -> ExactOneSampleKSTest

First, draw n realisations of each uncertain value in ud and pool them together. Then perform a one-sample exact Kolmogorov–Smirnov test of the null hypothesis that the pooled values comes from the distribution d against the alternative hypothesis that the sample is not drawn from d.

source

Element-wise test

UncertainData.UncertainStatistics.ExactOneSampleKSTestElementWise — Function

ExactOneSampleKSTestElementWise(ud::UncertainDataset,
    d::UnivariateDistribution, n::Int = 1000) -> Vector{ExactOneSampleKSTest}

First, draw n realisations of each uncertain value in ud, keeping one pool of values for each uncertain value.

Then, perform an element-wise (pool-wise) one-sample exact Kolmogorov–Smirnov test of the null hypothesis that each value pool comes from the distribution d against the alternative hypothesis that the sample is not drawn from d.

source