8 Statistics Functions

▼ |  | Math Library

1 | Constants and Elementary Functions
2 | Flonums
3 | Special Functions
4 | Number Theory
5 | Arbitrary-Precision Floating-Point Numbers (Bigfloats)
6 | Arrays
7 | Matrices and Linear Algebra
8 | Statistics Functions
9 | Probability Distributions
10 | Stuff That Doesn’t Belong Anywhere Else

► | 8 | Statistics Functions

8.1 | Expected Values
8.2 | Running Expected Values
8.3 | Correlation
8.4 | Counting and Binning
8.5 | Order Statistics
8.6 | Simulations

On this page:

8.1`` Expected Values
mean
variance
stddev
skewness
kurtosis
variance/ mean
stddev/ mean
skewness/ mean
kurtosis/ mean
8.2`` Running Expected Values
statistics
empty-statistics
update-statistics
update-statistics*
statistics-range
statistics-mean
statistics-variance
statistics-stddev
statistics-skewness
statistics-kurtosis
8.3`` Correlation
covariance
correlation
covariance/ means
correlation/ means
8.4`` Counting and Binning
samples->hash
count-samples
sample-bin
bin-samples
bin-samples/ key
sample-bin-compact
sample-bin-total
8.5`` Order Statistics
sort-samples
median
quantile
absdev
absdev/ median
hpd-interval
hpd-interval/ sorted
real-hpd-interval
real-hpd-interval/ sorted
8.6`` Simulations
mc-variance
mc-stddev
mc-stddev/ mean
mc-variance/ mean
indicator
mc-probability
mc-prob-dist

8.15
top contents ← prev up next →

### 8`` Statistics Functions🔗ℹ

Neil Toronto <ntoronto@racket-lang.org>

(require math/statistics) | package: math-lib

This module exports functions that compute statistics, meaning summary values for collections of samples, and functions for managing sequences of weighted or unweighted samples.

Most of the functions that compute statistics accept a sequence of nonnegative reals that correspond one-to-one with sample values. These are used as weights; equivalently counts, pseudocounts or unnormalized probabilities. While this makes it easy to work with weighted samples, it introduces some subtleties in bias correction. In particular, central moments must be computed without bias correction by default. See Expected Values for a discussion.

8.1 Expected Values
8.2 Running Expected Values
8.3 Correlation
8.4 Counting and Binning
8.5 Order Statistics
8.6 Simulations

#### 8.1`` Expected Values🔗ℹ

Functions documented in this section that compute higher central moments, such as variance, stddev and skewness, can optionally apply bias correction to their estimates. For example, when variance is given the argument #:bias #t, it multiplies the result by (/ n (- n 1)), where n is the number of samples.

The meaning of “bias correction” becomes less clear with weighted samples, however. Often, the weights represent counts, so when moment-estimating functions receive #:bias #t, they interpret it as “use the sum of ws for n.” In the following example, the sample 4 is first counted twice and then given weight 2; therefore n = 5 in both cases:

> > (variance '(1 2 3 4 4) #:bias #t)
> - : Real [more precisely: Nonnegative-Real]
> 17/10
> > (variance '(1 2 3 4) '(1 1 1 2) #:bias #t)
> - : Real [more precisely: Nonnegative-Real]
> 17/10

However, sample weights often do not represent counts. For these cases, the #:bias keyword can be followed by a real-valued pseudocount, which is used for n:

> > (variance '(1 2 3 4) '(1/2 1/2 1/2 1) #:bias 5)
> - : Real [more precisely: Nonnegative-Real]
> 17/10

Because the magnitude of the bias correction for weighted samples cannot be known without user guidance, in all cases, the bias argument defaults to #f.

> procedure (mean xs [ws]) → Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f

When ws is #f (the default), returns the sample mean of the values in xs. Otherwise, returns the weighted sample mean of the values in xs with corresponding weights ws.

Examples:

> > (mean '(1 2 3 4 5))
> - : Real
> 3
> > (mean '(1 2 3 4 5) '(1 1 1 1 10.0))
> - : Real
> 4.285714285714286
> > (define d (normal-dist))
> > (mean (sample d 10000))
> - : Real
> -0.006816560948985608
> > (define arr (array-strict (build-array #(5 1000) (λ (_) (sample d)))))
> > (array-map mean (array->list-array arr 1))
> - : #(struct:Array
> (Indexes Index (Boxof Boolean) (-> Void) (-> Indexes Real))
> #<syntax:/pbulk/work/lang/racket/work/racket-8.15/share/pkgs/math-lib/math/private/array/typed-array-struct.rkt:55:13 prop:custom-write>
> #<syntax:/pbulk/work/lang/racket/work/racket-8.15/share/pkgs/math-lib/math/private/array/typed-array-struct.rkt:56:13 prop:equal+hash>
> #<syntax:/pbulk/work/lang/racket/work/racket-8.15/share/pkgs/math-lib/math/private/array/typed-array-struct.rkt:54:13 prop:custom-print-quotable>)
> (array
> #[-0.013888824050639608
> 0.000780078802202017
> 0.004182118634936793
> 0.004408593058137577
> 0.021974179407850607])

> procedure (variance xs [ws #:bias bias]) → Nonnegative-Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (stddev xs [ws #:bias bias]) → Nonnegative-Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (skewness xs [ws #:bias bias]) → Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (kurtosis xs [ws #:bias bias]) → Nonnegative-Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f

If ws is #f, these compute the sample variance, standard deviation, skewness and excess kurtosis the samples in xs. If ws is not #f, they compute weighted variations of the same.

Examples:

> > (stddev '(1 2 3 4 5))
> - : Real [more precisely: Nonnegative-Real]
> 1.4142135623730951
> > (stddev '(1 2 3 4 5) '(1 1 1 1 10))
> - : Real [more precisely: Nonnegative-Real]
> 1.2777531299998799

See Expected Values for the meaning of the bias keyword argument.

> procedure (variance/mean m xs [ws #:bias bias]) → Nonnegative-Real
> m : Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (stddev/mean m xs [ws #:bias bias]) → Nonnegative-Real
> m : Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (skewness/mean m xs [ws #:bias bias]) → Real
> m : Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (kurtosis/mean m xs [ws #:bias bias]) → Nonnegative-Real
> m : Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f

Like variance, stddev, skewness and kurtosis, but computed using known mean m.

#### 8.2`` Running Expected Values🔗ℹ

The statistics object allows computing the sample minimum, maximum, count, mean, variance, skewness, and excess kurtosis of a sequence of samples in O(1) space.

To use it, start with empty-statistics, then use update-statistics to obtain a new statistics object with updated values. Use statistics-min, statistics-mean, and similar functions to get the current estimates.

Example:

> > (let* ([s empty-statistics]
> [s (update-statistics s 1)]
> [s (update-statistics s 2)]
> [s (update-statistics s 3)]
> [s (update-statistics s 4 2)])
> (values (statistics-mean s)
> (statistics-stddev s #:bias #t)))
> - : (values Flonum Flonum) [more precisely: (Values Flonum Nonnegative-Flonum)]
> 2.8
> 1.3038404810405297

> struct (struct statistics (min max count))
> min : Flonum
> max : Flonum
> count : Nonnegative-Flonum

Represents running statistics.

The min and max fields are the minimum and maximum value observed so far, and the count field is the total weight of the samples (which is the number of samples if all samples are unweighted). The remaining, hidden fields are used to compute moments, and their number and meaning may change in future releases.

> value empty-statistics : statistics

The empty statistics object.

Examples:

> > (statistics-min empty-statistics)
> - : Flonum
> +inf.0
> > (statistics-max empty-statistics)
> - : Flonum
> -inf.0
> > (statistics-range empty-statistics)
> - : Flonum [more precisely: Nonnegative-Flonum]
> +nan.0
> > (statistics-count empty-statistics)
> - : Flonum [more precisely: Nonnegative-Flonum]
> 0.0
> > (statistics-mean empty-statistics)
> - : Flonum
> +nan.0
> > (statistics-variance empty-statistics)
> - : Flonum [more precisely: Nonnegative-Flonum]
> +nan.0
> > (statistics-skewness empty-statistics)
> - : Flonum
> +nan.0
> > (statistics-kurtosis empty-statistics)
> - : Flonum [more precisely: Nonnegative-Flonum]
> +nan.0

> procedure (update-statistics s x [w]) → statistics
> s : statistics
> x : Real
> w : Real = 1.0

Returns a new statistics object that includes x in the computed statistics. If w is given, x is weighted by w in the moment computations.

> procedure (update-statistics* s xs [ws]) → statistics
> s : statistics
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f

Like update-statistics, but includes all of xs, possibly weighted by corresponding elements in ws, in the returned statistics object.

Examples:

> > (define s (update-statistics* empty-statistics '(1 2 3 4) '(1 1 1 2)))
> > (statistics-mean s)
> - : Flonum
> 2.8
> > (statistics-stddev s #:bias #t)
> - : Flonum [more precisely: Nonnegative-Flonum]
> 1.3038404810405297

This function uses O(1) space regardless of the length of xs.

> procedure (statistics-range s) → Nonnegative-Flonum
> s : statistics
> procedure (statistics-mean s) → Flonum
> s : statistics
> procedure (statistics-variance s [#:bias bias]) → Nonnegative-Flonum
> s : statistics
> bias : (U #t #f Real) = #f
> procedure (statistics-stddev s [#:bias bias]) → Nonnegative-Flonum
> s : statistics
> bias : (U #t #f Real) = #f
> procedure (statistics-skewness s [#:bias bias]) → Flonum
> s : statistics
> bias : (U #t #f Real) = #f
> procedure (statistics-kurtosis s [#:bias bias]) → Nonnegative-Flonum
> s : statistics
> bias : (U #t #f Real) = #f

Compute the range, mean, variance, standard deviation, skewness, and excess kurtosis of the observations summarized in s.

See Expected Values for the meaning of the bias keyword argument.

#### 8.3`` Correlation🔗ℹ

> procedure (covariance xs ys [ws #:bias bias]) → Real
> xs : (Sequenceof Real)
> ys : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (correlation xs ys [ws #:bias bias]) → Real
> xs : (Sequenceof Real)
> ys : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f

Compute the sample covariance and correlation of xs and ys, optionally weighted by ws.

Examples:

> > (define xs (sample (normal-dist) 10000))
> > (define ys (map (λ: ([x : Real]) (sample (normal-dist x))) xs))
> > (correlation xs ys)
> - : Real
> 0.7040973858673851

Removing the correlation using importance weights:

> > (define ws (map (λ: ([x : Real] [y : Real])
> (/ (pdf (normal-dist) y)
> (pdf (normal-dist x) y)))
> xs ys))
> > (correlation xs ys (ann ws (Sequenceof Real)))
> - : Real
> -0.062072634513794564

See Expected Values for the meaning of the bias keyword argument.

> procedure (covariance/means mx my xs ys [ws #:bias bias]) → Real
> mx : Real
> my : Real
> xs : (Sequenceof Real)
> ys : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (correlation/means mx my xs ys [ws #:bias bias]) → Real
> mx : Real
> my : Real
> xs : (Sequenceof Real)
> ys : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f

Like covariance and correlation, but computed using known means mx and my.

#### 8.4`` Counting and Binning🔗ℹ

> procedure (samples->hash xs) → (HashTable A Positive-Integer)
> xs : (Sequenceof A)
> (samples->hash xs ws) → (HashTable A Nonnegative-Real)
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real))

Returns a hash table mapping each unique element in xs (under equal?) to its count, or, if ws is not #f, to its total weight.

Examples:

> > (samples->hash '(1 2 3 4 4))
> - : (HashTable Integer Positive-Integer)
> '#hash((1 . 1) (2 . 1) (3 . 1) (4 . 2))
> > (samples->hash '(1 1 2 3 4) '(1/2 1/2 1 1 2))
> - : (HashTable Integer Nonnegative-Real)
> '#hash((1 . 1) (2 . 1) (3 . 1) (4 . 2))

> procedure (count-samples xs)
> → (Values (Listof A) (Listof Positive-Integer))
> xs : (Sequenceof A)
> (count-samples xs ws)
> → (Values (Listof A) (Listof Nonnegative-Real))
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real))

Like samples->hash, but returns two lists. The elements in the returned (Listof A) are in order of first appearance in xs.

Examples:

> > (count-samples '(1 2 3 4 4))
> - : (values (Listof Positive-Byte) (Listof Positive-Integer))
> '(1 2 3 4)
> '(1 1 1 2)
> > (count-samples '(1 1 2 3 4) '(1/2 1/2 1 1 2))
> - : (values (Listof Positive-Byte) (Listof Nonnegative-Real))
> '(1 2 3 4)
> '(1 1 1 2)

> struct (struct sample-bin (min max values weights))
> min : B
> max : B
> values : (Listof A)
> weights : (U #f (Listof Nonnegative-Real))

Represents a bin, or a group of samples within an interval in a total order. The values and bounds have a different type to allow bin-samples/key to group elements based on a function of their values.

> procedure (bin-samples bounds lte? xs ws) → (Listof (sample-bin A A))
> bounds : (Sequenceof A)
> lte? : (A A -> Any)
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real))

Similar to (sort xs lte?), but additionally groups samples into bins. The bins’ bounds are sorted before binning xs.

If n = (length bounds), then bin-samples returns at least (- n 1) bins, one for each pair of adjacent (sorted) bounds. If some values in xs are less than the smallest bound, they are grouped into a single bin in front. If some are greater than the largest bound, they are grouped into a single bin at the end.

Examples:

> > (bin-samples '() <= '(0 1 2 3 4 5 6))
> - : (Listof
> #(struct:sample-bin
> (Byte Byte (Listof Byte) (U (Listof Nonnegative-Real) False))))
> (list (sample-bin 0 6 '(0 1 2 3 4 5 6) #f))
> > (bin-samples '(3) <= '(0 1 2 3 4 5 6))
> - : (Listof
> #(struct:sample-bin
> (Byte Byte (Listof Byte) (U (Listof Nonnegative-Real) False))))
> (list (sample-bin 0 3 '(0 1 2 3) #f) (sample-bin 3 6 '(4 5 6) #f))
> > (bin-samples '(2 4) <= '(0 1 2 3 4 5 6))
> - : (Listof
> #(struct:sample-bin
> (Byte Byte (Listof Byte) (U (Listof Nonnegative-Real) False))))
> (list
> (sample-bin 0 2 '(0 1 2) #f)
> (sample-bin 2 4 '(3 4) #f)
> (sample-bin 4 6 '(5 6) #f))
> > (bin-samples '(2 4) <=
> '(0 1 2 3 4 5 6)
> '(10 20 30 40 50 60 70))
> - : (Listof
> #(struct:sample-bin
> (Byte Byte (Listof Byte) (U (Listof Nonnegative-Real) False))))
> (list
> (sample-bin 0 2 '(0 1 2) '(10 20 30))
> (sample-bin 2 4 '(3 4) '(40 50))
> (sample-bin 4 6 '(5 6) '(60 70)))

If lte? is a less-than-or-equal relation, the bins represent half-open intervals (min, max] (except possibly the first, which may be closed). If lte? is a less-than relation, the bins represent half-open intervals [min, max) (except possibly the last, which may be closed). In either case, the sorts applied to bounds and xs are stable.

Because intervals used in probability measurements are normally open on the left, prefer to use less-than-or-equal relations for lte?.

If ws is #f, bin-samples returns bins with #f weights.

> procedure (bin-samples/key bounds lte? key xs ws) → (Listof (sample-bin A B))
> bounds : (Sequenceof B)
> lte? : (B B -> Any)
> key : (A -> B)
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real))

Similar to (sort xs lte? #:key key #:cache-keys? #t), but additionally groups samples into bins.

Example:

> > (bin-samples/key '(2 4) <= (inst car Real String)
> '((1 . "1") (2 . "2") (3 . "3") (4 . "4") (5 . "5")))
> - : (Listof
> #(struct:sample-bin
> (Real
> Real
> (Listof (Pairof Positive-Byte String))
> (U (Listof Nonnegative-Real) False))))
> (list
> (sample-bin 1 2 '((1 . "1") (2 . "2")) #f)
> (sample-bin 2 4 '((3 . "3") (4 . "4")) #f)
> (sample-bin 4 5 '((5 . "5")) #f))

See bin-samples for the simpler, one-type variant.

> procedure (sample-bin-compact bin) → (sample-bin A B)
> bin : (sample-bin A B)

Compacts bin by applying count-samples to its values and weights.

Example:

> > (sample-bin-compact (sample-bin 1 4 '(1 2 3 4 4) #f))
> - : #(struct:sample-bin
> (Positive-Byte
> Positive-Byte
> (Listof Positive-Byte)
> (U (Listof Nonnegative-Real) False)))
> (sample-bin 1 4 '(1 2 3 4) '(1 1 1 2))

> procedure (sample-bin-total bin) → Nonnegative-Real
> bin : (sample-bin A B)

If (sample-bin-weights bin) is #f, returns the number of samples in bin; otherwise, returns the sum of their weights.

Examples:

> > (sample-bin-total (sample-bin 1 4 '(1 2 3 4 4) #f))
> - : Real [more precisely: Nonnegative-Real]
> 5
> > (sample-bin-total (sample-bin-compact (sample-bin 1 4 '(1 2 3 4 4) #f)))
> - : Real [more precisely: Nonnegative-Real]
> 5

#### 8.5`` Order Statistics🔗ℹ

> procedure (sort-samples lt? xs) → (Listof A)
> lt? : (A A -> Any)
> xs : (Sequenceof A)
> (sort-samples lt? xs ws)
> → (Values (Listof A) (Listof Nonnegative-Real))
> lt? : (A A -> Any)
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real))

Sorts possibly weighted samples according to lt?, which is assumed to define a total order over the elements in xs.

Examples:

> > (sort-samples < '(5 2 3 1))
> - : (Listof Positive-Byte)
> '(1 2 3 5)
> > (sort-samples < '(5 2 3 1) '(50 20 30 10))
> - : (values (Listof Positive-Byte) (Listof Nonnegative-Real))
> '(1 2 3 5)
> '(10 20 30 50)
> > (sort-samples < '(5 2 3 1) #f)
> - : (values (Listof Positive-Byte) (Listof Nonnegative-Real))
> '(1 2 3 5)
> '(1 1 1 1)

Because sort-samples is defined in terms of sort, the sort is only guaranteed to be stable if lt? is strictly a less-than relation.

> procedure (median lt? xs [ws]) → A
> lt? : (A A -> Any)
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real)) = #f

Equivalent to (quantile 1/2 lt? xs ws).

> procedure (quantile p lt? xs [ws]) → A
> p : Real
> lt? : (A A -> Any)
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real)) = #f

Computes the inverse of the empirical cdf represented by the samples xs, which are optionally weighted by ws.

Examples:

> > (quantile 0 < '(1 3 5))
> - : Integer [more precisely: Positive-Byte]
> 1
> > (quantile 0.5 < '(1 2 3 4))
> - : Integer [more precisely: Positive-Byte]
> 2
> > (quantile 0.5 < '(1 2 3 4) '(0.25 0.2 0.2 0.35))
> - : Integer [more precisely: Positive-Byte]
> 3

If p = 0, quantile returns the smallest element of xs under the ordering relation lt?. If p = 1, it returns the largest element.

For weighted samples, quantile sorts xs and ws together (using sort-samples), then finds the least x for which the proportion of its cumulative weight is greater than or equal to p.

For unweighted samples, quantile uses the quickselect algorithm to find the element that would be at index (ceiling (- (* p n) 1)) if xs were sorted, where n is the length of xs.

> procedure (absdev xs [ws]) → Nonnegative-Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f

Computes the average absolute difference between the elements in xs and (median < xs ws). If ws is not #f, it is a weighted average.

> procedure (absdev/median median xs [ws]) → Nonnegative-Real
> median : Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f

Like (absdev xs ws), but computed using known median median.

> procedure (hpd-interval lt? δ p xs [ws]) → (Values A A)
> lt? : (A A -> Any)
> δ : (A A -> Real)
> p : Real
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real)) = #f
> procedure (hpd-interval/sorted δ p xs [ws]) → (Values A A)
> δ : (A A -> Real)
> p : Real
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real)) = #f

Estimates the smallest interval for which the distribution that produced xs (optionally weighted by ws) assigns probability p, which must be positive. The type A represents an ordered metric space with order lt? and metric δ.

To compute an HPD interval from sorted samples, use hpd-interval/sorted.

You almost certainly want to use real-hpd-interval or real-hpd-interval/sorted instead, which are defined in terms of these.

> procedure (real-hpd-interval p xs [ws]) → (Values Real Real)
> p : Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> procedure (real-hpd-interval/sorted p xs [ws]) → (Values Real Real)
> p : Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f

Equivalent to (hpd-interval < - p xs ws) and (hpd-interval/sorted - p xs ws).

Examples:

> > (define beta32 (beta-dist 3 2))
> > (real-dist-hpd-interval beta32 0.8)
> - : (values Flonum Flonum)
> 0.3654299174284618
> 0.8939657937826784
> > (real-hpd-interval 0.8 (sample beta32 10000))
> - : (values Real Real)
> 0.378367142657237
> 0.9118671782855396

#### 8.6`` Simulations🔗ℹ

The functions in this section support Monte Carlo simulation; for example, quantifying uncertainty about statistics estimated from samples.

> procedure (mc-variance xs [ws #:bias bias]) → Nonnegative-Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (mc-stddev xs [ws #:bias bias]) → Nonnegative-Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f

Estimate the variance and standard deviation of expected values computed from random samples.

If xs are random variable samples, then

> (define θ (mean xs ws))

is also a random variable sample. These two values:

> (mc-variance xs ws #:bias bias)
> (mc-stddev xs ws #:bias bias)

estimate the variance and standard deviation of θ. The latter is simply the square root of the variance, and bias correction is applied as described in Expected Values.

Two different ways to estimate the standard deviation of a mean computed from 1000 samples are

> > (mc-stddev (sample (normal-dist 0 1) 1000))
> - : Real [more precisely: Nonnegative-Real]
> 0.030918250273558406
> > (stddev (for/list : (Listof Real) ([_ (in-range 100)])
> (mean (sample (normal-dist 0 1) 1000))))
> - : Real [more precisely: Nonnegative-Real]
> 0.03374782414109147

Using mc-stddev is 100 times faster in this case, and in most statistical applications, replicating a sampling process 100 times is infeasible.

> procedure (mc-stddev/mean m xs [ws #:bias bias]) → Nonnegative-Real
> m : Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f
> procedure (mc-variance/mean m xs [ws #:bias bias]) → Nonnegative-Real
> m : Real
> xs : (Sequenceof Real)
> ws : (U #f (Sequenceof Real)) = #f
> bias : (U #t #f Real) = #f

Use these in the exceedingly rare cases in which you know the mean m but are estimating uncertainty in an estimate of the mean anyway.

> procedure (indicator pred?) → (A -> (U 0 1))
> pred? : (A -> Any)

Converts a predicate into an indicator function.

Examples:

> > (fl (mean (map (indicator (λ ([x : Real]) (< -inf.0 x -1)))
> (sample (normal-dist 0 1) 5000))))
> - : Flonum
> 0.1482
> > (real-dist-prob (normal-dist 0 1) -inf.0 -1)
> - : Flonum
> 0.15865525393145705

> procedure (mc-probability pred? xs [ws]) → Nonnegative-Real
> pred? : (A -> Any)
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real)) = #f

Estimates the probability of pred? from possibly weighted samples. Equivalent to (mean (sequence-map (indicator pred?) xs) ws).

Example:

> > (fl (mc-probability (λ ([x : Real]) (< -inf.0 x -1))
> (sample (normal-dist 0 1) 5000)))
> - : Flonum [more precisely: Nonnegative-Flonum]
> 0.1628

> procedure (mc-prob-dist pred? xs [ws]) → Beta-Dist
> pred? : (A -> Any)
> xs : (Sequenceof A)
> ws : (U #f (Sequenceof Real)) = #f

Returns a beta distribution estimated from possibly weighted samples whose mean is (mc-probability pred? xs ws).

Computing a confidence interval for a probability whose endpoints are guaranteed to be between 0 and 1:

> > (real-dist-hpd-interval
> (mc-prob-dist (λ ([x : Real]) (< -inf.0 x -1))
> (sample (normal-dist 0 1) 5000))
> 0.95)
> - : (values Flonum Flonum)
> 0.15182876351995664
> 0.1722470940829688

top contents ← prev up next →