Shannon, Renyi and Tsallis entropies. All three share one documentation topic so that every argument is described exactly once.
Usage
bd_entropy_shannon(
a,
b,
c,
k,
terms = 20000L,
method = c("closed", "quadrature"),
rel.tol = 1e-10,
subdivisions = 4000L
)
bd_entropy_renyi(a, b, c, k, order = 2, rel.tol = 1e-10, subdivisions = 4000L)
bd_entropy_tsallis(
a,
b,
c,
k,
order = 2,
rel.tol = 1e-10,
subdivisions = 4000L
)Arguments
- a
Shape parameter (beta generator).
- b
Shape parameter governing the tail.
- c
Shape parameter (baseline).
- k
Scale parameter (baseline).
- terms
Number of series terms before the analytic tail is applied (Shannon only).
- method
`"closed"` (default) for the closed form, or `"quadrature"` for direct numerical integration of \(-f \log f\), which is slower but independent of the series (Shannon only).
- rel.tol
Relative accuracy, passed to [stats::integrate()].
- subdivisions
Subdivision limit, passed to [stats::integrate()].
- order
Entropy order \(q\), positive and not equal to one (Renyi and Tsallis only).
Details
Writing \(u = G(Z) \sim \mathrm{Beta}(a,b)\) and \(v = u^{1/c}\), the Shannon entropy has the closed form $$H = \log B(a,b) - \log(ck) - (a - 1/c)\{\psi(a) - \psi(a+b)\} - (b - 1)\{\psi(b) - \psi(a+b)\} + 2\sum_{i \ge 1} \frac{B(a + i/c,\, b)}{i\, B(a,b)},$$ the final sum arising from \(-2E\log(1 - v)\) expanded as a power series.
With \(I_q = \int_0^\infty f(z)^q dz\), the Renyi entropy is \(\log(I_q)/(1-q)\) and the Tsallis entropy is \((1 - I_q)/(q-1)\). Both reduce to the Shannon entropy as \(q \to 1\), which is excluded; use `bd_entropy_shannon()` at \(q = 1\).
\(I_q\) is evaluated as \(\int_0^1 f(z(u))^{q-1} d\mathrm{Beta}(u;a,b)\) on the finite Beta scale rather than over the half line, which keeps the heavy upper tail from dominating the quadrature.
Series truncation
The Shannon series terms decay like \(i^{-(b+1)}\), so truncating at \(M\) omits a tail of order \(M^{-b}\). That is negligible for large \(b\) and is not for small \(b\): against high-precision integration the plain truncated sum at \(M = 2000\) is out by about \(10^{-8}\) at \(b = 3\) but by about \(0.11\) at \(b = 0.5\). The analytic tail \(\Gamma(b)c^{b}M^{-b}/\{b B(a,b)\}\) is therefore added, restoring agreement to roughly eight digits across the range.
Examples
bd_entropy_shannon(a = 1.5, b = 3, c = 2, k = 1)
#> [1] 1.499916
# Independent check by quadrature
bd_entropy_shannon(a = 1.5, b = 3, c = 2, k = 1, method = "quadrature")
#> [1] 1.499916
bd_entropy_renyi(a = 1.5, b = 3, c = 2, k = 1, order = 2)
#> [1] 1.147539
bd_entropy_tsallis(a = 1.5, b = 3, c = 2, k = 1, order = 2)
#> [1] 0.682583