LogExpFunctions

xlogx(x)

Return x * log(x) for x ≥ 0, handling $x = 0$ by taking the downward limit.

julia> xlogx(0)
0.0

xlogy(x, y)

Return x * log(y) for y > 0 with correct limit at $x = 0$.

julia> xlogy(0, 0)
0.0

xlog1py(x, y)

Return x * log(1 + y) for y ≥ -1 with correct limit at $x = 0$.

julia> xlog1py(0, -1)
0.0

xexpx(x)

Return x * exp(x) for x > -Inf, or zero if x == -Inf.

julia> xexpx(-Inf)
0.0

xexpy(x, y)

Return x * exp(y) for y > -Inf, or zero if y == -Inf or if x == 0 and y is finite.

julia> xexpy(1.0, -Inf)
0.0

logistic(x)

The logistic sigmoid function mapping a real number to a value in the interval $[0,1]$,

\[\sigma(x) = \frac{1}{e^{-x} + 1} = \frac{e^x}{1+e^x}.\]

Its inverse is the logit function.

logit(x)

The logit or log-odds transformation, defined as

\[\operatorname{logit}(x) = \log\left(\frac{x}{1-x}\right)\]

for $0 < x < 1$.

Its inverse is the logistic function.

logcosh(x)

Return log(cosh(x)), carefully evaluated without intermediate calculation of cosh(x).

The implementation ensures logcosh(-x) = logcosh(x).

logabssinh(x)

Return log(abs(sinh(x))), carefully evaluated without intermediate calculation of sinh(x).

The implementation ensures logabssinh(-x) = logabssinh(x).

log1psq(x)

Return log(1+x^2) evaluated carefully for abs(x) very small or very large.

log1pexp(x)

Return log(1+exp(x)) evaluated carefully for largish x.

This is also called the "softplus" transformation, being a smooth approximation to max(0,x). Its inverse is logexpm1.

See:

• Martin Maechler (2012) “Accurately Computing log(1 − exp(− |a|))”

log1mexp(x)

Return log(1 - exp(x))

See:

• Martin Maechler (2012) “Accurately Computing log(1 − exp(− |a|))”

Note: different than Maechler (2012), no negation inside parentheses

log2mexp(x)

Return log(2 - exp(x)) evaluated as log1p(-expm1(x))

logexpm1(x)

Return log(exp(x) - 1) or the “invsoftplus” function. It is the inverse of log1pexp (aka “softplus”).

log1pmx(x)

Return log(1 + x) - x.

Use naive calculation or range reduction outside kernel range. Accurate ~2ulps for all x. This will fall back to the naive calculation for argument types different from Float64.

logmxp1(x)

Return log(x) - x + 1 carefully evaluated. This will fall back to the naive calculation for argument types different from Float64.

logaddexp(x, y)

Return log(exp(x) + exp(y)), avoiding intermediate overflow/undeflow, and handling non-finite values.

logsubexp(x, y)

Return log(abs(exp(x) - exp(y))), preserving numerical accuracy.

logsumexp(X)

Compute log(sum(exp, X)).

X should be an iterator of real or complex numbers. The result is computed in a numerically stable way that avoids intermediate over- and underflow, using a single pass over the data.

See also logsumexp!.

References

Sebastian Nowozin: Streaming Log-sum-exp Computation

logsumexp(X; dims)

Compute log.(sum(exp.(X); dims=dims)).

The result is computed in a numerically stable way that avoids intermediate over- and underflow, using a single pass over the data.

See also logsumexp!.

References

Sebastian Nowozin: Streaming Log-sum-exp Computation

logsumexp!(out, X)

Compute logsumexp of X over the singleton dimensions of out, and write results to out.

The result is computed in a numerically stable way that avoids intermediate over- and underflow, using a single pass over the data.

See also logsumexp.

References

Sebastian Nowozin: Streaming Log-sum-exp Computation

softmax!(r::AbstractArray{<:Real}, x::AbstractArray{<:Real}=r; dims=:)

Overwrite r with the softmax transformation of x over dimension dims.

That is, r is overwritten with exp.(x), normalized to sum to 1 over the given dimensions.

See also: softmax

softmax(x::AbstractArray{<:Real}; dims=:)

Return the softmax transformation of x over dimension dims.

That is, return exp.(x), normalized to sum to 1 over the given dimensions.

See also: softmax!

cloglog(x)

Compute the complementary log-log, log(-log(1 - x)).

cexpexp(x)

Compute the complementary double exponential, 1 - exp(-exp(x)).