Exponential Family of Distributions

In this part, we will provide a canonical form the density function of the exponential family of distributions. First, some basic results for normal density is obtained. Then, the distribution function of the exponential family of distributions specified in the natural parameter will be transformed to an expression using the expectation parameter and the derivatives of the potential function.

Startup

This section initializes the Mathematica session.

Normal distribution

This section first calculates the moments of the normal variables, which will be used to calculate the expected value of the exponential of normal variables.

the moments of the multivariate normal distribution

We consider the multivariate normal random vector of dim-dimensions with mean , and the identity covariance matrix. The density function is . In this subsection, we define "ruleintx" to calculate the central moments of such as , , and for .

the expectation of the exponential of polynomial functions

Here we would like to calculate the log of the expectation of  the exponential of , where a1, a2, and a3 are of order , and a4 is . Note that here indicates the -th element of instead of the -th power of .   is first obtained for , and next for general up to terms.

Exponential family

The density function of the exponential family is first specified by using the natural parameter vector. This standard form is transformed into our canonical expression of the density with respect to the expectation parameter vector. Since the exponential family is not uniquely expressed up to the affine transformation, we assume without loss of generality that origin of the expectation parameter coincides with that of the natural parameter, and the covariance matrix of the random variable is identity at the origin. We consider only the continuous random variable throughout. The canonical form will be stored in "logdensityy".

the standard form

Let denote the density function of random variable with the natural parameter vector . The log of the density function is specified by logdensity=log f(y;θ) using the cumulant function ψ(θ) and the measure function h(y).

the expression of ψ(θ) in terms of η

In this subsection,  we derive the expression of ψ(θ) using η and  the φ derivatives. The result will be stored in "psieta".

the expression of h(y) in terms of φ derivatives

In this subsection, we derive the expression of h(y) using φ derivatives. The result will be stored in "hinyphi".

the canonical form

Now we get the canonical form of in "logdensityy".