Families of Unnormalized Densities

Returning to the overstated claim with which we began, what did the phrase `write down a model' mean? The models our methods can handle are those specified by families of unnormalized densities.

The notion of an unnormalized density is elementary, occurring in the problem often assigned in introductory probability courses of finding the constant that normalizes a given function to be a probability density. Thus this section contains nothing deep. It merely gives new language for old concepts. But this is important. Language constrains thought, and new language leads to new ways of looking at old problems.

Let be a measure space and h a nonnegative real function on . If

 

is finite and nonzero, then

defines a probability density f with respect to of a measure P. We say that

• h is an unnormalized density of P with respect to .
• f is the normalized density of P with respect to .
• c is the normalizing constant for h.

Use of the term `unnormalized density' implies that the normalizing constant (1.1) is finite and nonzero.

This game can be played for families of densities specifying a statistical model. Let be a family of unnormalized densities with respect to . This terminology implies that

is finite and nonzero for all . Also

 

defines a probability density with respect to of a measure for all . Then is a statistical model specified by the family of probability densities with respect to . We say that

• is a family of unnormalized densities for the model .
• is the family of normalized densities for .
• is the normalizing function for the family .

This terminology was introduced in Geyer (1994). It arises naturally from consideration of the most general case in which the methods of Monte Carlo maximum likelihood work.