The Mathematical Garden

Estimation of the Parameter a

In order to define the HMM, one has to estimate a from a observed data sequence. Suppose that a sequence (in the steady state) of number of dots is observed as follows:

1, 2, 1, 2, 1, 2, 2, 4, 1, 2, 2, 1, 3, 3, 4, 1,

then the question is: what is the "best" a? To answer this question, we note that

P12 =

æ ç ç ç ç ç è a 1 - a 0 0 0 0 ö ÷ ÷ ÷ ÷ ÷ ø a 1 - a 0 0 0 0 0 0 1/6+a/12 1/6+a/12 1/3-a/12 1/3-a/12 0 0 1/6+a/12 1/6+a/12 1/3-a/12 1/3-a/12 0 0 1/6+a/12 1/6+a/12 1/3-a/12 1/3-a/12 0 0 1/6+a/12 1/6+a/12 1/3-a/12 1/3-a/12

The structure of P₁² can be explained by the fact that each state has a period of two. If we ignore the hidden states (the first diagonal block of P₁²), then the observable states follow the transition probability matrix given by

P12	=	é ê ê ê ë 1/6+a/12 1/6+a/12 1/3-a/12 1/3-a/12 ù ú ú ú û 1/6+a/12 1/6+a/12 1/3-a/12 1/3-a/12 1/6+a/12 1/6+a/12 1/3-a/12 1/3-a/12 1/6+a/12 1/6+a/12 1/3-a/12 1/3-a/12
	=	é ê ê ê ë a 1-a ù ú ú ú û a 1-a a 1-a a 1-a	é ë 1/4 1/4 1/4 1/4 ù û 1/6 1/6 1/3 1/3

It is easy to see that the stationary probability distribution of is given by

p = ( 1/6 + a/12, 1/6 + a/12, 1/3 - a/12, 1/3 - a/12)

because ₁ = (1, 1, 1, 1)^Tp and therefore p₁ = p. This probability distribution p should be consistent with the observed distribution q of the observed sequence, i.e.

p = ( 1/6 + a/12, 1/6 + a/12, 1/3 - a/12, 1/3 - a/12)
» q = ( 6/16, 6/16, 2/16, 2/16)

This suggests a method to estimate a. The unknown transition probability a can be obtained by solving

min 0 £ a £ 1

||p - q||

If we choose ||.|| to be the L₂-norm then one may consider the following minimization problem:

	min 0 £ a £ 1 ||p - q||22
=	min 0 £ a £ 1 4 S i=1 (p - q)2
=	min 0 £ a £ 1 {a2/36 - 5a/36 + 25/144}

This is a standard constrained least square problem. The optimal value of a in this case is equal to 1. For the problem of statistical inference of a HMM, we refer readers to [3,4].