### Rectangular matrix non oriented inverse approximation

Posted:

**Tue May 24, 2016 10:23 am**Hello,

I am currently trying to use linear algebra to solve equations involving probability distributions and marginalization : P(X | Y) = \sum_z P(X | z) P(z | Y).

Such marginalization can be done using matrices.

Let denote A the matrix encoding P(X | Y), B the matrix encoding P(X | Z) and C for P(Z | Y).

I now have 3 columns stochastic matrices (summing at 1 for each column) and the above marginalization is equivalent to A = B.C in linear algebra.

My objective is to find P(Z | Y) using this equality, thus try to compute C by the equality B^-1 A = B^-1 B C = C.

However, since the B matrix is not squared, due to discrete random variables with different domain sizes, the inverse does not exist.

Thus, i tried to substitute it with the Moore Penrose pseudo-inverse : B^+ A = B^+ B C.

However again, it appears that i have less values in my random variables associated to rows than the one associated to columns.

As a consequence, the pseudo inverse i compute is automatically a right one. Thus, i can have guarantees on the product B B^+ but not B^+ B which is of interest to me.

Of course, if i change the domain of the problematic random variable, to have a B squared matrix or full rank by column, i can compute what i want.

But it seems "unfair" that the solution of my problem be dependent on the representation i have of my random variables !!

My question is thus the following : is it possible to obtain a left pseudo inverse instead of a right one with a rectangle full rank matrix by rows ?

Or, can we compute an approximation of the inverse which is not oriented, so that i can use it on the left hand size of my equation?

I know this is quite contrary to what Linear Algebra says in books but maybe there is something we can still do in practice.

Thanks in advance for your help.

AC.

I am currently trying to use linear algebra to solve equations involving probability distributions and marginalization : P(X | Y) = \sum_z P(X | z) P(z | Y).

Such marginalization can be done using matrices.

Let denote A the matrix encoding P(X | Y), B the matrix encoding P(X | Z) and C for P(Z | Y).

I now have 3 columns stochastic matrices (summing at 1 for each column) and the above marginalization is equivalent to A = B.C in linear algebra.

My objective is to find P(Z | Y) using this equality, thus try to compute C by the equality B^-1 A = B^-1 B C = C.

However, since the B matrix is not squared, due to discrete random variables with different domain sizes, the inverse does not exist.

Thus, i tried to substitute it with the Moore Penrose pseudo-inverse : B^+ A = B^+ B C.

However again, it appears that i have less values in my random variables associated to rows than the one associated to columns.

As a consequence, the pseudo inverse i compute is automatically a right one. Thus, i can have guarantees on the product B B^+ but not B^+ B which is of interest to me.

Of course, if i change the domain of the problematic random variable, to have a B squared matrix or full rank by column, i can compute what i want.

But it seems "unfair" that the solution of my problem be dependent on the representation i have of my random variables !!

My question is thus the following : is it possible to obtain a left pseudo inverse instead of a right one with a rectangle full rank matrix by rows ?

Or, can we compute an approximation of the inverse which is not oriented, so that i can use it on the left hand size of my equation?

I know this is quite contrary to what Linear Algebra says in books but maybe there is something we can still do in practice.

Thanks in advance for your help.

AC.