Faisal Qureshi
Professor
Faculty of Science
Ontario Tech University
Oshawa ON Canada
http://vclab.science.ontariotechu.ca
© Faisal Qureshi
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Projection of a point in a world plane to the image plane. The location of the point and that of its projection is related via homography.
Generally speaking, points that lie on two planes are related via homography. This also means that the projections of points (that lie on a common plane) in two cameras are related via homography.
How many correspondences do we need to solve $\mathbf{H}$, where
$$ \mathbf{H} = \left[ \begin{array}{ccc} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & h_{33} \end{array} \right]. $$This matrix can only be known upto a scale, so it has 8 degrees of freedom. We need at least 4 correspondences to solve this matrix.
This matrix often cannot be solved using pseudo-inverse technique due to numerical issues.
Re-write homography relationship as homogeneous equations
$$ \begin{align} x'_i &= \frac{h_{11} x_i + h_{12} y_i + h_{13}}{h_{31} x_i + h_{32} y_i + h_{33}} \\ y'_i &= \frac{h_{21} x_i + h_{22} y_i + h_{23}}{h_{31} x_i + h_{32} y_i + h_{33}} \end{align} $$We can then write these as matrix-vector product
$$ \left[ \begin{array}{ccccccccc} x_i & y_i & 1 & 0 & 0 & 0 & -x_i x'_i & -y_i x'_i & - x'_i \\ 0 & 0 & 0 & x_i & y_i & 1 & -x_i y'_i & -y_i y'_i & - y'_i \end{array} \right] \left( \begin{array}{c} h_{11} \\ h_{12} \\ h_{13} \\ h_{21} \\ h_{22} \\ h_{23} \\ h_{31} \\ h_{32} \\ h_{33} \end{array} \right) = \mathbf{0} $$Given $n$ point correspondences between two images, we set up $mathbf{A} \mathbf{x} = 0$ as follows:
$$ \begin{align} \left[ \begin{array}{ccccccccc} x_1 & y_1 & 1 & 0 & 0 & 0 & -x_1 x'_1 & -y_1 x'_1 & - x'_1 \\ 0 & 0 & 0 & x_1 & y_1 & 1 & -x_1 y'_1 & -y_1 y'_1 & - y'_1 \\ &&&&& \vdots &&& \\ x_i & y_i & 1 & 0 & 0 & 0 & -x_i x'_i & -y_i x'_i & - x'_i \\ 0 & 0 & 0 & x_i & y_i & 1 & -x_i y'_i & -y_i y'_i & - y'_i \\ &&&&& \vdots &&& \\ x_n & y_n & 1 & 0 & 0 & 0 & -x_n x'_n & -y_n x'_n & - x'_n \\ 0 & 0 & 0 & x_n & y_n & 1 & -x_n y'_n & -y_n y'_n & - y'_n \end{array} \right] \left( \begin{array}{c} h_{11} \\ h_{12} \\ h_{13} \\ h_{21} \\ h_{22} \\ h_{23} \\ h_{31} \\ h_{32} \\ h_{33} \end{array} \right) &= \mathbf{0} \\ \mathbf{A} \mathbf{x} & = \mathbf{0} \end{align} $$$\mathbf{x}$ is the vector of unknowns.
Extract features
Find matches
Use RANSAC to estimate homography
Perform image stitching