Homography¶
Faisal Qureshi
Professor
Faculty of Science
Ontario Tech University
Oshawa ON Canada
http://vclab.science.ontariotechu.ca
Copyright information¶
© Faisal Qureshi
License¶
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Lesson Plan¶
- Homography
- Homography application: Image stitching
- Solving for Homography
Homography¶
Projection of a point in a world plane to the image plane. The location of the point and that of its projection is related via homography.
Generally speaking, a homography relates points on one plane to their images on another plane. This also means that the projections of points (that lie on a common plane) in two cameras are related via homography.
Image Stitching¶
Exercise 1¶
How many correspondences do we need to solve $\mathbf{H}$, where
$$ \mathbf{H} = \left[ \begin{array}{ccc} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & h_{33} \end{array} \right]. $$
Solution Exercise 1¶
This matrix can only be known up to a scale, so it has 8 degrees of freedom. We need at least 4 correspondences to solve this matrix.
We cannot solve for this matrix with the pseudo-inverse. The equations below are homogeneous ($\mathbf{A} \mathbf{x} = \mathbf{0}$), so the pseudo-inverse returns the trivial solution $\mathbf{x} = \mathbf{0}$. Instead, we fix the scale with $\| \mathbf{x} \| = 1$.
Solving for Homography¶
Re-write homography relationship as homogeneous equations
$$ \begin{align} x'_i &= \frac{h_{11} x_i + h_{12} y_i + h_{13}}{h_{31} x_i + h_{32} y_i + h_{33}} \\ y'_i &= \frac{h_{21} x_i + h_{22} y_i + h_{23}}{h_{31} x_i + h_{32} y_i + h_{33}} \end{align} $$
We can then write these as matrix-vector product
$$ \left[ \begin{array}{ccccccccc} x_i & y_i & 1 & 0 & 0 & 0 & -x_i x'_i & -y_i x'_i & - x'_i \\ 0 & 0 & 0 & x_i & y_i & 1 & -x_i y'_i & -y_i y'_i & - y'_i \end{array} \right] \left( \begin{array}{c} h_{11} \\ h_{12} \\ h_{13} \\ h_{21} \\ h_{22} \\ h_{23} \\ h_{31} \\ h_{32} \\ h_{33} \end{array} \right) = \mathbf{0} $$
Given $n$ point correspondences between two images, we set up $\mathbf{A} \mathbf{x} = \mathbf{0}$ as follows:
$$ \begin{align} \left[ \begin{array}{ccccccccc} x_1 & y_1 & 1 & 0 & 0 & 0 & -x_1 x'_1 & -y_1 x'_1 & - x'_1 \\ 0 & 0 & 0 & x_1 & y_1 & 1 & -x_1 y'_1 & -y_1 y'_1 & - y'_1 \\ &&&&& \vdots &&& \\ x_i & y_i & 1 & 0 & 0 & 0 & -x_i x'_i & -y_i x'_i & - x'_i \\ 0 & 0 & 0 & x_i & y_i & 1 & -x_i y'_i & -y_i y'_i & - y'_i \\ &&&&& \vdots &&& \\ x_n & y_n & 1 & 0 & 0 & 0 & -x_n x'_n & -y_n x'_n & - x'_n \\ 0 & 0 & 0 & x_n & y_n & 1 & -x_n y'_n & -y_n y'_n & - y'_n \end{array} \right] \left( \begin{array}{c} h_{11} \\ h_{12} \\ h_{13} \\ h_{21} \\ h_{22} \\ h_{23} \\ h_{31} \\ h_{32} \\ h_{33} \end{array} \right) &= \mathbf{0} \\ \mathbf{A} \mathbf{x} & = \mathbf{0} \end{align} $$
$\mathbf{x}$ is the vector of unknowns.
Estimating $\mathbf{x}$¶
- Estimate using least-squares fitting $$ \mathbf{x}^* = \underset{x}{\operatorname{argmin}} \| \mathbf{A} \mathbf{x} \|^2\ \mathrm{s.t.}\ \| \mathbf{x} \|=1 $$
- Note that the solution is also the right null-space of $\mathbf{A}$, so the solution is the eigenvector corresponding to the smallest eigenvalue of $\mathbf{A}^T \mathbf{A}$.
Image Stitching - Steps¶
Extract features
Find matches
Use RANSAC to estimate homography
Perform image stitching
