Homogoraphy: A Case Study in Model Fitting

Faisal Qureshi
Professor
Faculty of Science
Ontario Tech University
Oshawa ON Canada
http://vclab.science.ontariotechu.ca

© Faisal Qureshi

License

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Homography

Projection of a point in a world plane to the image plane. The location of the point and that of its projection is related via homography.

Generally speaking, points that lie on two planes are related via homography. This also means that the projections of points (that lie on a common plane) in two cameras are related via homography.

Image Stitching

Exercise 1

How many correspondences do we need to solve $\mathbf{H}$, where

$$ \mathbf{H} = \left[ \begin{array}{ccc} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & h_{33} \end{array} \right]. $$

Solution Exercise 1

This matrix can only be known upto a scale, so it has 8 degrees of freedom. We need at least 4 correspondences to solve this matrix.

This matrix often cannot be solved using pseudo-inverse technique due to numerical issues.

Solving for Homography

Re-write homography relationship as homogeneous equations

$$ \begin{align} x'_i &= \frac{h_{11} x_i + h_{12} y_i + h_{13}}{h_{31} x_i + h_{32} y_i + h_{33}} \\ y'_i &= \frac{h_{21} x_i + h_{22} y_i + h_{23}}{h_{31} x_i + h_{32} y_i + h_{33}} \end{align} $$

We can then write these as matrix-vector product

$$ \left[ \begin{array}{ccccccccc} x_i & y_i & 1 & 0 & 0 & 0 & -x_i x'_i & -y_i x'_i & - x'_i \\ 0 & 0 & 0 & x_i & y_i & 1 & -x_i y'_i & -y_i y'_i & - y'_i \end{array} \right] \left( \begin{array}{c} h_{11} \\ h_{12} \\ h_{13} \\ h_{21} \\ h_{22} \\ h_{23} \\ h_{31} \\ h_{32} \\ h_{33} \end{array} \right) = \mathbf{0} $$

Given $n$ point correspondences between two images, we set up $mathbf{A} \mathbf{x} = 0$ as follows:

$$ \begin{align} \left[ \begin{array}{ccccccccc} x_1 & y_1 & 1 & 0 & 0 & 0 & -x_1 x'_1 & -y_1 x'_1 & - x'_1 \\ 0 & 0 & 0 & x_1 & y_1 & 1 & -x_1 y'_1 & -y_1 y'_1 & - y'_1 \\ &&&&& \vdots &&& \\ x_i & y_i & 1 & 0 & 0 & 0 & -x_i x'_i & -y_i x'_i & - x'_i \\ 0 & 0 & 0 & x_i & y_i & 1 & -x_i y'_i & -y_i y'_i & - y'_i \\ &&&&& \vdots &&& \\ x_n & y_n & 1 & 0 & 0 & 0 & -x_n x'_n & -y_n x'_n & - x'_n \\ 0 & 0 & 0 & x_n & y_n & 1 & -x_n y'_n & -y_n y'_n & - y'_n \end{array} \right] \left( \begin{array}{c} h_{11} \\ h_{12} \\ h_{13} \\ h_{21} \\ h_{22} \\ h_{23} \\ h_{31} \\ h_{32} \\ h_{33} \end{array} \right) &= \mathbf{0} \\ \mathbf{A} \mathbf{x} & = \mathbf{0} \end{align} $$

$\mathbf{x}$ is the vector of unknowns.

Estimating $\mathbf{x}$

  • Estimate using least-square fitting $$ \mathbf{x}^* = \underset{x}{\operatorname{argmax}} \| \mathbf{A} \mathbf{x} \|^2\ \mathrm{s.t.}\ \| \mathbf{x} \|=1 $$
  • Note that the solution is also the right null-space of $\mathbf{A}$, so the solution is the eigenvector corresponding to the smallest eigenvalue of $\mathbf{A}^T \mathbf{A}$.

Image Stitching

  • Estimate homography
  • Use it to fill in the colors from the "other" image

Image Stitching - Steps

Extract features

Find matches

Use RANSAC to estimate homography

Perform image stitching

In [ ]: