No description has been provided for this image

CSCI 3240U: Computer Vision I¶

Lab 8: Homography and Image Stitching¶

Faisal Z. Qureshi
Faculty of Science, Ontario Tech University
Oshawa ON Canada
http://vclab.science.ontariotechu.ca

Fall 2026

Copyright information¶

© Faisal Qureshi

License¶

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

About this notebook¶

Read the handout, lab8.html, first. This notebook loads the image pair, supplies eight checked correspondences, and gives you a stitch helper. The cells marked TODO are yours; everything else runs as shipped.

Section here Handout
Load the pair Data
Supplied correspondences Part 1
Task 1: homography_dlt Part 1, Task 1
Task 2: four points against eight Part 1, Task 2
Task 3: stitch Part 1, Task 3
Task 4: SIFT and the ratio test Part 2, Task 4
Task 5: RANSAC Part 2, Task 5
Task 6: iteration count, and no RANSAC Part 2, Task 6

Setup¶

In [ ]:
import os

import cv2
import numpy as np
import matplotlib
import matplotlib.pyplot as plt

%matplotlib inline

print('OpenCV    ', cv2.__version__)
print('NumPy     ', np.__version__)
print('Matplotlib', matplotlib.__version__)

Helper code¶

Nothing below is graded. These helpers load images, display them, and draw labelled points. Read them before you use them.

Two conventions matter and they are easy to mix up.

  • OpenCV point coordinates are (x, y), that is (column, row).
  • NumPy array indices are [row, column].

Every point array in this notebook is (N, 2) holding (x, y), in float.

In [ ]:
DATA_FOLDER = '.'


def load_rgb(filename):
    """Read an image and return it as RGB, ready for plt.imshow.

    cv2.imread gives you BGR.  Passing that straight to plt.imshow swaps red and
    blue, which is the classic first bug of every OpenCV lab.  The version of
    this notebook shipped before Fall 2026 had exactly that bug.
    """
    path = os.path.join(DATA_FOLDER, filename)
    bgr = cv2.imread(path)
    if bgr is None:
        raise FileNotFoundError(
            path + '\n'
            'Both images ship beside this notebook.  On Colab, upload '
            'boat1.jpg and boat2.jpg into the working directory, or set '
            'DATA_FOLDER to wherever you put them.')
    return cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)


def show(img, title=None, size=(10, 10), cmap=None):
    """Display one image.  Pass an RGB array, not a BGR one."""
    plt.figure(figsize=size)
    if title:
        plt.title(title)
    if img.ndim == 2 and cmap is None:
        cmap = 'gray'
    plt.imshow(img, cmap=cmap)
    plt.axis('off')
    plt.show()


def show_pair(img_a, img_b, title_a='', title_b='', size=(18, 8)):
    """Display two RGB images side by side."""
    plt.figure(figsize=size)
    for i, (img, title) in enumerate([(img_a, title_a), (img_b, title_b)]):
        plt.subplot(1, 2, i + 1)
        plt.title(title)
        plt.imshow(img)
        plt.axis('off')
    plt.tight_layout()
    plt.show()


def draw_points(img, pts, colour=(255, 0, 0), radius=None, thickness=None,
                label=True):
    """Draw numbered circles on a copy of an RGB image.

    cv2.circle needs integer pixel coordinates.  Handing it NumPy floats raises
    a TypeError in OpenCV 4.10 and later, so round and cast here.
    """
    out = img.copy()
    h, w = out.shape[:2]
    if radius is None:
        radius = max(4, int(round(0.006 * max(h, w))))
    if thickness is None:
        thickness = max(2, radius // 3)
    for i, (x, y) in enumerate(np.asarray(pts, dtype=np.float64)):
        centre = (int(round(x)), int(round(y)))
        cv2.circle(out, centre, radius, colour, thickness)
        if label:
            cv2.putText(out, str(i), (centre[0] + radius, centre[1] - radius),
                        cv2.FONT_HERSHEY_SIMPLEX, radius / 12.0, colour,
                        thickness)
    return out


def stitch(left, right, H):
    """Warp `right` into the frame of `left` with H, and paste both on one canvas.

    H must map points in `right` to points in `left`.  The canvas is sized from
    the warped corners of `right`, unioned with `left`, and everything is
    translated so that no coordinate is negative.  cv2.warpPerspective does the
    backward warping and bilinear interpolation for you.
    """
    h1, w1 = left.shape[:2]
    h2, w2 = right.shape[:2]
    corners = np.float32([[0, 0], [w2, 0], [w2, h2], [0, h2]]).reshape(-1, 1, 2)
    warped = cv2.perspectiveTransform(corners, H).reshape(-1, 2)
    allpts = np.vstack([warped, [[0, 0], [w1, 0], [w1, h1], [0, h1]]])
    xmin, ymin = np.floor(allpts.min(axis=0)).astype(int)
    xmax, ymax = np.ceil(allpts.max(axis=0)).astype(int)
    width, height = xmax - xmin, ymax - ymin
    if width * height > 25_000_000:
        raise ValueError(f'the canvas would be {width} x {height} pixels; '
                         'H is almost certainly wrong')
    T = np.array([[1, 0, -xmin], [0, 1, -ymin], [0, 0, 1]], dtype=np.float64)
    canvas = cv2.warpPerspective(right, T @ H, (int(width), int(height)))
    canvas[-ymin:-ymin + h1, -xmin:-xmin + w1] = left
    return canvas

The image pair¶

boat1.jpg and boat2.jpg are 3888 by 2592 pixels each. Everything below works on a copy scaled by SCALE = 0.25, which is 972 by 648 pixels. Every number in the handout refers to this working resolution, so leave SCALE alone.

In [ ]:
SCALE = 0.25   # working resolution; the handout's numbers assume 0.25

left_full = load_rgb('boat1.jpg')
right_full = load_rgb('boat2.jpg')

left = cv2.resize(left_full, None, fx=SCALE, fy=SCALE,
                  interpolation=cv2.INTER_AREA)
right = cv2.resize(right_full, None, fx=SCALE, fy=SCALE,
                   interpolation=cv2.INTER_AREA)

print('full resolution   ', left_full.shape, right_full.shape)
print('working resolution', left.shape, right.shape)

show_pair(left, right, 'boat1.jpg (left)', 'boat2.jpg (right)')

Supplied correspondences¶

Eight correspondences, in full-resolution pixels. Row $i$ of PTS_LEFT and row $i$ of PTS_RIGHT name the same point in the scene. The cell scales them to the working resolution. Run the next cell and check a few of them by eye: point 3 on the left must sit on the same thing as point 3 on the right.

In [ ]:
PTS_LEFT = [
    [1243, 1585], [3829, 1373], [2385,  215], [2586, 1437],
    [1588,  596], [3049,  858], [1898, 1317], [3233, 1518],
]
PTS_RIGHT = [
    [  31, 1631], [2613, 1385], [1253,  233], [1463, 1457],
    [ 419,  585], [1902,  889], [ 756, 1341], [2081, 1526],
]

pts_left = np.asarray(PTS_LEFT, dtype=np.float64) * SCALE
pts_right = np.asarray(PTS_RIGHT, dtype=np.float64) * SCALE
print(len(pts_left), 'correspondences at the working resolution')
In [ ]:
show_pair(draw_points(left, pts_left, colour=(255, 0, 0)),
          draw_points(right, pts_right, colour=(255, 0, 0)),
          'left: supplied points', 'right: matching points')

Task 1: homography_dlt¶

Each correspondence $(x, y) \to (u, v)$ contributes two rows to $\mathbf{A}$. Solve $\mathbf{A}\mathbf{h} = \mathbf{0}$ with the SVD: $\mathbf{h}$ is the last row of Vt. Reshape it to $3 \times 3$ and scale so that $h_{33} = 1$.

Estimate $\mathbf{H}$ that maps the right image into the frame of the left, so call it as homography_dlt(pts_right, pts_left). Then compare with cv2.findHomography(pts_right, pts_left, 0).

In [ ]:
# TODO (Task 1)
def homography_dlt(src, dst):
    """Return the 3x3 H, with H[2, 2] = 1, mapping src points to dst points."""
    raise NotImplementedError('Task 1: homography_dlt')

Task 2: four points against eight¶

Estimate $\mathbf{H}$ from the first four correspondences and from all eight. For each, report the reprojection error over all eight points, mean and maximum. cv2.perspectiveTransform(pts.reshape(-1, 1, 2), H) applies a homography to a set of points.

With exactly four points, $\mathbf{A}$ is $8 \times 9$ and np.linalg.svd returns only eight singular values; the ninth, which is zero, is not listed.

In [ ]:
# TODO (Task 2)

Your Task 2 answer goes here. What do the extra points buy you?

Task 3: stitch¶

In [ ]:
# TODO (Task 3): H = homography_dlt(pts_right, pts_left); show(stitch(left, right, H))

Part 2: doing it automatically¶

Task 4: SIFT and the ratio test¶

Work on greyscale copies of left and right. Create the detector with cv2.SIFT_create() and call detectAndCompute(gray, None) on each image. Match with cv2.BFMatcher().knnMatch(des_left, des_right, k=2), and keep a match m only if m.distance < 0.8 * n.distance, where n is its second-best match. m.queryIdx indexes the left keypoints and m.trainIdx the right ones.

Show a sample with cv2.drawMatches; drawing all of them is unreadable.

In [ ]:
# TODO (Task 4)

Task 5: RANSAC¶

cv2.findHomography(src, dst, cv2.RANSAC, 3.0) returns H and a mask with one entry per match: 1 for an inlier, 0 for an outlier. Use the right-image points as src and the left-image points as dst, as in Part 1.

In [ ]:
# TODO (Task 5)

Task 6: how many iterations, and what if you skip RANSAC?¶

Compute $N$ from your observed inlier ratio with $s = 4$ and $p = 0.99$, and compare it with OpenCV's default of 2000. Then fit $\mathbf{H}$ to all the putative matches with cv2.findHomography(src, dst, 0) and stitch with it.

In [ ]:
# TODO (Task 6)

Your Task 6 answers go here.

Submission¶

Submit a single Jupyter notebook via Canvas. Run it top to bottom before you submit, so every figure and every number is visible in the file. Code that has not been executed cannot be marked.

No description has been provided for this image