CSCI 3240U --- Computer Vision I

Lab 7 --- Interest Points and Local Descriptors

Faisal Z. Qureshi
Faculty of Science, Ontario Tech University
Oshawa ON Canada
http://vclab.science.ontariotechu.ca

Fall 2026

Copyright information

© Faisal Qureshi

License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Outline

• Image derivatives
• Image gradients
• Gradient magnitudes and angles
• Quantization
• Histogram computation

Tasks

1

Complete the various steps needed to compute the rotation invariant descriptor. You can use the following image to test your code

DATA_FOLDER = '.'
img = cv.imread(os.path.join(DATA_FOLDER, 'local-features-construction-2.jpg'), 0)
img = cv.resize(img, (256, 256))

plt.figure(figsize=(5,5))
plt.imshow(img, cmap='gray');
plt.plot([74],[175],'r.')

2

Combine this code in the following method

descriptor, I_patch, grad_patch, grad_patch_weighted, grad_color_patch = my_local_descriptor(img, r, c)

This method takes in an image img and row r and column c location. The method returns a 128-dimensional descriptor. It should also return $4$ $16\times16$ patches around (r,c) locations. I_patch is image intensity patch, grad_patch is gradient magnitudes, grad_patch_weighted is weighted gradient magnitudes, and grad_color_patch returns the colorized gradient magnitudes.

3

Use this method to compute descriptors as follows:

DATA_FOLDER = '.'
img1 = cv.imread(os.path.join(DATA_FOLDER, 'local-features-construction-2.jpg'), 0)
img1 = cv.resize(img1, (256, 256))

img2 = cv.imread(os.path.join(DATA_FOLDER, 'local-features-construction-1.jpg'), 0)
img2 = cv.resize(img2, (256, 256))

desc1, p1, g1, gw1, gc1 = my_local_descriptor(img1, 175, 74)
desc2, p2, g2, gw2, gc2 = my_local_descriptor(img2, 197, 106) 
desc3, p3, g3, gw3, gc3 = my_local_descriptor(img2, 100, 54)

4

Next compute the distances between the three descriptors as follows:

X = np.vstack([desc1, desc2, desc3])

from scipy.spatial import distance_matrix
distance_matrix(X, X)

5

Comment whether or not desc1 is closer to desc2 than to desc3.

Starter code

import cv2 as cv
import numpy as np
import scipy as sp
from scipy import signal
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import os

DATA_FOLDER = '.'
img = cv.imread(os.path.join(DATA_FOLDER, 'local-features-construction-2.jpg'), 0)
img = cv.resize(img, (256, 256))

plt.figure(figsize=(5,5))
plt.imshow(img, cmap='gray')
plt.plot([74],[175],'r.');

Image derivatives

plt.figure(figsize=(15,15))
plt.subplot(221)
plt.title('Ix')
plt.imshow(Ix, cmap='gray')
plt.subplot(222)
plt.title('Iy')
plt.imshow(Iy, cmap='gray');
plt.subplot(223)
plt.title('Grad')
plt.imshow(Igrad, cmap='gray')
plt.subplot(224)
plt.title('Angle visualization')
plt.imshow(Igrad_color);

16x16 window patch

Extract a $16x16$ window around location $(x,y)$. Notice how images are stored in numpy. Unlike the 2D $xy$ coordinate system that we are used to, here $x$ corresponds to columns and increase from left to right. $y$ corresponds to rows and increase from top to bottom.

plt.figure(figsize=(5,5))
plt.imshow(grad_patch, cmap='gray');

Weighted gradient magnitudes

Get gradient magnitude and multiply it by a $16 \times 16$ Gaussian window of $\sigma=2.5$

plt.figure(figsize=(15,5))
plt.subplot(131)
plt.title('Gradient magnitude')
plt.imshow(grad_patch, cmap='gray')
plt.subplot(132)
plt.title('Gaussian kernel')
plt.imshow(g, cmap='gray')
plt.subplot(133)
plt.title('Gradient magnitude weighted')
plt.imshow(grad_patch_weighted, cmap='gray');

Finding dominant angle

Quantize angles to 36 bins (from 0 to 360 degrees). Find the dominant angle (the angle that is most repeated). Subtract dominant angle from every angle to achieve rotation invariant.

Construct feature descriptor

Divide $16 \times 16$ into $16$ blocks. Each block is $4 \times 4$. Within each block, quantize angles into 8 bins (from 0 to 360 degrees). Computed gradient magnitude weighted, normalized, angle histograms. Each block thus returns an $8$-dimensional vector. Concatenate $16$ $8$-dimensional vectors to construct a $128$-dimensional vector.

Putting it all together

Now put the code together in one convenient function that computes the 128 dimensional descriptor at a particular location of a given grayscale image.

def my_local_descriptor(img, r, c):

    descriptor = None 
    I_patch = None 
    grad_patch = None 
    grad_patch_weighted = None 
    grad_color_patch = None
    
    return descriptor, I_patch, grad_patch, grad_patch_weighted, grad_color_patch

Check for rotation invariance

DATA_FOLDER = '.'
img1 = cv.imread(os.path.join(DATA_FOLDER, 'local-features-construction-2.jpg'), 0)
img1 = cv.resize(img1, (256, 256))

img2 = cv.imread(os.path.join(DATA_FOLDER, 'local-features-construction-1.jpg'), 0)
img2 = cv.resize(img2, (256, 256))

plt.figure(figsize=(10,5))
plt.subplot(121)
plt.imshow(img1, cmap='gray');
plt.plot([74],[175],'r.')
plt.subplot(122)
plt.imshow(img2, cmap='gray');
plt.plot([106],[196],'r.');
plt.plot([54],[100],'b.');

desc1, p1, g1, gw1, gc1 = my_local_descriptor(img1, 175, 74)
desc2, p2, g2, gw2, gc2 = my_local_descriptor(img2, 197, 106) 
desc3, p3, g3, gw3, gc3 = my_local_descriptor(img2, 100, 54)

dominant angle = 130.0 degrees
dominant angle = 150.0 degrees
dominant angle = 260.0 degrees

plt.figure(figsize=(15, 15))
plt.subplot(331)
plt.imshow(p1, cmap='gray')
plt.subplot(332)
plt.imshow(p2, cmap='gray')
plt.subplot(333)
plt.imshow(p3, cmap='gray')
plt.subplot(334)
plt.imshow(gc1, cmap='gray')
plt.subplot(335)
plt.imshow(gc2, cmap='gray')
plt.subplot(336)
plt.imshow(gc3, cmap='gray')
plt.subplot(337)
plt.imshow(gw1, cmap='gray')
plt.subplot(338)
plt.imshow(gw2, cmap='gray')
plt.subplot(339)
plt.imshow(gw3, cmap='gray');

Lets compute distance between the three descriptors, and confirm that descriptor 1,2 are closer to each other than descriptors 1,3 and 2,3.

Submission

Include code and results on the images found in the above folder in a single jupyter notebook. Submit via canvas.