DFBRegister¶

class DFBRegister(patch_size=(224, 224))[source]¶

Deep Feature based Registration (DFBR).

This class implements a CNN feature based method for registering a pair of histology images, as presented in the paper [1]. This work is adapted from [2].

References

[1] Awan, R., Raza, S.E.A., Lotz, J. and Rajpoot, N.M., 2022. Deep Feature based Cross-slide Registration. arXiv preprint arXiv:2202.09971.

[2] Yang, Z., Dan, T. and Yang, Y., 2018. Multi-temporal remote sensing image registration using deep convolutional features. Ieee Access, 6, pp.38544-38555.

Examples

>>> from tiatoolbox.tools.registration.wsi_registration import DFBRegister
>>> import cv2
>>> df = DFBRegister()
>>> fixed_image = np.repeat(np.expand_dims(fixed_gray, axis=2), 3, axis=2)
>>> moving_image = np.repeat(np.expand_dims(moving_gray, axis=2), 3, axis=2)
>>> transform = df.register(fixed_image, moving_image, fixed_mask, moving_mask)
>>> registered = cv2.warpAffine(
...     moving_gray, transform[0:-1], fixed_gray.shape[:2][::-1]
... )

Initialize DFBRegister.

Methods

compute_feature_distances

Compute feature distance.

estimate_affine_transform

Compute affine transformation matrix.

extract_features

CNN based feature extraction for registration.

feature_mapping

Find mapping between CNN features.

filtering_matching_points

Filter the matching points.

find_points_inside_boundary

Find indices of points lying inside the boundary.

finding_match

Computes matching points.

get_tissue_regions

Extract tissue region.

perform_dfbregister

Perform DFBR to align a pair of image.

perform_dfbregister_block_wise

Perform DFBR to align a pair of images in a block wise manner.

register

Perform whole slide image registration.
Parameters:

patch_size (tuple[int, int])

static compute_feature_distances(features_x, features_y, factor)[source]¶

Compute feature distance.

This function computes Euclidean distance between features of fixed and moving images.

Parameters:

  • features_x (numpy.ndarray) – Features computed for a fixed image.

  • features_y (numpy.ndarray) – Features computed for a moving image.

  • factor (int) – A number multiplied by the feature size for getting the referenced feature size.

Returns:

A feature distance array.

Return type:

numpy.ndarray

static estimate_affine_transform(points_0, points_1)[source]¶

Compute affine transformation matrix.

This function estimates transformation parameters using linear least squares for a given set of matched points.

Parameters:

  • points_0 (numpy.ndarray) – An Nx2 array of points in a fixed image.

  • points_1 (numpy.ndarray) – An Nx2 array of points in a moving image.

Returns:

A 3x3 transformation matrix.

Return type:

numpy.ndarray

extract_features(fixed_img, moving_img)[source]¶

CNN based feature extraction for registration.

This function extracts multiscale features from a pre-trained VGG-16 model for an image pair.

Parameters:
Returns:

A dictionary containing the multiscale features. The expected format is {layer_name: features}.

Return type:

Dict

feature_mapping(features, num_matching_points=128)[source]¶

Find mapping between CNN features.

This function maps features of a fixed image to that of a moving image on the basis of Euclidean distance between them.

Parameters:

  • features (Dict) – Multiscale CNN features.

  • num_matching_points (int) – Number of required matching points.

  • self (DFBRegister)

Returns:

Parameters for estimating transformation parameters.

  • numpy.ndarray - A matching 2D point set in the fixed image.

  • numpy.ndarray - A matching 2D point set in the moving image.

  • numpy.ndarray - A 1D array, where each element represents quality of each matching point.

Return type:

tuple

filtering_matching_points(fixed_mask, moving_mask, fixed_matched_points, moving_matched_points, quality)[source]¶

Filter the matching points.

This function removes the duplicated points and the points which are identified outside the tissue region.

Parameters:

  • fixed_mask (numpy.ndarray) – A binary tissue mask for the fixed image.

  • moving_mask (numpy.ndarray) – A binary tissue mask for the moving image.

  • fixed_matched_points (numpy.ndarray) – (N, 2) array of coordinates.

  • moving_matched_points (numpy.ndarray) – (N, 2) array of coordinates.

  • quality (numpy.ndarray) – An array representing quality of each matching point.

  • self (DFBRegister)

Returns:

  • np.ndarray - Filtered matching points for a fixed image.

  • np.ndarray - Filtered matching points for a moving image.

  • np.ndarray - Quality of matching points.

Return type:

tuple

static find_points_inside_boundary(mask, points)[source]¶

Find indices of points lying inside the boundary.

This function returns indices of points which are enclosed by an area indicated by a binary mask.

Parameters:
Returns:

Indices of points enclosed by a boundary.

Return type:

numpy.ndarray

static finding_match(feature_dist)[source]¶

Computes matching points.

This function computes all the possible matching points between fixed and moving images.

Parameters:

feature_dist (numpy.ndarray) – A feature distance array.

Returns:

  • numpy.ndarray - An array of matching points.

  • numpy.ndarray - An array of floating numbers representing quality of each matching point.

Return type:

tuple

static get_tissue_regions(fixed_image, fixed_mask, moving_image, moving_mask)[source]¶

Extract tissue region.

This function uses binary mask for extracting tissue region from the image.

Parameters:

  • fixed_image (numpy.ndarray) – A fixed image.

  • fixed_mask (numpy.ndarray) – A binary tissue mask for the fixed image.

  • moving_image (numpy.ndarray) – A moving image.

  • moving_mask (numpy.ndarray) – A binary tissue mask for the moving image.

Returns:

  • numpy.ndarray - A cropped image containing tissue region from fixed image.

  • numpy.ndarray - A cropped image containing tissue mask from fixed image.

  • numpy.ndarray - A cropped image containing tissue region from moving image.

  • numpy.ndarray - A cropped image containing tissue mask from moving image.

  • tuple - Bounds of the tissue region.

    • int - Top (start y value)

    • int - Left (start x value)

    • int - Bottom (end y value)

    • int - Right (end x value)

Return type:

tuple

perform_dfbregister(fixed_img, moving_img, fixed_mask, moving_mask)[source]¶

Perform DFBR to align a pair of image.

This function aligns a pair of images using Deep Feature based Registration (DFBR) method.

Parameters:
Returns:

Return type:

tuple

perform_dfbregister_block_wise(fixed_img, moving_img, fixed_mask, moving_mask)[source]¶

Perform DFBR to align a pair of images in a block wise manner.

This function divides the images into four equal parts and then perform feature matching for each part from the tissue and moving images. Matching features of all the parts are then concatenated for the estimation of affine transform.

Parameters:
Returns:

Return type:

tuple

register(fixed_img, moving_img, fixed_mask, moving_mask, transform_initializer=None)[source]¶

Perform whole slide image registration.

This function aligns a pair of images using Deep Feature based Registration (DFBR) method.

Parameters:
Returns:

An affine transformation matrix.

Return type:

numpy.ndarray