SemanticSegmentor

class SemanticSegmentor(batch_size=8, num_loader_workers=0, num_postproc_workers=0, model=None, pretrained_model=None, pretrained_weights=None, verbose=True, auto_generate_mask=False, dataset_class=<class 'tiatoolbox.models.engine.semantic_segmentor.WSIStreamDataset'>)[source]

Pixel-wise segmentation predictor.

Note, if model is supplied in the arguments, it will ignore the pretrained_model and pretrained_weights arguments.

Parameters

  • model (nn.Module) – Use externally defined PyTorch model for prediction with. weights already loaded. Default is None. If provided, pretrained_model argument is ignored.

  • pretrained_model (str) – Name of the existing models support by tiatoolbox for processing the data. For a full list of pretrained models, refer to the docs. By default, the corresponding pretrained weights will also be downloaded. However, you can override with your own set of weights via the pretrained_weights argument. Argument is case insensitive.

  • pretrained_weights (str) – Path to the weight of the corresponding pretrained_model.

  • batch_size (int) – Number of images fed into the model each time.

  • num_loader_workers (int) – Number of workers to load the data. Take note that they will also perform preprocessing.

  • num_postproc_workers (int) – This value is there to maintain input compatibility with tiatoolbox.models.classification and is not used.

  • verbose (bool) – Whether to output logging information.

  • dataset_class (obj) – Dataset class to be used instead of default.

  • auto_generate_mask (bool) – To automatically generate tile/WSI tissue mask if is not provided.

Examples

>>> # Sample output of a network
>>> wsis = ['A/wsi.svs', 'B/wsi.svs']
>>> predictor = SemanticSegmentor(model='fcn-tissue_mask')
>>> output = predictor.predict(wsis, mode='wsi')
>>> list(output.keys())
[('A/wsi.svs', 'output/0.raw') , ('B/wsi.svs', 'output/1.raw')]
>>> # if a network have 2 output heads, each head output of 'A/wsi.svs'
>>> # will be respectively stored in 'output/0.raw.0', 'output/0.raw.1'

Methods

filter_coordinates

Indicates which coordinate is valid basing on the mask.

get_coordinates

Calculate patch tiling coordinates.

get_reader

Define how to get reader for mask and source image.

merge_prediction

Merge patch-level predictions to form a 2-dimensional prediction map.

predict

Make a prediction for a list of input data.
static filter_coordinates(mask_reader, bounds, resolution=None, units=None)[source]

Indicates which coordinate is valid basing on the mask.

To use your own approaches, either subclass to overwrite or directly assign your own function to this name. In either cases, the function must obey the API defined here.

Parameters

  • mask_reader (VirtualReader) – A virtual pyramidal reader of the mask related to the WSI from which we want to extract the patches.

  • bounds (ndarray and np.int32) – Coordinates to be checked via the func. They must be in the same resolution as requested resolution and units. The shape of coordinates is (N, K) where N is the number of coordinate sets and K is either 2 for centroids or 4 for bounding boxes. When using the default func=None, K should be 4, as we expect the coordinates to be refer to bounding boxes in [start_x, start_y, end_x, end_y] format.

  • resolution (Optional[Union[float, int]]) –

  • units (Optional[str]) –

Returns

List of flags to indicate which coordinate is valid.

Return type

ndarray

Examples

>>> # API of function expected to overwrite `filter_coordinates`
>>> def func(reader, bounds, resolution, units):
...   # as example, only select first bound
...   return np.array([1, 0])
>>> coords = [[0, 0, 256, 256], [128, 128, 384, 384]]
>>> segmentor = SemanticSegmentor(model='unet')
>>> segmentor.filter_coordinates = func
static get_coordinates(image_shape, ioconfig)[source]

Calculate patch tiling coordinates.

By default, internally, it will call the PatchExtractor.get_coordinates. To use your own approach, either subclass to overwrite or directly assign your own function to this name. In either cases, the function must obey the API defined here.

Parameters

  • image_shape (tuple(int), numpy.ndarray) – This argument specifies the shape of mother image (the image we want to) extract patches from) at requested resolution and units and it is expected to be in (width, height) format.

  • ioconfig (IOSegmentorConfig) – Object that contains information about input and ouput placement of patches. Check IOSegmentorConfig for details about available attributes.

Returns

Tuple containing:
patch_inputs (list): A list of corrdinates in

[start_x, start_y, end_x, end_y] format indicating the read location of the patch in the mother image.

patch_outputs (list): A list of corrdinates in

[start_x, start_y, end_x, end_y] format indicating the write location of the patch in the mother image.

Return type

(tuple)

Examples

>>> # API of function expected to overwrite `get_coordinates`
>>> def func(image_shape, ioconfig):
...   patch_inputs = np.array([[0, 0, 256, 256]])
...   patch_outputs = np.array([[0, 0, 256, 256]])
...   return patch_inputs, patch_outputs
>>> segmentor = SemanticSegmentor(model='unet')
>>> segmentor.get_coordinates = func
static get_reader(img_path, mask_path, mode, auto_get_mask)[source]

Define how to get reader for mask and source image.

Parameters

  • img_path (str) –

  • mask_path (str) –

  • mode (str) –

  • auto_get_mask (bool) –

static merge_prediction(canvas_shape, predictions, locations, save_path=None, cache_count_path=None, free_prediction=True)[source]

Merge patch-level predictions to form a 2-dimensional prediction map.

When accumulating the raw prediction onto a same canvas (via calling the function multiple times), save_path and cache_count_path must be the same. If either of these two do not exist, the function will create new files. However, if save_path is None, the function will perform the accumulation using CPU-RAM as storage.

Parameters

  • canvas_shape (numpy.ndarray) – HW of the supposed assembled image.

  • predictions (list) – List of nd.array, each item is a patch prediction, assuming to be of shape HWC.

  • locations (list) – List of nd.array, each item is the location of the patch at the same index within predictions. The location is in the to be assembled canvas and of the form (top_left_x, top_left_y, bottom_right_x, bottom_right_x).

  • save_path (str) – Location to save the assembled image.

  • cache_count_path (str) – Location to store the canvas for counting how many times each pixel get overlapped when assembling.

  • free_prediction (bool) – If this is True, predictions will be modified in place and each patch will be replace with None once processed. This is to save memory when assembling.

Returns

An image contains merged data.

Return type

numpy.ndarray

Examples:

>>> SemanticSegmentor.merge_prediction(
...     canvas_shape=[4, 4],
...     predictions=[
...         np.full((2, 2), 1),
...         np.full((2, 2), 2)],
...     locations=[
...         [0, 0, 2, 2],
...         [2, 2, 4, 4]],
...     save_path=None,
...     free_prediction=False,
... )
array([[1, 1, 0, 0],
    [1, 1, 0, 0],
    [0, 0, 2, 2],
    [0, 0, 2, 2]])
predict(imgs, masks=None, mode='tile', on_gpu=True, ioconfig=None, patch_input_shape=None, patch_output_shape=None, stride_shape=None, resolution=1.0, units='baseline', save_dir=None, crash_on_exception=False)[source]

Make a prediction for a list of input data.

By default, if the input model at the object instantiation time is a pretrained model in the toolbox as well as patch_input_shape, patch_output_shape, stride_shape, resolution, units and ioconfig are None. The method will use the ioconfig retrieved together with the pretrained model. Otherwise, either patch_input_shape, patch_output_shape, stride_shape, resolution, units or ioconfig must be set else a Value Error will be raised.

Parameters

  • imgs (list, ndarray) – List of inputs to process. When using patch mode, the input must be either a list of images, a list of image file paths or a numpy array of an image list. When using tile or wsi mode, the input must be a list of file paths.

  • masks (list) – List of masks. Only utilised when processing image tiles and whole-slide images. Patches are only processed if they are within a masked area. If not provided, then a tissue mask will be automatically generated for whole-slide images or the entire image is processed for image tiles.

  • mode (str) – Type of input to process. Choose from either tile or wsi.

  • ioconfig (IOSegmentorConfig) – Object defines information about input and ouput placement of patches. When provided, patch_input_shape, patch_output_shape, stride_shape, resolution, and units arguments are ignored. Otherwise, those arguments will be internally converted to a IOSegmentorConfig object.

  • on_gpu (bool) – Whether to run model on the GPU.

  • patch_input_shape (tuple) – Size of patches input to the model. The value are at requested read resolution and must be positive.

  • patch_output_shape (tuple) – Size of patches output by the model. The values are at the requested read resolution and must be positive.

  • stride_shape (tuple) – Stride using during tile and WSI processing. The values are at requested read resolution and must be positive. If not provided, stride_shape=patch_input_shape is used.

  • resolution (float) – Resolution used for reading the image.

  • units (str) – Units of resolution used for reading the image. Choose from either level, power or mpp.

  • save_dir (str or pathlib.Path) – Output directory when processing multiple tiles and whole-slide images. By default, it is folder output where the running script is invoked.

  • crash_on_exception (bool) – If True, the running loop will crash if there is any error during processing a WSI. Otherwise, the loop will move on to the next wsi for processing.

Returns

A list of tuple(input_path, save_path) where

input_path is the path of the input wsi while save_path corresponds to the output predictions.

Return type

output (list)

Examples

>>> # Sample output of a network
>>> wsis = ['A/wsi.svs', 'B/wsi.svs']
>>> predictor = SemanticSegmentor(model='fcn-tissue_mask')
>>> output = predictor.predict(wsis, mode='wsi')
>>> list(output.keys())
[('A/wsi.svs', 'output/0.raw') , ('B/wsi.svs', 'output/1.raw')]
>>> # if a network have 2 output heads, each head output of 'A/wsi.svs'
>>> # will be respectively stored in 'output/0.raw.0', 'output/0.raw.1'