DeepFeatureExtractor

class DeepFeatureExtractor(batch_size=8, num_loader_workers=0, num_postproc_workers=0, model=None, pretrained_model=None, pretrained_weights=None, dataset_class=<class 'tiatoolbox.models.engine.semantic_segmentor.WSIStreamDataset'>, *, verbose=True, auto_generate_mask=False)[source]

Generic CNN Feature Extractor.

AN engine for using any CNN model as a feature extractor. Note, if model is supplied in the arguments, it will ignore the pretrained_model and pretrained_weights arguments.

Parameters:

  • model (nn.Module) – Use externally defined PyTorch model for prediction with weights already loaded. Default is None. If provided, pretrained_model argument is ignored.

  • pretrained_model (str) – Name of the existing models support by tiatoolbox for processing the data. By default, the corresponding pretrained weights will also be downloaded. However, you can override with your own set of weights via the pretrained_weights argument. Argument is case-insensitive. Refer to tiatoolbox.models.architecture.vanilla.CNNBackbone for list of supported pretrained models.

  • pretrained_weights (str) – Path to the weight of the corresponding pretrained_model.

  • batch_size (int) – Number of images fed into the model each time.

  • num_loader_workers (int) – Number of workers to load the data. Take note that they will also perform preprocessing.

  • num_postproc_workers (int) – This value is there to maintain input compatibility with tiatoolbox.models.classification and is not used.

  • verbose (bool) – Whether to output logging information.

  • dataset_class (obj) – Dataset class to be used instead of default.

  • auto_generate_mask (bool) – To automatically generate tile/WSI tissue mask if is not provided.

Examples

>>> # Sample output of a network
>>> from tiatoolbox.models.architecture.vanilla import CNNBackbone
>>> wsis = ['A/wsi.svs', 'B/wsi.svs']
>>> # create resnet50 with pytorch pretrained weights
>>> model = CNNBackbone('resnet50')
>>> predictor = DeepFeatureExtractor(model=model)
>>> output = predictor.predict(wsis, mode='wsi')
>>> list(output.keys())
[('A/wsi.svs', 'output/0') , ('B/wsi.svs', 'output/1')]
>>> # If a network have 2 output heads, for 'A/wsi.svs',
>>> # there will be 3 outputs, and they are respectively stored at
>>> # 'output/0.position.npy'   # will always be output
>>> # 'output/0.features.0.npy' # output of head 0
>>> # 'output/0.features.1.npy' # output of head 1
>>> # Each file will contain a same number of items, and the item at each
>>> # index corresponds to 1 patch. The item in `.*position.npy` will
>>> # be the corresponding patch bounding box. The box coordinates are at
>>> # the inference resolution defined within the provided `ioconfig`.

Initialize DeepFeatureExtractor.

Methods

predict

Make a prediction for a list of input data.
predict(imgs, masks=None, mode='tile', ioconfig=None, patch_input_shape=None, patch_output_shape=None, stride_shape=None, resolution=1.0, units='baseline', save_dir=None, *, on_gpu=True, crash_on_exception=False)[source]

Make a prediction for a list of input data.

By default, if the input model at the time of object instantiation is a pretrained model in the toolbox as well as patch_input_shape, patch_output_shape, stride_shape, resolution, units and ioconfig are None. The method will use the ioconfig retrieved together with the pretrained model. Otherwise, either patch_input_shape, patch_output_shape, stride_shape, resolution, units or ioconfig must be set - else a Value Error will be raised.

Parameters:

  • imgs (list, ndarray) – List of inputs to process. When using “patch” mode, the input must be either a list of images, a list of image file paths or a numpy array of an image list. When using “tile” or “wsi” mode, the input must be a list of file paths.

  • masks (list) – List of masks. Only utilised when processing image tiles and whole-slide images. Patches are only processed if they are within a masked area. If not provided, then a tissue mask will be automatically generated for each whole-slide image or all image tiles in the entire image are processed.

  • mode (str) – Type of input to process. Choose from either tile or wsi.

  • ioconfig (IOSegmentorConfig) – Object that defines information about input and output placement of patches. When provided, patch_input_shape, patch_output_shape, stride_shape, resolution, and units arguments are ignored. Otherwise, those arguments will be internally converted to a IOSegmentorConfig object.

  • on_gpu (bool) – Whether to run the model on the GPU.

  • patch_input_shape (IntPair) – Size of patches input to the model. The values are at requested read resolution and must be positive.

  • patch_output_shape (tuple) – Size of patches output by the model. The values are at the requested read resolution and must be positive.

  • stride_shape (tuple) – Stride using during tile and WSI processing. The values are at requested read resolution and must be positive. If not provided, stride_shape=patch_input_shape is used.

  • resolution (Resolution) – Resolution used for reading the image.

  • units (Units) – Units of resolution used for reading the image.

  • save_dir (str) – Output directory when processing multiple tiles and whole-slide images. By default, it is folder output where the running script is invoked.

  • crash_on_exception (bool) – If True, the running loop will crash if there is any error during processing a WSI. Otherwise, the loop will move on to the next wsi for processing.

  • self (DeepFeatureExtractor)

Returns:

A list of tuple(input_path, save_path) where input_path is the path of the input wsi while save_path corresponds to the output predictions.

Return type:

list

Examples

>>> # Sample output of a network
>>> from tiatoolbox.models.architecture.vanilla import CNNBackbone
>>> wsis = ['A/wsi.svs', 'B/wsi.svs']
>>> # create resnet50 with pytorch pretrained weights
>>> model = CNNBackbone('resnet50')
>>> predictor = DeepFeatureExtractor(model=model)
>>> output = predictor.predict(wsis, mode='wsi')
>>> list(output.keys())
[('A/wsi.svs', 'output/0') , ('B/wsi.svs', 'output/1')]
>>> # If a network have 2 output heads, for 'A/wsi.svs',
>>> # there will be 3 outputs, and they are respectively stored at
>>> # 'output/0.position.npy'   # will always be output
>>> # 'output/0.features.0.npy' # output of head 0
>>> # 'output/0.features.1.npy' # output of head 1
>>> # Each file will contain a same number of items, and the item at each
>>> # index corresponds to 1 patch. The item in `.*position.npy` will
>>> # be the corresponding patch bounding box. The box coordinates are at
>>> # the inference resolution defined within the provided `ioconfig`.