Back to Tensorflow

ImageSearcher

tensorflow/lite/g3doc/api_docs/java/org/tensorflow/lite/task/vision/searcher/ImageSearcher.html

2.21.015.3 KB
Original Source

public final class ImageSearcher

Performs similarity search on images.

The API expects a TFLite model with optional, but strongly recommended, TFLite Model Metadata..

  • Input image tensor (kTfLiteUInt8/kTfLiteFloat32)

    • image input of size [batch x height x width x channels].
    • batch inference is not supported (batch is required to be 1).
    • only RGB inputs are supported (channels is required to be 3).
    • if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.
  • Output tensor (kTfLiteUInt8/kTfLiteFloat32)

  • N components corresponding to the N dimensions of the returned feature vector for this output layer.

  • Either 2 or 4 dimensions, i.e. [1 x N] or [1 x 1 x 1 x N].

TODO(b/180502532): add pointer to example model.

TODO(b/222671076): add factory create methods without options, such as createFromFile, once the single file format (index file packed in the model) is supported.

Nested Classes

| class | ImageSearcher.ImageSearcherOptions | Options for setting up an ImageSearcher. |

Public Methods

| static ImageSearcher | createFromBufferAndOptions(ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options) Creates an ImageSearcher instance with a model buffer and ImageSearcher.ImageSearcherOptions.

| | static ImageSearcher | createFromBufferAndOptionsImpl(ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options, int indexFd) | | static ImageSearcher | createFromFileAndOptions(Context context, String modelPath, ImageSearcher.ImageSearcherOptions options) Creates an ImageSearcher instance from ImageSearcher.ImageSearcherOptions.

| | static ImageSearcher | createFromFileAndOptions(File modelFile, ImageSearcher.ImageSearcherOptions options) Creates an ImageSearcher instance.

| | List<NearestNeighbor> | search(TensorImage image, ImageProcessingOptions options) Performs embedding extraction on the provided TensorImage with ImageProcessingOptions, followed by nearest-neighbor search in the index.

| | List<NearestNeighbor> | search(MlImage image) Performs embedding extraction on the provided MlImage, followed by nearest-neighbor search in the index.

| | List<NearestNeighbor> | search(MlImage image, ImageProcessingOptions options) Performs embedding extraction on the provided MlImage with ImageProcessingOptions, followed by nearest-neighbor search in the index.

| | List<NearestNeighbor> | search(TensorImage image) Performs embedding extraction on the provided TensorImage, followed by nearest-neighbor search in the index.

|

Inherited Methods

From class org.tensorflow.lite.task.core.BaseTaskApi

| synchronized void | close() Release the memory allocated from C++ and deregister the library from the static holder.

| | long | getNativeHandle() | | boolean | isClosed() |

From class java.lang.Object

| boolean | equals(Object arg0) | | final Class<?> | getClass() | | int | hashCode() | | final void | notify() | | final void | notifyAll() | | String | toString() | | final void | wait(long arg0, int arg1) | | final void | wait(long arg0) | | final void | wait() |

From interface java.io.Closeable

| abstract void | close() |

From interface java.lang.AutoCloseable

| abstract void | close() |

Public Methods

public static ImageSearcher createFromBufferAndOptions (ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options)

Creates an ImageSearcher instance with a model buffer and ImageSearcher.ImageSearcherOptions.

Parameters

| modelBuffer | a direct ByteBuffer or a MappedByteBuffer of the search model | | options | |

Throws

| IllegalArgumentException | if the model buffer is not a direct ByteBuffer or a MappedByteBuffer | | IOException | if an I/O error occurs when loading the index file | | IllegalStateException | if there is an internal error | | RuntimeException | if there is an otherwise unspecified error |

public static ImageSearcher createFromBufferAndOptionsImpl (ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options, int indexFd)

Parameters

| modelBuffer | | | options | | | indexFd | |

public static ImageSearcher createFromFileAndOptions (Context context, String modelPath, ImageSearcher.ImageSearcherOptions options)

Creates an ImageSearcher instance from ImageSearcher.ImageSearcherOptions.

Parameters

| context | | | modelPath | path of the search model with metadata in the assets | | options | |

Throws

| IOException | if an I/O error occurs when loading the tflite model or the index file | | IllegalArgumentException | if an argument is invalid | | IllegalStateException | if there is an internal error | | RuntimeException | if there is an otherwise unspecified error |

public static ImageSearcher createFromFileAndOptions (File modelFile, ImageSearcher.ImageSearcherOptions options)

Creates an ImageSearcher instance.

Parameters

| modelFile | the search model File instance | | options | |

Throws

| IOException | if an I/O error occurs when loading the tflite model or the index file | | IllegalArgumentException | if an argument is invalid | | IllegalStateException | if there is an internal error | | RuntimeException | if there is an otherwise unspecified error |

public List<NearestNeighbor> search (TensorImage image, ImageProcessingOptions options)

Performs embedding extraction on the provided TensorImage with ImageProcessingOptions, followed by nearest-neighbor search in the index.

ImageSearcher supports the following options:

  • Region of interest (ROI) (through ImageProcessingOptions.Builder.setRoi(Rect)). It defaults to the entire image.
  • image rotation (through ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)). It defaults to ImageProcessingOptions.Orientation.TOP_LEFT.

ImageSearcher supports the following TensorImage color space types:

  • ColorSpaceType.RGB
  • ColorSpaceType.NV12
  • ColorSpaceType.NV21
  • ColorSpaceType.YV12
  • ColorSpaceType.YV21
Parameters

| image | a UINT8 TensorImage object that represents an RGB or YUV image | | options | |

Throws

| IllegalArgumentException | if the color space type of image is unsupported |

public List<NearestNeighbor> search (MlImage image)

Performs embedding extraction on the provided MlImage, followed by nearest-neighbor search in the index.

Parameters

| image | an MlImage object that represents an image |

Throws

| IllegalArgumentException | if the storage type or format of the image is unsupported |

public List<NearestNeighbor> search (MlImage image, ImageProcessingOptions options)

Performs embedding extraction on the provided MlImage with ImageProcessingOptions, followed by nearest-neighbor search in the index.

ImageSearcher supports the following options:

  • Region of interest (ROI) (through ImageProcessingOptions.Builder.setRoi(Rect)). It defaults to the entire image.
  • image rotation (through ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)). It defaults to ImageProcessingOptions.Orientation.TOP_LEFT. MlImage.getRotation() is not effective.
Parameters

| image | a MlImage object that represents an image | | options | configures options including ROI and rotation |

Throws

| IllegalArgumentException | if the storage type or format of the image is unsupported |

public List<NearestNeighbor> search (TensorImage image)

Performs embedding extraction on the provided TensorImage, followed by nearest-neighbor search in the index.

ImageSearcher supports the following TensorImage color space types:

  • ColorSpaceType.RGB
  • ColorSpaceType.NV12
  • ColorSpaceType.NV21
  • ColorSpaceType.YV12
  • ColorSpaceType.YV21
Parameters

| image | a UINT8 TensorImage object that represents an RGB or YUV image |

Throws

| IllegalArgumentException | if the color space type of image is unsupported |