python/examples/advanced/Image_Logging.ipynb
🚩 Create a free WhyLabs account to get more value out of whylogs!
Did you know you can store, visualize, and monitor whylogs profiles with the WhyLabs Observability Platform? Sign up for a free WhyLabs account to leverage the power of whylogs and WhyLabs together!
Whylogs has optional support for logging data about images. This example demonstrates how to use whylogs with image data.
Whylogs uses Pillow for working with images. To install whylogs with the optional image support enabled, use the following:
# Note: you may need to restart the kernel to use updated packages.
%pip install 'whylogs[image,whylabs]'
This will install all the dependencies whylogs needs to work with image data.
The log_image() function provides a simple interface for logging images.
from PIL import Image
from whylogs.extras.image_metric import log_image
img1 = Image.effect_mandelbrot((256, 256), (-3, -2.5, 2, 2.5), 9)
img2 = Image.effect_mandelbrot((256, 256), (-3, -2.5, 2, 2.5), 20)
display(img1)
display(img2)
results = log_image(img1)
print(results.view().get_column("image").to_summary_dict())
As you can see above, just passing in an Image results in a column named "image" in the profile. You can pass in a list of images, which will append an index to each column name:
results = log_image([img1, img2])
print(list(results.view().get_columns()))
print(results.view().get_column("image_1").to_summary_dict())
You can change the default name if you like:
results = log_image([img1, img2], default_column_prefix="awesome_image")
print(list(results.view().get_columns()))
print(results.view().get_column("awesome_image_0").to_summary_dict())
You can also pass a dictionary of images to give each "column" a unique name:
results = log_image({"left_camera": img1, "right_camera": img2})
print(results.view().get_column("right_camera").to_summary_dict())
whylogs will include any EXIF tags with numeric or string values in the image profile. PIL's EXIF facilities are rudimentary, so we'll use the piexif library to construct EXIF tags for this example. See the Advanced Image Logging section for how to control which EXIF tags are tracked.
%pip install piexif
import piexif
zeroth_ifd = {
piexif.ImageIFD.Artist: u"someone",
piexif.ImageIFD.XResolution: (96, 1),
piexif.ImageIFD.YResolution: (96, 1),
piexif.ImageIFD.Software: u"piexif"
}
exif_ifd = {
piexif.ExifIFD.DateTimeOriginal: u"2099:09:29 10:10:10",
piexif.ExifIFD.LensMake: u"LensMake",
piexif.ExifIFD.Sharpness: 65535,
piexif.ExifIFD.LensSpecification: ((1, 1), (1, 1), (1, 1), (1, 1)),
}
exif_dict = {"0th": zeroth_ifd, "Exif": exif_ifd}
exif_bytes = piexif.dump(exif_dict)
img2.save("out.jpg", exif=exif_bytes)
img3 = Image.open("out.jpg")
print(img3._getexif())
results = log_image(img3)
print(results.view().get_column("image").to_summary_dict())
The log_image() function provides a flexible interface for logging image data. If you want to log images along with other types of data, you can do so by setting up a custom DatasetSchema to specify which columns contain images. For this example, we'll compute a couple of features from the image and log them along with the image itself in a Pandas DataFrame. We subclass the StandardResolver to use an ImageMetric for any image columns, and the standard metrics for any other columns.
from typing import Dict
from PIL.ImageStat import Stat
import pandas as pd
import whylogs as why
from whylogs.core.datatypes import DataType
from whylogs.core.metrics import Metric
from whylogs.core.resolvers import StandardResolver
from whylogs.core.schema import DatasetSchema, ColumnSchema
from whylogs.extras.image_metric import ImageMetric, ImageMetricConfig
class ImageResolver(StandardResolver):
def resolve(self, name: str, why_type: DataType, column_schema: ColumnSchema) -> Dict[str, Metric]:
if "image" in name:
return {ImageMetric.get_namespace(): ImageMetric.zero(column_schema.cfg)}
return super(ImageResolver, self).resolve(name, why_type, column_schema)
schema = DatasetSchema(resolvers=ImageResolver(), default_configs=ImageMetricConfig())
stats = Stat(img1)
df = pd.DataFrame({"median": stats.median, "sum": stats.sum, "images": img1})
results = why.log(df, schema=schema).view()
print(results.get_column("median").to_summary_dict())
print(results.get_column("sum").to_summary_dict())
print(results.get_column("images").to_summary_dict())
By default, any numerical or string valued EXIF tags will be included in the image profile. You can control which tags are included via the allowed_exif_tags and forbidden_exif_tags members of the ImageMetricConfig. These are both sets of strings naming the allowed or forbidden EXIF tag names. If allowed_exif_tags is non-empty, only the tags it contains will be included in the profile. Any tags listed in forbidden_exif_tags will be excluded from the profile (handy in case any of the tags are considered PII). In this example, we'll exclude the Artist tag from the profile.
no_pii_config = ImageMetricConfig(forbidden_exif_tags={"Artist"})
no_pii_schema = DatasetSchema(resolvers=ImageResolver(), default_configs=no_pii_config)
results = log_image(img3, schema=no_pii_schema)
print(results.view().get_column("image").to_summary_dict())
The profiles produced by log_image() or log() with a DatasetSchema that includes an ImageMetric can be sent to whylabs just like any whylogs profile. Edit this cell to use the organization and model ID you setup when you created your whylabs account.
import getpass
import os
from datetime import datetime
from typing import Dict
from whylogs.api.writer.whylabs import WhyLabsWriter
# set your org-id here - should be something like "org-xxxx"
os.environ["WHYLABS_DEFAULT_ORG_ID"] = "org-XXXX"
# set your datased_id (or model_id) here - should be something like "model-xxxx"
os.environ["WHYLABS_DEFAULT_DATASET_ID"] = "model-XXXX"
# set your API key here
print("Enter your WhyLabs API key")
os.environ["WHYLABS_API_KEY"] = getpass.getpass()
print("Using API Key ID: ", os.environ["WHYLABS_API_KEY"][0:10])
profile = log_image(img3, schema=no_pii_schema).profile()
profile.set_dataset_timestamp(datetime.now())
view = profile.view()
writer = WhyLabsWriter()
writer.write(profile=view)
Each tracked EXIF tag becomes a feature when viewed in the whylabs profile.