ContextQMD
Back to Chromium

QR Scanner

ios/chrome/browser/qr_scanner/README.md

149.0.7827.211.9 KB
Original Source

QR Scanner

The QR Scanner provides a way of scanning QR codes and bar codes directly from Chrome. It is developed behind the EnableQRCodeReader experimental flag.

[TOC]

Usage

  1. Create a delegate implementing the QRScannerViewControllerDelegate protocol.
  2. Initialize QRScannerViewController with this delegate.
  3. Present the view controller returned by viewControllerToPresent.

Behavior

The QR Scanner is presented as a full-screen view controller displaying a video preview, a control for the camera's torch, and a control for closing the QR scanner.

Presentation

The QR Scanner is presented using a custom transition animation which makes it appear to be originally positioned under the presenting view controller.

Scanning

  • Codes are only recognized inside the viewport.
  • A flash animation is played when a code is recognized.
  • If VoiceOver is enabled, an announcement is played instead of the animation.
  • Scanning a QR code or any other code type which can contain alphanumeric strings places the scanned result in the Omnibox, and the user has to press the "Go" button on the keyboard to load the result. Non-URL strings will be loaded in search.
  • Scanning a bar code type which can only contain numbers will load search results containing the bar code immediately without waiting for user confirmation.

Torch

  • Torch is switched off every time the QR scanner is opened, closed, or the camera session is interrupted.
  • The torch button always reflects the current torch state.
  • The torch button is in a disabled state if the camera does not have torch, the torch is unavailable, or the camera is not yet loaded.

Errors

  • A dialog is displayed if the camera is unavailable, camera permissions are not granted, the camera is in use by another application, or the application is in Split View on iPad.
  • Pressing the "Cancel" button of any error dialog dismisses the QR Scanner view controller.
  • If the camera becomes available when a dialog is presented, the dialog is automatically dismissed.

Entry points

The QR scanner can be accessed from the 3D Touch application shortcuts on supported devices. The SpotlightActions experiment allows the QR scanner to be accessed from Spotlight search. More info about Spotlight actions can be found at go/chrome-ios-spotlight and crbug.com/608733. Planned and rejected entry points are described in the design doc at go/chrome-ios-qr-code.

Tests for QR Scanner are a part of ios_chrome_ui_egtests.

Controller architecture

  • QRScannerViewController is the entry point for the feature. It connects the CameraController and the QRScannerView and is responsible for displaying alerts from QRScannerAlerts.
  • CameraController manages the AVCaptureSession for the camera. It is responsible for loading the camera, listening for camera notifications, receiving the scanned result and informing the QRScannerViewController about changes to the state of the camera or the torch.

Operations performed by CameraController are done on a separate dispatch queue, as recommended by the documentation for AVCaptureSession.

Initialization

QRScannerViewController owns an instance of CameraController and QRScannerView and is their delegate.

  1. The viewControllerToPresent method checks if camera permission is granted by calling checkPermissionsAndLoadCamera of the CameraController.
  2. If the camera permission is denied, an error dialog will be returned from viewControllerToPresent. If the user has not previously granted camera permission to the application, the QRScannerViewController instance will be returned, and an error will be displayed by the QR scanner if the user denies the permission in the system dialog. The error dialog prompts the user to change this setting and includes a link to the Settings app, if available.
  3. If the camera permission is granted, the QRScannerViewController will be returned as the view controller to present, and the CameraController will start loading the camera on a separate dispatch queue.

Camera

Camera initialization is handled by the loadCaptureSession method of the CameraController.

  1. Camera state is set to CAMERA_UNAVAILABLE and an error dialog is displayed if:
    • The back camera of the device is not found,
    • There was an error initializing the camera input,
    • The video input cannot be attached to the AVCaptureSession,
    • The metadata output cannot be attached to the AVCaptureSession,
    • The metadata output does not support QR code recognition.
  2. After a successful initialization, camera state is set to CAMERA_AVAILABLE, which is reported asynchronously to QRScannerViewController, and CameraController starts listening for camera notifications.
  3. Camera starts recording on viewWillAppear.

Torch

  • Torch availability is checked when the camera initialization completes.
  • Torch is considered available, if the properties hasTorch and isTorchAvailable of the AVCaptureDevice are both YES.
  • During initialization, it is also checked if the torch supports the torch modes AVCaptureTorchModeOn and AVCaptureTorchModeOff.
  • Torch mode is set to off on initialization.

Camera preview

The AVCaptureVideoPreviewLayer is created by the QRScannerView:

  1. The QRScannerView is initialized by the QRScannerViewController.
  2. On viewDidLoad, QRScannerViewController calls loadVideoPreviewLayer: with the loaded preview. If the camera is already loaded, CameraController attaches the preview to the AVCaptureSession. Otherwise the preview is attached immediately after the AVCaptureSession is initialized.

Viewport

The rectangle of interest for the metadata output of the capture session is calculated to lie exactly inside the viewport drawn by the QRScannerView. Resetting the viewport causes the video preview to freeze for a short while, that is why the viewport is only set when the preview is hidden.

  1. QRScannerViewController sets the viewport on viewDidAppear, to make sure that the preview layer is of the correct size and position when the viewport is calculated.
  2. If the capture session is loaded, the viewport is set immediately. Otherwise the viewport is set after the capture session is loaded.
  3. CameraController calls the cameraIsReady method of its delegate to notify the QRScannerViewController that the viewport was successfully set and the camera preview can be displayed.

Camera state and notifications

CameraController is listening for the following notifications:

  1. AVCaptureSessionRuntimeErrorNotification, handled by setting the camera state to CAMERA_UNAVAILABLE,
  2. AVCaptureSessionWasInterruptedNotification, handled by setting the camera state to one of:
    • APPLICATION_IN_BACKGROUND,
    • CAMERA_IN_USE_BY_ANOTHER_APPLICATION,
    • MULTIPLE_FOREGROUND_APPS, based on the value of AVCaptureSessionInterruptionReasonKey in the notification's user info.
  3. AVCaptureSessionInterruptionEndedNotification, handled by setting the camera state to CAMERA_AVAILABLE.
  4. AVCaptureDeviceWasDisconnected, handled by setting the camera state to CAMERA_UNAVAILABLE.

Torch state

The current state of the torch is obtained using key-value observing of the AVCaptureDevice object.

  • Torch state is set based on the value of the torchActive property, and the delegate is informed using torchStateChanged:.
  • Torch availability is set based on the values of hasTorch and torchAvailable. Torch is only considered available if both properties are YES and the delegate is informed using torchAvailabilityChanged:.

The delegate sets the value of the torch using setTorchMode: and is informed of the outcome asynchronously.

Scanning

CameraController implements the AVCaptureMetadataOutputObjectsDelegate and receives the scanned result on the main queue.

  • The scanned result must be an AVMetadataMachineReadableCodeObject.
  • Only results which are non-empty strings are passed on to the QRScannerViewController.
  • CameraController checks the type of the scanned code, and if the code can only contain numbers, sets the loadImmediately argument to YES.
  • If a valid code was scanned, the CameraController stops the capture session.

Supported codes (from Machine Readable Object Types):

  • Numeric-only bar codes: UPC-E, EAN-8, EAN-13, Interleaved 2 of 5, ITF-14
  • Alphanumeric bar codes: Code 39, Code 39 Mod 43, Code 93, Code 128
  • 2D alphanumeric codes: PDF417, AztecCode, DataMatrix, QR Code

Views

The QR scanner consists of three views:

  • VideoPreviewView holds the camera preview.
  • PreviewOverlayView holds layers drawing the darker preview overlay and the viewport border.
  • QRScannerView contains the VideoPreviewView, PreviewOverlayView and controls as subviews.

Transition animation

QRScannerTransitioningDelegate implements a custom transition animation: the QRScannerViewController appears to be positioned below its presenting view controller. The presenting view controller slides up for presentation and down for dismissal.

Screen rotation

  • QRScannerView and PreviewOverlayView rotate normally. VideoPreviewView does not rotate: on viewWillTransitionToSize: the view is animated to rotate in the opposite direction. This avoids resetting the viewport rectangle of interest on every screen rotation, because resetting it causes the video to pause for a while. The view is not positioned using AutoLayout.
  • PreviewOverlayView is a square that is sqrt(2)-times bigger than max(width, height) of the QRScannerView, to avoid redrawing the viewport. This also makes the viewport rotate in place. The view is positioned using AutoLayout.

Split View

  • Camera is unavailable in Split View.
  • When the camera becomes available, the viewport rectangle is reset, otherwise the viewport would be in the wrong place when Split View is cancelled.

Accessibility

  • Standard UI elements have accessibility labels and identifiers.
  • The value of the torch button is communicated using its accessibility value.
  • If VoiceOver is on, an accessibility announcement is played when a code is scanned, and the result is loaded after the announcement finishes.
  • If the result is loaded immediately, no additional accessibility notifications are posted.
  • If the result is placed in Omnibox for the user to review, the Omnibox should be focused afterwards.

Metrics

The following metrics are collected:

  • IOS.Spotlight.Action when the user opens the QR scanner from searching for it in Spotlight.
  • ApplicationShortcut.ScanQRCodePressed when the user opens the QR scanner from 3D Touch application shortcuts.
  • MobileQRScannerClose when the user closes the QR scanner without scanning a code.
  • MobileQRScannerError when the user closes the QR scanner from an error dialog.
  • MobileQRScannerScannedCode when the user scans a code.
  • MobileQRScannerTorchOn when the user switches on the torch.

Known issues

Screen rotation on iPad is not handled the same way as on iPhone, as of iOS 9. The counter-rotation animation is not played at the same time as the screen rotation animation, and the camera preview appears to be rotating. This effect is most visible when rotating the screen by 180 degrees, which results in an apparent double-rotation by 360 degrees.

See also

  • go/chrome-ios-qr-code for the original design doc.