Intelligent Vision Sensors(AI Sensor)
Technology
Edge AI & Sensing

Intelligent Vision Sensors(AI Sensor)

  • Home
  • Technology
  • Edge AI & Sensing
  • Intelligent Vision Sensors(AI Sensor)

Overview

The Intelligent Vision Sensor is a revolutionary image sensor that enables high-speed edge AI processing within the sensor unit, on top of image processing.
We have built an AI engine and dedicated memory into the logic chip. By stacking this logic chip and pixel chip leveraging Sony Semiconductor Solutions Corporation(SSS)’s stacking technology, we have successfully integrated AI processing function into a tiny sensor.

This has made it possible to reduce the amount of data transfer, which was a challenge for conventional systems that transmit image data to the cloud for AI processing, and also to address privacy concerns and reduce power consumption and telecommunications costs.

Technical Features

World's first*1 image sensor equipped with AI processing functionality

The pixel chip is back-illuminated and has approximately 12.3 effective megapixels for capturing information across a wide angle of view. In addition to the conventional image sensor operation circuit, the logic chip is equipped with Sony's original DSP(Digital Signal Processor) dedicated to AI signal processing, and memory for the AI model. This configuration eliminates the need for high-performance processors or external memory, making it ideal for edge AI systems.

*1) Among image sensors. According to research by Sony (as of an announcement on May 14, 2020).

World's first image sensor equipped with AI processing functionality

Metadata output

Signals acquired by the pixel chip are run through an ISP (Image Signal Processor) and AI processing is done in the process stage on the logic chip, and the extracted information is output as metadata, reducing the amount of data handled. Ensuring that image information is not output helps to reduce security risks and address privacy concerns. In addition to the image recorded by the conventional image sensor, users can select the data output format according to their needs and uses, including ISP format output images (YUV/RGB) and ROI (Region of Interest) specific area extract images.

Metadata output

Data output format selectable to meet various needs

High-speed AI processing

When a video is recorded using a conventional image sensor, it is necessary to send data for each individual output image frame for AI processing, resulting in increased data transmission and making it difficult to deliver real-time performance. The new sensor products from Sony perform ISP processing and high-speed AI processing (3.1 milliseconds processing for MobileNet V1*2) on the logic chip, completing the entire process in a single video frame. This design makes it possible to deliver high-precision, real-time tracking of objects while recording video.

*2) An image analysis AI model for object recognition on mobile devices.

High-speed AI processing

Example of real-time tracking with product and task at a register

Selectable AI model

Users can write the AI models of their choice to the embedded memory and can rewrite and update it according to its requirements or the conditions of the location where the system is being used. For example, when multiple cameras employing this product are installed in a retail location, a single type of camera can be used with versatility across different locations, circumstances, times, or purposes. When installed at the entrance to the facility it can be used to count the number of visitors entering the facility; when installed on the shelf of a store it can be used to detect stock shortages; when on the ceiling it can be used for heat mapping store visitors (detecting locations where many people gather), and the like. Furthermore, the AI model in a given camera can be rewritten from one used to detect heat maps to one for identifying consumer behavior, and so on.

Selectable AI model

Example of camera usages in a facility

Related Products & Solutions

  • Image sensor Edge AI Sensing Platform AITRIOS™

    Click here for information on AITRIOS™, an Edge AI Sensing platform equipped with this technology.
    *) AITRIOS and AITRIOS logos are the registered trademarks or trademarks of Sony Group Corporation or its affiliated companies.

Contact Us

For inquiries about Sony Semiconductor Solutions Group and products / solutions, specifications, quotation / purchase requests, etc., please contact us using the Inquiry form from the button below.