An Event-based Vision Sensor (EVS) realizes high-speed data output with low latency by limiting the output data to luminance changes from each pixel, combined with information on coordinates and time. With a focus on movement, they can be applied in a wide variety of fields.
What is EVS?
EVS captures movements (luminance changes)
EVS is designed to emulate how the human eye senses light.The human eye functions in such a way that the receptors on the retina, when exposed to light, convert it to visual signals to be sent to the brain. The subsequent neuronal cells identify the light and shade, and the information is conveyed to the visual cortex in the brain via the retinal ganglion cells.
In the EVS the incident light is converted into electric signals in the imager’s light receiving circuit. The signals pass through the amp unit and reach a comparator where the differential luminance data is separated and divided into positive and negative signals which are then processed and output as events.
In the Event-based Vision Sensor, the luminance changes detected by each pixel are filtered to extract only those that exceed the preset threshold value. This event data is then combined with the pixel coordinate, time, and polarity information before being output. Each pixel operates asynchronously, independently from any other.
Figures illustrate how the sensor captures the ball movement.
Event-based Vision Sensor Data Output
Each pixel consists of a light receiving and luminance detection unit. The incident light is converted into a voltage in the light receiving unit. The differential detection circuit in the luminance detection unit detects the changes between the reference voltage and the converted incident light voltage. If the changes are greater than the set threshold value in either a positive or negative direction then the comparator identifies it as an event and this data is output.
The circuit is reset with the detected event luminance as the reference, and the threshold values are set from this new reference voltage in the positive (light) and negative (dark) directions. If the incident light changes in luminance by an amount greater than the value set as a threshold in the positive direction (i.e. the output voltage surpasses the positive threshold) a positive event is output; conversely, if the voltage is lower than the negative threshold, a negative event is output.
(1) The reference voltage and positive/negative thresholds are set.
(2) A negative event is output when the incident light luminance goes below the negative threshold value.
(3) The reference voltage and positive/negative threshold values are reset based on the value at the event output.
(4) Another negative event is output if the incident light luminance further lowers below the negative threshold.
(5) The reference voltage and positive/negative threshold values are reset again based on the value at the second event output.
(6) If subsequently the luminance increases and surpasses the positive threshold value, a positive event is output.
As illustrated in the diagram below, pixels convert incident light luminance into electric voltage logarithmically. This makes it possible for the sensor to detect subtle differences in the low luminance range while it responds to a wide luminance difference in the high luminance range to prevent event saturation, allowing it to realize a wide dynamic range.
This mechanism produces EVS images as shown below (right).
Since changes in the luminance of pixels will occur when a target has moved, images of moving targets will appear as though their outlines have been extracted (photograph was taken by a camera equipped with EVS mounted on an automobile dashboard).
The industry’s smallest*1 pixel: miniaturized high-resolution sensor made possible by utilizing a stacked structure
Unlike conventional technology that has the light-receiving circuit and luminance detection circuit on the same layer, this technology incorporates them on different layers: a pixel chip (upper layer) and logic chip (lower layer) which includes integrated signal processing circuits. These chips are stacked and connected using Cu-Cu technology within each pixel. The industry’s smallest pixel (4.86 μm) is integrated with the logic chip on a 40nm process, resulting in a 1/2.5 type sensor with HD resolution of 1,280 x 720.
*1) Among stacked event-based vision sensors. According to Sony research (as of announcement on September 9, 2021).
Highspeed and lower latency
Each pixel detects luminance changes asynchronously and will output event data immediately. When multiple pixels produce events the arbitration circuit controls the output order based on the earliest-received event. In this way the sensor outputs events as they are generated, making it possible to only output necessary data at the microsecond order while keeping power consumption low.
Built-in Hardware event filter
In order to cater to various applications the sensor is equipped with several filter functions specifically designed for event data.
This feature enables removal of unnecessary events such as periodical events due to LEDs flickering and other events that are highly unlikely to be the outline of a moving object. It can also restrict the data volume when necessary to ensure it falls below the event rate that can be processed in downstream systems.
By turning on the filter (right image), the overall data volume can be reduced while retaining the information required for a given application.
The right image (with the filter turned on) shows the white lines on the road as the significant information.
Examples of Event Signal Processing
(1) Liquid monitoring
(2) Human tracking
(3) Metal process monitoring
(4) 3D measurement
(5) Vibration monitoring
*) Metavision® Intelligence Suite is the Event-Based Vision software developed by Prophesee.
Metavision® is a registered trademark of PROPHESEE S.A.
Visual inspections and contaminant detection inspections
EVS can detect the luminance differences of scratches or stains by capturing relative images of changes in moving products. For example, the application of EVS is expected in inspections such as for scratches and dust, or for damage in materials such as large film wound into rolls.
Their small data sizes and compatibility with machine learning are also notable features. With RGB cameras, the way scratches are perceived will differ depending on the way colors appear, so their data volume will become larger. Comparatively, EVS detects differences in luminance for each pixel, and only use the differential data for output after combining it with coordinate and temporal information, so their data is small in size and high in output speed. In the future, they will be able to contribute to the automation of inspections in combination with machine learning.
Machine abnormality detection
Since EVS can detect and track the movement of objects moving at high speeds, they can identify even minor abnormalities in behavior and output data which is small in size with no latency. They can be applied to systems for detecting abnormalities in combination with machine learning.
In the robotics industry, demand for autonomous robots is currently growing. EVS can detect movement by converting the external characteristics of objects into silhouettes. They are capable of high-speed processing for obstacle detection, so they are expected to be applied in automatically-traveling robots such as robots for automatic transport or home delivery. Their use as care-giving robots is also being investigated at sites for long-term care support.
Monitoring with consideration for privacy
The movement of humans is perceived by EVS as changes in luminance. They do not capture specific details such as human faces, color, or shapes, so they are well suited to applications for safety monitoring with consideration for privacy at medical or long-term care sites.
Also, since their wide dynamic range characteristics allow them to detect movement without being affected by lighting conditions, they can output data which is small in size with no latency. For example, they are able to monitor the movement paths of people even in dark locations where image sensors for security applications have difficulty in capturing images. By combining other types of image sensors with EVS, even more advanced monitoring systems can be created.
Scientific measurement and investigation
EVS is applicable to the scientific measurement, as it can detect high-speed moving objects. In the field of customer behavior investigation, EVS can achieve privacy-conscious investigations since they will only acquire data in response to movement.
Event-based Vision Sensors perceive the movement of individual pixels as changes in luminance and can output only data which has changed with low latency, so they are ideal for gesture tracking such as of hand movements. Their application in combination with ToF image sensors is also greatly expected.
Related Products & Solutions
Image sensors for industrial
Event-based Vision Sensor(EVS)
Find out more about Event-based Vision Sensor.
Product & Service Inquiries
Click here to request for a datasheet/quotation (RFQ)!
* This button will redirect you to the salesforce.com Co., Ltd. website,
which we have entrusted.
Find the latest information on our newsletter for industrial and security applications.