INDUSTRIAL

Image Sensor

[Wishing to Record Emotional Moments Just as You See Them] Efforts to Further Increase the Performance of Image Sensors for Mobile

January 23, 2023

”Image sensors” are semiconductors that convert light which has entered from a lens into electrical signals. The market for image sensors is currently continuing to grow and expand. According to research by Sony Semiconductor Solutions Corporation (SSS), their global compound annual growth rate (CAGR) will increase by 10% from FY2022 to FY2030, and SSS is now in the process of acquiring a top global share.*

SSS’s share of the market on a monetary base was 43% in FY2021, and their goal is to reach 60% in FY2025 without fail.

This business growth is being driven by image sensors for mobile, which are installed in smartphones and other devices. They are packed full of new technology for the smartphone age in addition to the knowledge and expertise that SSS has cultivated over many years, as they provide emotional experiences to everyday of end users across the world.

What are the strengths of SSS‘s image sensors for mobile? We asked Seiji Nakata from the Mobile Systems Business Division, which is involved in the development of image sensors equipped with leading-edge pixel technology, and Fumiaki Suzuki, who is responsible for business management operations.

*) According to Sony research
Reference: https://www.sony-semicon.com/en/company/group/index.html

Nakata Masashi

Sony Semiconductor Solutions Corporation
Mobile Systems Business Division

Profile:Joined Sony Corporation in 2006. After being responsible for the development of light collecting structures for image sensors, was involved in image quality design which included signal processing. After this, introduced image plane phase technology for uses such as mobile applications. In 2015, served as project leader for the IMX318. In 2017, transferred to Sony Interactive Entertainment and was involved in the production of live-action VR content using leading-edge cameras in collaboration with Sony Music. From 2019, returned to his current business division to promote the development of image sensors for mobile. In 2021, became Senior Manager of a department responsible for processes from product development to mass production, which is his current position.

Suzuki Fumiaki

Sony Semiconductor Solutions Corporation
Mobile Systems Business Division

Profile:Joined Sony Corporation in 2005. After positions as PMO of camera platform development for α, Handycam, and Cyber-Shot, and as President staff, from 2010 became responsible for new business planning, and cloud services and application planning. In 2014, transferred to semiconductor sales & marketing and after gaining experience there, in 2016 was transferred to San Jose. Contributed to greatly improving the profit rate of MMIC (monolithic microwave integrated circuit) businesses and acquiring new analog LSI businesses in North America, and in 2019 returned to his current business division. In the following year, as Senior Manager of new business development, was involved in branding as well as alliance and contract support.

What are the technological strengths and benefits of image sensors for mobile, cultivated through development over more than 50 years?

From CCD to CMOS, it is no exaggeration to say that the development of image sensors by the Sony Group reflects the history of the evolution of image sensors throughout the world.

Mr. Nakata proudly explains, “For images to be output from image sensors, they must pass through the pipeline consisting of pixels that receive light, circuits which convert that light into digital values, and signal processing to put the finishing touches on their image quality. Each of these processes includes technology that is a source of pride for the SSS.”

Specifically, what points are particularly notable?

“Even after creating the world’s first back-illuminated CMOS image sensor in 2009, we have led the industry with our pixel technology that has achieved a high SNR (signal to noise ratio), wide dynamic range, and high-speed autofocus (AF). Our unique characteristics also include circuit technology for high-speed readout with low noise, and signal processing technology that is capable of high-quality output.” (Nakata)

Since the start of research and development of CCD, the technology accumulated during the Sony Group’s involvement with image sensors for more than 50 years has been applied also to mobile applications, and this can be considered to be the source of the Group’s strength.

Larger sensor sizes to capture beautiful night scenes or starry skies

So, how is SSS’s image sensor technology for mobile useful to people?

In recent years, the most common request for smartphone cameras as indicated by everyone alike, has been to be able to take beautiful photographs even in dark places.

Mr. Nakata explains as follows about the mechanism for taking photographs and the role played by image sensors.

“To take clear photographs or videos, it is necessary for large amounts of light to be taken in by a sensor and for information to be output with low noise. However, even with low noise, the photographs or videos will be unacceptable if they are out of focus or there is camera shake. There are many requirements for image sensors in order to prevent this, such as outputting information from sensors to ensure proper focus, or releasing the shutter at high speeds to prevent shake or distortion.”

SSS therefore focused on a shift to larger sizes in smartphone cameras to fulfill the needs of end users. This shift to larger sizes actually refers to increasing the area of image sensors.

Illustration 拡大 拡大アイコン

“For example, when photographing night scenes, caution must be taken to prevent camera shake, and when attempting to photograph a starry sky, only the brightest stars are the limit that can be captured using small optical sizes. However, if the light receiving surface area is enlarged by increasing the size of sensors, camera shake can be lessened since the shutter time can be made that much shorter. Also, a greater amount of light can be taken in during the same shutter time, making it possible to photograph even dark targets.”

Enlarging the sensor size can provide benefits not only for photography in dark places, but in bright conditions as well. That benefit is the prevention of blowout.

“Photodiodes that store electrons generated by photoelectric conversion can be made larger, so blowout in bright situations can be prevented. This means that larger sensor sizes can allow images to be accurately recorded by properly perceiving the conditions of dark places, and preventing blowout even in bright places which require dynamic range.” (Nakata)

There is also a separate reason for the background behind the advancement of larger sensor sizes. For smartphone cameras, the shift to systems with multiple cameras are installed was common for a long time, but they are now reaching their peak.

The shift to larger sizes is a fundamental step for advancement together with multiple cameras, and is a technology capable of resolving issues for both end users and manufacturers.

New technology for further evolution of smartphone cameras

Although the trend toward larger sensor sizes is predicted to continue from here on, issues related to it are also emerging. The dimensional area of mobile devices is limited, so with consideration for matters such as the convenience of users, the thickness created by protrusions of smartphone camera components cannot be increased to be thicker than it currently is. So then, what should be done?

This is where new, original technology developed by SSS can be applied. One example is “2-layer transistor pixel” which were announced in December 2021.

Illustration 拡大 拡大アイコン

“In image sensors up until now, photodiodes as well as transistors to drive their pixels were formed on the same substrate, So the area for each other was limited. With 2-layer transistor pixels, however, the photodiode and transistor are formed on separate substrates, so increasing photodiode volume can be created and the dynamic range can be widened. In addition, amplifier transistors are also enlarged making it possible to significantly reduce noise.” (Nakata)

With this technology, there will be no blowout or blackout even in scenes with large differences in brightness such as backlit locations, and high quality photographs with low noise can be taken even in dark situations such as indoor spaces and night scenery.

Another example of our technology is “all-pixel autofocus.” To improve photography in dark locations, one point that is just as important as SNR is properly aligning the focus with the target. In the past, a method for allocating a small area of an image sensor to pixels for acquiring focus information was used, and based on the information obtained from these specialized pixels, images were brought into focus by a subsequent camera system.

Illustration 拡大 拡大アイコン

However, since the accuracy of focus information is dependent on the area covered by these specialized pixels, noise in dark locations made it impossible to achieve sufficient performance. This issue can be resolved by the “Dual PD” and “Octa PD” methods for all-pixel autofocus.

“Dual PD” is technology where there are pairs of two PDs (photodiodes) on the left and right sides that are treated as single pixels, which can be placed over an entire image sensor.

Mr. Nakata emphasizes, “This allows focus information to be obtained using 100% of the area, making it easier to focus properly even in dark places. Furthermore, it makes it possible to focus on a desired location, such as the eyes of people or small targets located far away.”

In ”Octa PD,” the pixels of Dual PD are lined up in 2 x 2 units of the same color. By reading out 2 x 2 pixels as single pixels in dark places, pixels which are essentially four times larger can be obtained with the same SNR. The characteristics of Octa PD are that while maintaining support for high sensitivity and HDR (high dynamic range), high-speed autofocus can be achieved simultaneously regardless of the brightness of a target. In bright situations, the 2 x 2 pixels can also be read out separately to prioritize resolution instead of SNR.

Illustration 拡大 拡大アイコン

“To be selected by users” - this is the wish that led to the creation of the new brand “LYTIA™.”

In these ways, SSS will take the lead in the image sensor sector by creating hybrids of various types of technology. We have begun a new initiative to allow us to provide emotional experiences to people around the world.

”Although we have a high market share and our technological capabilities are recognized by smartphone manufacturers, we believe that many general customers (smartphone users) are not particularly knowledgeable about image sensors themselves. We want those users to be aware that SSS is supporting the technology of smartphone cameras, and we wish to change the market environment so that they will select and purchase smartphones which are equipped with image sensors from SSS,” says Mr. Suzuki.

Starting from November, we have therefore started an initiative to strengthen our branding and raise our recognition in general society. This is the “LYTIA” product brand, which was created with the goal of answering the wishes of users to have more freedom in expressing and sharing emotional moments, and to bring them even more creative experiences than expected.

“We started investigating the launch of this brand more than two years ago, and continually held many discussions on it. The result that we finally reached is LYTIA. We want to unleash the creativity of smartphone users, and improve their awareness that excellent photographs can be taken using smartphones equipped with LYTIA.” (Suzuki)

Our unending efforts to record photographs with scenes just as you see them

Mr. Nakata commented enthusiastically that one future challenge is to pursue higher image quality. For example, there are still many people who use interchangeable lens cameras or video cameras to record events such as their children’s sports festivals or presentations, signaling that the image quality of smartphone cameras is still insufficient. Mr. Nakata also noted with some frustration that even in the area of VR, which has recently been becoming more and more active in various fields such as entertainment, the poor image quality of stretched video images is highly apparent.

”If the image sensors that we ourselves have developed are incorporated into smartphones numbering 100 million or more units and are used to take photographs, an enormous amount of photographs and videos will be created in our world. We therefore have an immense sense of duty and responsibility toward the memories that users record and their image quality.” (Nakata)

Mr. Suzuki also continues,

”Users take photographs at moments when their hearts are moved. So we must ask, can our image sensors really allow users to capture images with scenes just as they see them? This is what we want to consider thoroughly.”

The enthusiasm poured into SSS’s image sensor development to successfully record the emotions and excitement of people just as they see them shows no signs of stopping.

Related links