Proposal for Optimal Multi-Spectral Cameras Using Multi-Band Filters

Sony Semiconductor Solutions Corporation (SSS) has proposed the concept of an optimal multi-spectral camera specialized in the detection of contaminants and identification of components by combining a commercially available multi-band filter and an SWIR sensor (IMX990^*) as a solution for issues in the food production industry.

*IMX990 is a high-performance SWIR image sensor powered by SenSWIR™ technology. It is designed so that a single unit can capture a wide range of spectrum from 0.4 μm to 1.7 μm, including visible light.（→SWIR image sensor）

Various spectroscopic methods are adopted by different multi-spectral cameras available on the market today. One of these is a method that uses a prism, diffraction grating, or other spectrometer to split the light coming through a slit; the wavelength information is then line-scanned and output by the image sensor. (See the figure below: Conventional multi-spectral cameras.)

This method is advantageous in that the image resolution is not compromised, but its disadvantages include that cameras are larger and more expensive. Also, due to the line-scan output, the use of the system is limited to contexts in which either the target or the camera is in motion at a constant speed. The system is widely adopted by manufacturers as their scenes of application often involve target products constantly flowing on conveyors.

Against this backdrop, SSS had the idea that, by adopting a spectrometer other than a diffraction grating or prism, it might be possible to create an optimal multi-spectral camera which is small and low cost.

Specifically, in this method, the spectrometer of a multi-spectral camera is replaced with a multi-band filter that transmits multiple different wavelengths. These multi-band filters are relatively inexpensive compared to prisms and diffraction gratings. They can be placed directly onto an image sensor, so it is possible to achieve a line-scan-type spectral camera that is no larger than a standard camera. (See the figure below: SSS’s proposed optimal multi-spectral camera.)

While multi-spectral cameras can be used to detect a variety of contaminants and components, we assume that using only four wavelengths would be sufficient to achieve high-accuracy identification when the camera is restricted to a specific use case. With this as a baseline, we compared the three types of systems as shown below. Optimal multi-spectral cameras offer a low-cost option, and the system is easy to introduce, making it simple to process captured data. The cameras are also small and easy to install.

Let's review some examples of actual inspections for various applications.
The camera used in the following case studies is equipped with a multi-band filter that selects four wavelength bands suitable for food inspection, leveraging the characteristics of components in food to absorb light at specific wavelengths.

Multi-band filters are used to capture images at specific wavelengths corresponding to the absorption of lipids, water, and proteins, which allows them to obtain information that ordinary color cameras cannot. Further, by using image processing appropriate for the subject on the captured multi-spectral images, we have succeeded in visualizing components and contaminants that color cameras or mono-bands struggle distinguish from each other.

This case study assumes the inspection of water content during the drying process in tea leaf production. The level of dryness of tea leaves can be determined by their water content, but this is difficult to identify visually. In this case study, four groups of tea leaves with different water contents are photographed. The water content of the tea leaves is visualized by utilizing the property of water to absorb light at 1,450 nm.

These are example images assuming detection of walnut shell contamination. Walnuts are typically crushed and then sifted to remove the shells, but it is difficult to remove all the cracked shells just by sifting. The remaining shells must be found and removed manually. In a test assuming this manual inspection stage, we used a color camera and an optimal multi-spectral camera for comparison.
In the image taken with a color camera, the walnut kernels and the shells that were not completely removed are almost the same color, and the shells that have a similar shape to the kernels are particularly difficult to identify.
In this case study, the images were captured at four wavelengths selected for distinguishing walnut kernels from walnut shells, and color mapping was applied based on these images so that the contaminants (shells) appeared in red. This resulted in easy identification of contaminants by improving detection accuracy through a combination of data from multiple wavelengths, although the difference can hardly be distinguished in the image taken with a color camera.

Note: Information on the major component analysis algorithm used in this case study will be available in a white paper that is scheduled for publication soon! If you’re interested, please pre-register to download it here.

In the above cases, tests using 2-band and 4-band images were presented. The combination of image processing and images from multiple wavelengths has the potential to further expand the range of inspection applications.
SSS is pursuing an optimal multi-spectral camera designed for various inspections for the food production industry and testing it with various use cases in mind. The optimal multi-spectral camera can also be adapted to applications in other industries by modifying the filter specifications.

If you’re interested in the optimal multi-spectral camera and its applications, contact us here.
You can download a list of cameras equipped with SWIR image sensors here. This list also includes optimal multi-spectral cameras, so it may also be useful if you are looking for a camera.