“Along with today’s trend of lower cost and higher performance industrial cameras, higher requirements are put forward on CMOS image sensors, and it is necessary to design a system-on-chip (SoC) to achieve this goal. To achieve this goal, it is necessary to integrate multiple image processing tasks into a single device through 3D chip stacking and back side illuminated (BSI) technology. In the future, there will be solutions with sophisticated machine learning and proprietary intelligent computing chips combined with image capture functions to create a compact high-speed computing vision system.
“
Author: Gareth Powell, Senior Marketing Engineer, Teledyne-e2v
Summary:
Along with today’s trend of lower cost and higher performance industrial cameras, higher requirements are put forward on CMOS image sensors, and it is necessary to design a system-on-chip (SoC) to achieve this goal. To achieve this goal, it is necessary to integrate multiple image processing tasks into a single device through 3D chip stacking and back side illuminated (BSI) technology. In the future, there will be solutions with sophisticated machine learning and proprietary intelligent computing chips combined with image capture functions to create a compact high-speed computing vision system.
However, before realizing the integration of new large-scale technologies, two major development obstacles must be removed-thermal management and power consumption of the chip.
Now, the advanced front side illuminated (FSI) CMOS sensor integrates analog and digital functions, realizing a solution with both cost and efficiency. To achieve these goals, the key lies in the clever separation of various factors that are conducive to system performance and embedding in the image sensor SOC. Here, the role of separating image system applications and existing image processing devices such as CPUs, FPGAs, and DSPs is the core factor, as the duplication of functions will lead to increased costs. The development of a standard image SOC product that can provide a feasible solution for the target application market to achieve mass production (to achieve the lowest production cost) requires an in-depth dialogue between the two modules beforehand.
The main market participants of terminal camera products, including hardware, software, system builders and optical engineers, as well as a multi-disciplinary image sensor development team, are all contributing their knowledge of semiconductor technology and terminal camera product technology and applications. Looking for innovative product solutions.
This article will introduce a series of CMOS image sensors for barcode readers and other embedded vision applications, their application examples, and some future development trends.
Barcode C The most popular coding system and related reading technology overview
In industries that consume retail and online shopping on holidays and seasonality, logistics, transportation, manufacturing, and wholesale industries scan more than 5 billion barcodes every day. As the world’s first barcode appeared on a pack of chewing gum in the 1970s, it is now clearly the most popular readable coding system, and new applications are constantly being developed. One-dimensional barcodes are the first barcodes to appear (and due to the advancement of printing technology, the area is getting smaller), and it is still the mainstream technology, which can be found in UPC (Unique Product Code) applications in retail, transportation and logistics, and other industries. Two-dimensional barcodes come in many different specifications, providing more programmable data than one-dimensional codes: one-dimensional codes can load up to 20-25 characters, depending on the type of barcode; two-dimensional codes can load more than 2000 at most character. In addition to the general function of writing product information and details, two-dimensional barcodes are also written with checksums and other correction techniques to ensure greater fault tolerance for incorrectly printed or damaged barcodes. Two-dimensional codes have been widely used in some specific industries, such as automated manufacturing, direct part marking (DPM) of parts, and so on.
The upgrade of two-dimensional barcode reading technology started about 15 years ago, because it can read two-dimensional codes and one-dimensional codes at the same time, and has become the mainstream of the market today.
Figure 1: Example of barcode
Figure 2: Not all QR code readers have the same functions
Not all QR code readers have the same functions
Barcode verification and decoding systems are rapidly developing and continuously improving to provide faster, more compact, lower price and more powerful reading functions.
Although laser-based one-dimensional readers are still in production and use, the most significant technological advancement in reading systems comes from two-dimensional readers. Through the image sensor, the two-dimensional reader enables it to provide significant arithmetic capabilities, bringing additional functions that were not possible before. These functions include taking photos and recording videos, as well as adding more advanced functions. Common examples include document scanning, OCR (orthogonal character recognition), object recognition, size measurement, and more.
Teledyne-e2v’s image sensor is a unique product facing this market, providing more advantages than a variety of two-dimensional sensor options. One of the main reasons is that it is designed specifically for bar code reading, rather than a product generally targeted at the multi-purpose, consumer or automotive market. This means a precise and powerful solution that can meet all the requirements of market-leading barcode reader products.
Teledyne-e2v has recently developed a series of small-pixel low-noise global shutter CMOS image sensor products with unique features that can bring to the market applications of Automatic Data Collection System (ADCS) and Automatic Identification (AI) Solutions for significant cost savings and/or performance improvements. In this market segment, although the cost of the sensor unit is the most important factor, cost reduction solutions such as lighting/optical lenses also need to be considered.
Figure 3: Snappy is an innovative application-oriented method for CMOS image sensing in industrial barcode readers
• 2mp and 5mp resolution image sensor
• CRA 12° and 0° available in black and white and color versions
• Small pixel high-performance 2.8µm low-noise global shutter pixel, with intra-pixel correlated double sampling (CDS) technology
• MIPI CSI-2 interface for non-sticky embedded imaging system
• 4×1.2Gbits channel (optional minimum single channel)
• On-chip automatic bar code detection, intelligent viewfinder window function to realize fast and accurate bar code identification
• Automatically adjust fast self-exposure mode
The QR code reading system requires very fast frame capture to avoid smear. This requires the shortest possible exposure time. On the other hand, to obtain the maximum depth of field (DOF) or scanning range, lenses with very small optical apertures (generally F/8 or smaller) are often used. The very small number of photons that can enter the image sensor pixel combined with a short integration time means that barcode reading can be performed in low-brightness applications (see Figure 5). The global shutter is also good for reading moving barcodes.
The main sensor parameters that affect the performance of terminal readers are therefore particularly suitable for bar code reading applications. Figure 4 lists some of the main sensor/barcode reading performance requirements and shows the advantages of the Snappy sensor series as an example of a dedicated CMOS image sensor.
I/O performance | Snappy 2MP and 5MP | Application advantage |
Pixel type | 2.8µ | Existing smallest GS pixel, providing uncompromising E/O performance |
Long product life (latest model) | ||
Global shutter efficiency (GSE) / parasitic photosensitivity (PLS) | 99.98% | Higher activity tolerance |
Increase production capacity for logistics and retail applications | ||
Peak quantum efficiency (%) | 56 | High sensitivity |
Best scanning performance in its class | ||
Dynamic range (DR)[dB] | 63 | + Patented on-chip high dynamic range (HDR) |
Better reading ability on reflective objects | ||
Total read noise [e] | ~3e- | Low brightness signal-to-noise ratio performance |
Reduce lighting/power consumption | ||
*Dark signal non-uniformity (DSNU) [e-/s] @65℃ | Better high temperature performance | |
Provides powerful reading performance under high ambient temperature |
* The dark signal increases twice as much at every 6 to 8°C temperature rise
Figure 4: Snappy’s main specifications for barcode scanning
Signal-to-noise ratio
Irradiation (photon/pixel)
Figure 5: The low-brightness signal-to-noise ratio of the Snappy sensor provides advantages for reducing the optical energy consumption and cost of system lighting
The effect of temperature rise
If you carefully observe the differences between the various components that constitute noise parameters at a temperature of 25°C, and the conditions of these components when the temperature rises above 65°C, the performance of some components has limitations. The influence of temperature rise should be considered in the selection process. The read noise of a fixed spatial rank is one of the particularly important parameters of barcode reading. Considering that the shape of fixed-pattern noise is similar to straight and horizontal lines, they are easily confused with barcodes, or may add wrong information when reading the barcode in the image.
The Snappy series of image sensors use advanced semiconductor technology. There are only a few dark signal photons at 25°C, and only 77 photons per second are measured even at 65°C. This helps the row and column built-in fixed-pattern noise cancellation algorithm, even at high operating temperatures, can achieve only a few percent of fixed-pattern noise.
Very low read noise (combining time and space elements) is typically 3 photons. It will not deteriorate even in a high temperature environment. If the performance of the sensor decreases at high temperatures, it means that more lighting is needed, and the introduction increases the cost of the system.
Unique black and white + color pixel filter mode-combines the advantages of low brightness and high sensitivity data of black and white and color pixels
The sensor can use colored pixels to facilitate the addition of additional object/label identification functions, and provide more security features to avoid spoofing or the situation where the barcode itself cannot be read. However, because of the lower transmission characteristics of the organic color filter of the color sensor, and the need to combine red, green and blue pixels to produce “color” pixels, this means that compared to monochrome image sensors, color Zen has a better performance. Low spatial resolution and lower sensitivity. Teledyne-e2v’s Jade image sensor is an interesting innovation. It uses monochrome pixels, but adds a color pixel to every four monochrome pixels. In this way, the spatial resolution and sensitivity essential for reading barcodes can be preserved, while lower-resolution color images can be captured at the same time.
Figure 6: Innovative color sensing applications do not need to compromise on reading performance
Exclusive features for innovative embedded applications
Achieving Snappy barcode reading is not only a result of frame read rate. Although fixed noise is a limiting factor, the Snappy sensor has not compromised for this. The sensor provides excellent performance with a depth of 8 bits close to 120 frames per second. The unique power-on mode ensures that the device is in the power-on or standby state when the first image (or fast self-exposure sub-image) is captured under the SNR specification. This is not a standard function of global shutter CMOS sensors for general drivers or other applications, because it means that the system must discard multiple full-frame images before it is fully stabilized and reaches the signal-to-noise ratio function value indicated in the data sheet. This unique ability to read the first frame after power-on can provide a differentiating factor for the camera, achieve the highest speed barcode reading, provide end users with “snapshot” scanning, and achieve higher productivity for enterprises. Following are the two most innovative patented functions invented by the Teledyne-e2v imaging team at Snappy’s Teledyne-e2v imaging team: They are designed for ultra-high-speed barcode reading, identification and interpretation applications for terminal product scanning.
1. Fast Self Exposure (FSE) mode (used for Snappy 2MP and 5MP CMOS sensors):
The fast self-exposure mode allows to optimize the exposure time under changing light (see Figure 7). Compared with the traditional auto exposure mode, FSE brings more advantages of integration time and powerful functions, including full user programming and providing end users with stable and fast reading advantages, adaptive to any light source or dynamic light source environment, and the frame rate Almost no effect.
Figure 7: New on-chip automatic exposure method for barcode reading and all machine vision applications
The patented FSE mode uses multiple on-chip components to achieve the following functions:
(a) The unique vertical analog-to-digital converter (ADC) allows 4 different exposure periods to be set in a continuous line segment, and then repeated across the entire array to produce 4 low-resolution images with different exposure values. This function can also be used as a powerful high dynamic range image capture function.
(b) Lateral incremental sub-sampling, maximum 1/64 line
(c) The on-chip statistical data includes saturated pixel values, and a 16-bin histogram output is provided at the same time, and the data of the open frame or area can be directly read in the image footer
(d) The viewfinder window (ROI) mode can support FSE sub-frames, multiple regions, and regions in regions
(e) Fine control can use histogram value, average value and a combination of the two
(f) Programmable buffer provides intuitive user control and settings
These functions bring the advantage of scanning speed to terminal applications, because the FSE mode generally only uses less than 10% of the frame period. The traditional embedded automatic exposure control (AEC) of other CMOS sensors uses asymptotic technology to avoid flashing and provide the target image, making the image fusion speed slower. In the whole process, a large amount of frame is consumed, making the speed unable to meet the requirements of barcode reading applications.
2. Smart ROI mode (for Snappy 5MP sensor):
The Smart ROI uses an on-chip algorithm to detect one or more barcodes on the side picture. The barcode decoding image processing system needs to separate the barcode-carrying range from the rest in order to process the useful part. This work is generally carried out on FPGA or CPU, because this task requires a large number of gates/real-time clock (RTC) and processing power, resulting in additional high costs and complex technical constraints encountered when selecting a processing engine.
Figure 8: Use the Smart_ROI function to detect multiple moving barcodes on the image sensor
Embedding this barcode detection function into the sensor can achieve overall cost savings. This is not only because the processing overhead is significantly reduced, but also because the task is completed in the sensor without processing through other digital signal terminals to realize the system Performance and stability advantages. The one-dimensional code or two-dimensional code detected in the valid frame will be in the form of a block, and X/Y coordinates will be added as part of the readable information in the image footnote area (invisible). The sensor can detect multiple areas (or barcodes) at the same time, as well as other codes, such as reading printed characters in optical character recognition (OCR) applications. This function can still work effectively even in applications where the barcode/object/camera is moving.
Significant cost savings and system simplification are the main advantages. The 5MP sensor is mainly used for high-end barcode applications, because it requires a larger lens and stronger processing power to perform 5MP real-time image processing/decoding, which offsets the main advantages provided by a larger scanning range or area. However, the small pixels of the Snappy series sensors and the savings in processing overhead brought about by the on-chip smart viewfinder window can achieve a smaller system-level cost. Due to its low power consumption and the ability to identify multiple bar code symbols in a single frame, this product is becoming a market driver for high-resolution sensors, and it is an obvious advantage in the e-commerce logistics industry.
The fast self-exposure and smart viewfinder functions can work on the Snappy 5MP sensor at the same time to ensure that it can still provide fast and powerful operating capabilities in an environment where the ambient light is constantly changing.
The Snappy sensor series has been designed and optimized to meet the needs of low-cost, low-power systems. The on-chip processing data of the sensor, which was also mentioned above, can significantly improve the working performance. Although it not only satisfies barcode reading applications and markets, but also satisfies other types of machine vision (MV) applications, including inspection, measurement, optical character recognition, etc., it can also benefit from the performance and embedded functions of the Snappy sensor. Other applications include embedded vision systems, IoT edge devices, drones, augmented reality, biometric systems, and more.
Figure 9: The Snappy sensor series is suitable for other machine vision, intelligent IoT and other industrial computing vision applications
Snappy sensor integrated lens and advanced micro-motor-based autofocus provide added value
Consumer-grade camera optical modules are not suitable for the harsh and long-term working environment of B2B industrial applications. In addition, for example, a lens used for bar code reading generally has specialized performance and optical characteristics, and requires a maximum working distance, while optical aberrations such as optical modulation transfer function (MTF) need to be minimized in order to effectively adjust features The barcode with a size smaller than the Nyquist frequency is decoded. The MIPI optical module (MOM) integrated with the powerful Snappy image sensor and high-performance lens can increase the value of the system and save development time and cost.
Figure 10: MIPI optical module (MOM) with integrated fixed focus lens and Snappy sensor
The MIPI optical module is an ideal solution for embedded vision applications. It allows the use of customized lenses to a certain extent, providing excellent performance and flexibility for small modules of 20mm x 20mm. Teledyne-e2v now provides end users with the first 2MP MIPI optical module samples, and plans to launch a 5MP version, as well as a powerful and lightweight 2MP autofocus version based on micro-motor technology. The biggest advantage of autofocus is that compared with fixed-focus optical systems, it allows the use of larger optical apertures to achieve the same level or better scanning range or working distance, but requires significantly less lighting power consumption. Future 2MP MIPI optical modules will provide more performance and/or cost improvements for industrial imaging applications. The details of the new product will be announced in mid-2020. However, there is now an evaluation platform that uses the new open-loop “multi-focus” function, which can provide the maximum working range and maximum frame rate, and has a MEMS autofocus component that can be used in the F-store Snappy 2MP demonstration kit.
New trends and changes brought about by the explosive growth of e-commerce
The huge double-digit compound annual growth rate (CAGR) target (greater than 25% per year) brought about by the explosive growth of e-commerce not only brings changes to the operation of logistics centers, but also provides protection for traditional physical retail points. The retail market is facing a huge shift, and needs to provide a better customer experience and shorten the checkout time by using an unmanned automatic “self-scanning” system. The key to the success of these systems is not only reliable barcode recognition and decoding capabilities, but also more refined object recognition tasks that require more use of color imaging devices.
Figure 11: The explosive growth of e-commerce has LED to new trends and changes in the requirements and functions of CMOS sensors
To fully realize the high growth potential, the first prerequisite is to achieve higher speed or higher productivity scanners and cameras. A higher resolution sensor with a wide screen will allow faster reading speed and achieve a larger surface area for reading codes (multiple packages and barcodes in the same image), so that in the future, a single sensor will cover the entire warehouse area.
Regarding image sensors, we see the advantages of the application of flexible sensor technology, which can reduce the complexity of external optical components and reduce the existing limitations on small pixels such as the diffraction limit of the lens. This development can provide two advantages, one is to simplify and save the optical cost, and the other is to allow small pixels below 2.5µ to reach the optical diffraction limit of the lens without reducing the MTF.
In warehouse applications, the demand for a small package and scanning the bar code on the small package often stems from the ever-increasing efficiency goals of e-commerce distribution centers. Every square centimeter of storage and transportation space must be maximized. To realize the three-dimensional size monitoring of the entire transportation and supply chain of the shipment, so that the relevant code/text label is read through the QR code, it is now necessary to use two separate cameras, one for three-dimensional (mostly using structured light sources or based on three-dimensional Technology of stereo vision), while the other uses a non-interconnected two-dimensional camera.
The current research focus is on the development of a two-dimensional and three-dimensional CMOS sensor that can provide traditional two-dimensional images and three-dimensional point clouds at the same time. Teledyne-e2v is committed to providing cutting-edge leading technology for these new market areas, and has formulated future product roadmaps and intellectual property rights that can match the next generation of cameras and imaging systems, so that related technologies can focus on application-specific requirements. I believe it will not take a long time. Foresee the advent of new products.
The Links: F-51430NFU-FW-AA https://www.slw-ele.com/lq150x1lgn7.html“> LQ150X1LGN7 IGBTMODULE