[Smart Leadership Lecture] ADAS Essentials: Why 140dB HDR Matters So Much

[Smart Leadership Lecture] ADAS Essentials: Why 140dB HDR is So Crucial

Introduction: This article provides a concise overview of HDR image processing techniques, focusing on how HDR applications enable us to capture images that are more realistic, vibrant, and detailed, while also enhancing overall image quality and visual appeal. Through an analysis of these technologies, we gain a comprehensive understanding of HDR.

What is HDR?

HDR stands for High Dynamic Range, a technical term that delivers greater dynamic range and image detail. Typically, when describing a scene, dynamic range refers to the ratio between its brightest and darkest areas. A High Dynamic Range Scene, therefore, is one where both extremely bright and very dark elements coexist simultaneously.

The Origin of HDR

In fact, HDR first gained worldwide recognition as a still photography technique. French photographer Gustave Le Gray was the first to use it in the 1850s; he understood the limitations of film negatives in terms of dynamic range and creatively employed two separate exposures to produce images that overcame these constraints.

In the wave of the digital age, with the advancement and breakthroughs in digital photography, these photographic technology concepts have become feasible thanks to camera imaging sensors and software. Moreover, the development of HDR technology spans across numerous hardware and software-related fields, including photography, image processing, and display technologies.

HDR's Scene Applications

High-dynamic-range images are limited by the image sensor’s full well capacity (FWC) and inherent background noise. If the full well capacity is small, it cannot capture much of the bright information; conversely, high background noise prevents the recording of detailed dark-area information. As a result, the dynamic range that an image sensor can capture is inherently limited.

So, by using alternative methods to capture scenes with such a high dynamic range—this is known as High Dynamic Range Imaging (HDR Imaging).

However, HDR images inevitably require dynamic range compression when rendered on current display technologies, as they are processed within the capabilities of HDR hardware. As the HDR ecosystem continues to advance—supported by both cutting-edge hardware and rich software content—people are already experiencing richer colors and higher contrast in the high-end display sector.

How can HDR display be achieved?

Typically, the mobile phone photos we take capture 10-bit RAW data, which fails to fully represent the dynamic range of the real-world scene. By employing multi-frame, multi-exposure high-dynamic-range imaging technology—capturing images with overexposure, underexposure, and normal exposure in a single shot—we generate three or more RAW files with varying exposure levels. After ISP image processing and HDR algorithm fusion, the bit depth is significantly enhanced, allowing us to reveal richer details in both bright and dark areas, ultimately delivering a visual experience that closely mimics what the human eye perceives.

In the automotive domain, HDR sensors utilize three or four Linear RAW frames captured with nearly simultaneous exposures—typically including long, medium, short, and ultra-short exposures—to merge into a high-depth HDR RAW image (usually 20-bit or 24-bit). This process enables the sensor hardware to output an HDR RAW image, effectively enhancing the dynamic range. The resulting HDR RAW data is then subjected to piecewise linear compression before being sent to the ISP chip for image processing, where image reconstruction and color restoration take place, ultimately delivering images that faithfully reproduce the clarity and vibrant colors of the real-world scene. Therefore, when configuring exposure settings, it’s crucial to adhere to specific guidelines to prevent potential anomalies from occurring.

For example, in the image below, when moving, there are prominent wide-edge artifacts with noticeable motion blur, and these artifacts even exhibit unusual colors, severely hindering object recognition in machine vision applications—and they also create a very poor visual experience for human observers.

To determine where the main issues lie, it’s necessary to analyze the performance of both HDR RAW and Linear RAW images, then proceed with targeted optimization.

By analyzing the issue, it was determined that the Linear frame exposure ratio was improperly configured. When the hardware synthesizes HDR images, discrepancies in the vector directions at the stitching edges lead to artifacts, preventing the seamless integration of a perfect, realistic image. Therefore, it is necessary to calculate and set the exposure time based on the exposure ratio recommended by the chip manufacturer, thereby optimizing this particular problem.

By adjusting the exposure time with an appropriate exposure ratio, the inter-frame offset time can be reduced, allowing the resulting HDR image to achieve an effect that aligns with human visual perception.

As for the abnormal colors in the artifact areas of the image, they represent the original colors from a specific frame taken before the HDR image was synthesized. Therefore, Linear frame correction is required, along with pre-processing WB gain adjustments to further enhance optimization. This ensures that, after the image is merged into an HDR format, edges will appear in neutral tones.

Similar issue points

For example, in the image below, the left picture shows a sky with the sun, where the sun appears abnormally pink. This occurs because, when setting the exposure time to achieve a wider dynamic range, the ultra-short frame exposure was set too briefly, resulting in an imbalanced exposure ratio and causing abnormal effects in the brightly lit areas of high-dynamic scenes.

After optimization and fine-tuning the appropriate exposure time, combined with other methods to address abnormal color artifacts, the issue has been improved. However, the high-brightness areas (such as the sun) are now slightly overexposed compared to before, specifically manifesting as a larger outline of sunlight and an increased area of luminous halos along the edges.

Summary

The human eye typically has a dynamic range of around 120 dB. As the "eyes" of ADAS and autonomous driving systems, forward-facing cameras must quickly identify subtle light and dark details under varying illumination conditions while the vehicle is moving at high speeds, ensuring accurate image capture—otherwise, driving safety could be compromised. A typical HDR requirement arises when a car exits a tunnel: the camera needs to simultaneously detect bright areas while preserving details in the darker shadows, or, in nighttime scenes, accurately recognize extremely dim pedestrians alongside intensely bright headlights, LED signal lights, and other luminous elements. Ideally, a camera’s dynamic range should surpass that of the human eye, reaching an impressive 140 dB for HDR performance.

Therefore, for ADAS applications, covering 120dB HDR is a basic requirement, while achieving 140dB HDR is the emerging trend in automotive machine vision.

Related news