DXOMARK scientists present the results of their research, including the development of innovative algorithms, at science conferences worldwide. Explore our extensive range of scientific publications as we work to shape the future of technology benchmarking. 

Objective color characterization of HDR videos captured by smartphones: Laboratory setups an analysis framework

A. Tigranyan, P. Mathieu, C. Nannini, F. Thomas, M. Patti, F. Guichard

This article provides elements to answer the question: How to judge general stylistic color rendering choices made by imaging devices capable of recording HDR formats in an objective manner? The goal of our work is to build a framework to analyze color rendering behaviors in targeted regions of any scene, supporting both HDR and SDR content.

Machine learning: Predicting audio quality for high SPL smartphone recordings (AES New-York 2023)

P. Guelen, D. Zhao, P. Terra Pizutti Dos Santos, A. Drouadene and J. Bacle

In this paper, we explore a machine learning approach to evaluate audio quality for high sound pressure level (SPL) smartphone recordings. Our study is based on perceptual evaluations conducted by technical experts on eight audio sub-attributes (tonal balance, treble, midrange, bass, dynamics, temporal artifacts, spectral artifacts, and other artifacts) of audio quality for 121 smartphones released from 2019 to 2021. 

An image quality assessment dataset for portraits (CVPR 2023)

N. Chahine, A. Calarasanu, D. Civiero, T. Cayla, S. Ferradans, J. Ponce

Year after year, the demand for ever-better smartphone photos continues to grow, in particular in the domain of portrait photography. Manufacturers thus use perceptual quality criteria throughout the development of smartphone cameras. This costly procedure can be partially replaced by automated learning-based methods for image quality assessment (IQA). 

Laboratory evaluation of smartphone audio zoom systems (AES Europe 2023)

D. Zhao, A. Drouadene, H. Maisonneuve, P. Guelen, J. Bacle

In this paper, we propose a rating protocol for evaluating smartphone audio zoom systems through objective and perceptual testing. Audio zoom is a newly developed function that helps isolate a sound source from its surroundings in accordance with the smartphone camera’s focal point and zoom level when recording videos with the camera app. 

Real-content based method for HDR tone curve characterization (SID 2023)

P. Bernardi, T. Cabana, F. Guichard

The classic window pattern with a black background is the basis of any display characterization, but nowadays, displays, particularly smartphone displays, integrate complex image processing and adaptations, meaning that they cannot be entirely characterized with these simple patterns. 

Improvement of the flare evaluation for cameras and imaging applications when using near-infrared lighting

E. Souksava, E. Baudin, C. Greco, H. Nguyen, L. Chanas, F. Guichard

The number of cameras designed for capturing the nearinfrared (NIR) spectrum (sometimes in addition to the visible) is increasing in automotive, mobile, and surveillance applications. Therefore, NIR LED light sources have become increasingly present in our daily lives. Nevertheless, camera evaluation metrics are still mainly focused on sensors in the visible spectrum. 

Noise quality estimation on portraits in realistic controlled scenarios

N. Chahine, S. Lahouar, S. Santos, S. Calarasanu, S. Ferradans, B. Pochon, F. Guichard

The wide use of cameras by the public has raised the interest of image quality evaluation and ranking. Current cameras embed complex processing pipelines that adapt strongly to the scene content by implementing, for instance, advanced noise reduction or local adjustment on faces. 

Evaluation of image quality metrics designed for DRI tasks with automotive cameras

V. Klein, T. Eleftheriou, Y. Li, E. Baudin, C. Greco, L. Chanas, F. Guichard

Nowadays, cameras are widely used to detect potential obstacles for driving assistance. The safety challenges have pushed the automotive industry to develop a set of image quality metrics to measure the intrinsic camera performances and degradations. However, more metrics are needed to correctly estimate computer vision algorithms performance, which depends on environmental conditions. 

An Image Quality Assessment Dataset for Portraits

N. Chahine, A. Calarasanu, D. Garcia-Civiero, T. Cayla, S. Ferradans, J. Ponce

Year after year, the demand for ever-better smartphone photos continues to grow, in particular in the domain of portrait photography. Manufacturers thus use perceptual quality criteria throughout the development of smartphone cameras. This costly procedure can be partially replaced by automated learning-based methods for image quality assessment (IQA).

Objective image quality evaluation of HDR videos captured by smartphones

C. Lajarge, F. Thomas, E.Souksava, L. Chanas, H. Nguyen, F. Guichard

High Dynamic Range (HDR) videos attract industry and consumer markets thanks to their ability to reproduce wider color gamuts, higher luminance ranges and contrast. While the cinema and broadcast industries traditionally go through a manual mastering step on calibrated color grading hardware, consumer cameras capable of HDR video capture without user intervention are now available. 

Image quality evaluation of video conferencing solutions with realistic laboratory scenes

R. Falcon, M. Patti, S. Brochard-Garnier, G. Pacianotto Gouveia, S. Torres Acevedo, T. Bergot, R. Alarcon, C. Bomstein, H. Macudzinski, P. Maitre, L. Chanas, H. Nguyen, B. Pochon, F. Guichard

Videoconferencing has become extremely relevant in the world in the past few  years. Traditional image and video quality evaluation techniques prove to be insufficient to properly assess the quality of these systems, since they often include special processing pipelines, for example, to improve face rendering. 

New visual noise measurement on a versatile laboratory setup in HDR conditions for smartphone camera testing

T. Bourbon , C. Hillairet , B. Pochon , F. Guichard

Cameras, especially camera phones, are using a large diversity of technologies, such as multi-frame stacking and local tone mapping to capture and render scenes with high dynamic range. ISO-defined charts for OECF estimation and visual noise measurement are not really designed for these specific use cases, especially when no manual control of the camera is available.

Automatic noise analysis on still life chart

S. Belkarfa, A. Choukarah, M. Tworsk

We tackle the issue of estimating the noise level of a camera, on its processed still images and as perceived by the user. Commonly, the characterization of the noise level of a camera is done using objective metrics determined on charts containing uniform patches at a given condition. 

Portrait quality assessment using multi-scale CNN

C. Nicolas, B. Salim

We propose a novel and standardized approach to the problem of camera-quality assessment on portrait scenes. Our goal is to evaluate the capacity of smartphone front cameras to preserve texture details on faces. We introduce a new portrait setup and an automated texture measurement. The setup includes two custom-built lifelike mannequin heads, shot in a controlled lab environment. 

Evaluation of the lens flare

E. Souksava, T. Corbier, Y. Li, F. Thomas, L. Chanas, F. Guichard

We present an objective metric for quantifying the amount of flare of the lens of a camera module. This includes hardware and software tools to measure the spread of the stray light in the image. A novel measurement setup has been developed to generate flare images in a reproducible way via a bright light source, close in apparent size and color temperature to the sun, both within and outside the field of view of the device. 

RAW image quality evaluation using information capacity

F.-X. Thomas, T. Corbier, Y. Li, E. Baudin, L. Chanas, and F. Guichard

We propose a comprehensive objective metric for estimating digital camera system performance. Using the DXOMARK RAW protocol, image quality degradation indicators are objectively quantified, and the information capacity is computed. The model proposed in this article is a significant improvement over previous digital camera systems evaluation protocols, wherein only noise, spectral response, sharpness, and pixel count were considered.

Quantitative measurement of contrast, texture, color, and noise for digital photography of HDR scenes

G. Facciolo, G. Pacianotto, M. Renaudin, C. Viard, F. Guichard

We describe image quality measurements for HDR scenes covering local contrast preservation, texture preservation, color consistency, and noise stability. By monitoring these four attributes in both the bright and dark parts of the image, over different dynamic ranges, we benchmarked four leading smartphone cameras using different technologies and contrasted the results with subjective evaluations.

Image quality benchmark of computational bokeh

W. Hauser, B. Neveu, J.-B. Jourdain, C. Viard, F. Guichard

We propose a method to quantitatively evaluate the quality of computational bokeh in a reproducible way, focusing on both the quality of the bokeh (depth of field, shape), as well as on artifacts brought by the challenge to accurately differentiate the face of a subject from the background, especially on complex transitions such as curly hair.

Towards a quantitative evaluation of multi-imaging systems

M. Renaudin, A.-C.  Vlachomitrou, G. Facciolo, W. Hauser, C. Sommelet, C. Viard, F. Guichard

This paper presents laboratory setups designed to exhibit the characteristics and artifacts that are peculiar to Multi-Image technologies. We also propose metrics towards the objective and quantitative evaluation of those artifacts.

Autofocus measurement for imaging devices

P. Robisson, J.-B. Jourdain, W. Hauser, C. Viard, F. Guichard

We propose an objective measurement protocol to evaluate the autofocus performance of a digital still camera. As most pictures today are taken with smartphones, we have designed the first implementation of this protocol for devices with touchscreen trigger.

Device and algorithms for camera timing evaluation 

L. Masson, F. Cao, C. Viard, F. Guichard

This paper presents a novel device and algorithms for measuring the different timings of digital cameras shooting both still images and videos. These timings include exposure (or shutter) time, electronic rolling shutter (ERS), frame rate, vertical blanking, time lags, missing frames, and duplicated frames.

Electronic trigger for capacitive touchscreen and extension of ISO 15781 standard time lag measurements to smartphones

F.-B. Bucher, F. Cao, C. Viard, F. Guichard

We present in this paper a novel capacitive device that stimulates the touchscreen interface of a smartphone (or of any imaging device equipped with a capacitive touchscreen) and synchronizes triggering with our LED Universal Timer to measure shooting time lag and shutter lag according to ISO 15781:2013.

Measurement and protocol for evaluating video and still stabilization systems

E. Cormier, F. Cao, F. Guichard, C. Viard

This article presents a system and a protocol to characterize image stabilization systems both for still images and videos.

Development of the I3A CPIQ spatial metrics

D. Baxter, F. Cao, H. Eliasson, J. Phillips

The I3A Camera Phone Image Quality (CPIQ) initiative aims to provide a consumer-oriented overall image quality metric for mobile phone cameras. In order to achieve this goal, a set of subjectively correlated image quality metrics has been developed. This paper describes the development of a specific group within this set of metrics, the spatial metrics. Contained in this group are the edge acutance, visual noise and texture acutance metrics. 

An objective protocol for comparing the noise performance of silver halide film and digital sensor

F. Cao, F. Guichard, H. Hornung, R. Teissières

Digital sensors have obviously invaded the photography mass market. However, some photographers with very high expectancy still use silver halide film. Are they only nostalgic reluctant to technology or is there more than meets the eye? The answer is not so easy if we remark that, at the end of the golden age, films were actually scanned before development. 

Performance of extended depth of field systems and theoretical diffraction limit

F. Guichard, F. Cao, I. Tarchouna, N. Bachelard

Extended depth of field (EDOF) cameras have recently emerged as a low-cost alternative to autofocus lenses. Different methods, either based on longitudinal chromatic aberrations or wavefront coding have been proposed and have reached the market. The purpose of this article is to study the theoretical performance and limitation of wavefront coding approaches. 

Information capacity: a measure of potential image quality of a digital camera

F. Cao, F. Guichard, H. Hornung

The aim of the paper is to define an objective measurement for evaluating the performance of a digital camera. The challenge is to mix different flaws involving geometry (as distortion or lateral chromatic aberrations), light (as luminance and color shading), or statistical phenomena (as noise). . 

Dead leaves model for measuring texture quality on a digital camera

F. Cao, F. Guichard, H. Hornung

We describe the procedure to evaluate the image quality of a camera in terms of texture preservation. We use a stochastic model coming from stochastic geometry, and known as the dead leaves model. It intrinsically reproduces occlusions phenomena, producing edges at any scale and any orientation with a possibly low level of contrast. 

Measuring texture sharpness of a digital camera

F. Cao, F. Guichard, Hervé Hornung

A method for evaluating texture quality as shot by a camera is presented. It is shown that usual sharpness measurements are not completely satisfying for this task. A new target based on random geometry is proposed. It uses the so-called dead leaves model. It contains objects of any size at any orientation and follows some common statistics with natural images. 

Sensor information capacity and spectral sensitivities

F. Cao, F. Guichard, H. Hornung, L. Masson

In this paper, we numerically quantify the information capacity of a sensor, by examining the different factors than can limit this capacity, namely sensor spectral response, noise, and sensor blur (due to fill factor, cross talk and diffraction, for given aperture). In particular, we compare the effectiveness of raw color space for different kinds of sensors. 

Characterization and measurement of color fringing

F. Cao, F. Guichard, H. Hornung and C. Sibade

This article explains the cause of the color fringing phenomenon that can be noticed in photographs, particularly on the edges of backlit objects. The nature of color fringing is optical, and particularly related to the difference of blur spots at different wavelengths. Therefore color fringing can be observed both in digital and silver halide photography. 

Does resolution really increase image quality?

C.-L. Tisse, F. Guichard, F. Cao

A general trend in the CMOS image sensor market is for increasing resolution (by having a larger number of pixels) while keeping a small form factor by shrinking photosite size. This article discusses the impact of this trend on some of the main attributes of image quality. The first example is image sharpness. A smaller pitch theoretically allows a larger limiting resolution which is derived from the Modulation Transfer Function (MTF). 

Sensor spectral sensitivities, noise measurements and color sensitivity

F. Cao, F. Guihard and H. Hornung

This article proposes new measurements for evaluating the image quality of a camera, particularly on the reproduction of colors. The concept of gamut is usually a topic of interest, but it is much more adapted to output devices than to capture devices (sensors). Moreover, it does not take other important characteristics of the camera into account, such as noise. 

Advances in Camera Phone Picture Quality

F. Guichard

A unique digital postprocessing technique compensates for performance problems posed by ever-shrinking pixels | by Dr. Guichard Frédéric, DXOMARK | From Photonics Spectra , November 2007. As camera phones become ubiquitous, consumer demand for a photographic experience similar to that of traditional digital cameras is growing. 

A measure of color sensibility for imaging devices

F. Guichard, J. Buzzi

We define color sensitivity or effective color depth based on the “number of reliably distinguished colors”, using ideas from information theory. This figure of merit allows the comparison of different sensors or cameras and we indicate how it can be used both for the design of imaging devices and to optimize their adaptation to the scene.

Noise in imaging chains : correlations and predictions

F. Guichard, J. Buzzi

Noise is an important factor in image quality. We analyze it in images produced by digital cameras. We show that, beyond the usual standard deviation measurement, spatial correlations also convey interesting information which allows to (i) better describe the perception of the noise, (ii) analyze an unknown imaging chain. Indeed, knowledge of these spatial correlations is necessary to predict the noise after the rescaling and sampling involved in a realistic imaging chains.

Uniqueness of blur measure

F. Guichard, J. Buzzi

After discussing usual approaches to measuring blur of optical chains, we show theoretically that there is essentially a unique way to quantify blur by a single number. It is the second derivative at the origin of the Fourier transform of the kernel. This somewhat surprisingly implies that blur is especially sensitive to attenuation of the low frequencies.