Camera 
Display 
Battery 
Smartphone 
Security 
DSLR & Mirrorless 
Tv Screens 
Automotive 
Medical 
3D Camera 
Industrial Vision 
Computing 
AR / VR 
Drone & Action camera 
Purpose of the study
Image quality is at the core of our expertise and because portraits (friends, family, kids and pets) represent most pictures taken with smartphones, it was the focus of DXOMARK’s first Insights to address the a major challenges and pain points that manufacturers need to consider to to satisfy consumers. Our research was undertaken to analyze users’ perceptions concerning skin-tone representation in photographs of themselves and others. Through this study, we saw trends in user preferences, and discovered the sources of satisfaction and dissatisfaction when it comes to portrait photography.
DXOMARK employed a distinctive methodology to gather comprehensive insights across common use cases, environments and conditions, that involved capturing a total of 1,620 images over more than 400 scenes.
A study with a diverse and inclusive panel of consumers and professional users
This exhaustive study comprised a diverse European panel with various skin tones, cultural backgrounds, ages and genders, making it the largest of its kind. It included:
-
- 83 regular consumers
- 30 professional photographers
- 10 DXOMARK image quality experts
- 52% women / 48% men
Various types of skin tones (defined and qualified as per the Fitzpatrick and Monk scales)
On a various set of portrait pictures, in various lighting environments
To assess user preferences on portraits, we included a large set of situations and lighting conditions. All pictures were taken by professional photographers to get perfectly comparable shots.
In the study, we used various types of smartphones, the flagship devices available at the time of the study as well as a professional camera (representing what the future of smartphone photography may hold). For each type of portrait, four different smartphones and a professional camera were used.
180 different scenes were taken with all devices, varying the number of models, location and lighting conditions.
A robust methodology in assessing consumer preferences
Based on our 20 years of experience in image quality testing, we have established a scientific and robust methodology to assess user preferences.
Combining various aspects: preference and rejection identified from a two-step survey:
Pairwise comparison: selecting the best picture from two presented
Relative rejection: from a set of pictures, identifying the ones that were rejected
Identifying at every step the reasons for the preference or rejection
Using the JOD (Just Objectional Differences) scale to rank pictures by crossing the results for several comparisons
One key metric to quantify user satisfaction: the Satisfaction Index, which combines
Pairwise comparison
Relative rejection
JOD scale
Ranging from 0 to 100 and qualifying the level of satisfaction for each picture
Major takeaways
Here are a few points about what the study revealed:
- Unfulfilled Expectations: Despite advancements in smartphone technology, today’s leading smartphones fall short of meeting user expectations for portrait rendering in pictures. This indicates a gap between technological capabilities and user satisfaction.
- Significant Disparities Among Smartphones: The study identified substantial differences in portrait rendering among various smartphones, leading to varying levels of user satisfaction. This highlights the need for manufacturers to address these disparities and strive for consistent high-quality performance.
- Inconsistency Across Skin Tones: An intriguing discovery was that the perceived quality of images did not remain consistent across all skin tones featured in the study. This prompts a crucial question about the inclusivity of smartphone camera technology and the need for improvements to ensure consistent performance across diverse skin tones.
- Room for Improvement in All Lighting Conditions: While smartphones have made strides in photo rendering, there is still ample room for improvement, particularly in achieving satisfying photo rendering in every lighting condition. This indicates a potential area for innovation and enhancement in future smartphone camera technologies.These findings present opportunities for collaboration and innovation within the industry landscape, with a focus on delivering enhanced smartphone photography experiences for users.
What to expect from this report?
DXOMARK Insights provides manufacturers with actionable data and tuning guidelines. From the report, you can get:
- A full benchmark on how the products compared with one another
- An identification of the user preferences, general or based on specific criteria
All this will help you leverage technical guidelines for your future products.