[img]https://1.bp..com/-ewDneWuoSVU/Xqe74s4YkZI/AAAAAAAAAdk/P8ejfLNWiR0PkIpVeTaJyIRw1RPIDt5BwCLcBGAsYHQ/s1600/07aaef37-d4be-4eaf-bedc-21986223a8a0.jpeg.jpg[/img]

When an anticipated flagship releases, as with most of the specifications lots of people will ask about the camera. Will it be the best yet? will Samsung beat apple this year? Will Huawei surprise us this time around? Etc. Now, these people rely on companies that test these smartphones and deduct which is better or possibly rank them according to their own specific testing categories. A popular name in smartphone camera testing is DxOmark. When they test and subsequently release the scores, most headlines on the web, twitter and YouTube will be busy with the highest scorer. But should you trust these people when searching for a good phone camera? I mean, money is on the line right?

Founded in 2008 and based in Boulogne-Billancourt, France, It's a company that aims to provide image quality evaluation and assessment for camera lenses, Digital cameras and smartphones. They also test Audio capability, but let's leave that for now. To fully start this article, let's first know how DxO makes their score.

1.0: How DxO makes their ratings:

The numbers that you know DxO (120,108,98) for aren't a one time score, its actually a bit of a collection/combination (not average) of the various sub-categories they use in testing. These categories also come with their own numbers, then an algorithm sums up the numbers into a proper figure. This final number is the one that gets posted everywhere on the headlines. Now, lets check out the categories. These are:

Exposure and contrast
Colour
Autofocus
Texture and Noise
Flash and Zoom
Bokeh
Stabilization
Wide
Night

1.1: Exposure and contrast: Refers to the ability of the Camera to balance the brightness of the object and background.

1.2: Colour: Means how accurately the camera can reproduce the object colour.

1.3: Autofocus: How quickly the Camera can detect and focus on the object under different lighting conditions.

1.4: Texture and Noise: Deals with how good the Camera can detect even the smallest details on the subject, even during low lighting. To do this a great deal of noise reduction techniques must be employed.

1.5: Flash and zoom: Flash dictates how good the in built flash illuminates the subject. While zoom focuses on the Upscale capabilities of the camera, or if it has an optical zoom(telephoto lens, separate lens) capability or digital zoom.

1.6: Bokeh: Basically portrait shots. Determines the degree of the depth effect.

1.7: Stabilization: For video only. How good the camera can eliminate any motor effects during filming.

1.8: Wide (September 2019 update): Refers to the ability of the camera to capture a higher and wider degree of view which increases the perspective. Some smartphones come with dedicated wide angle lens for this feature.

1.9: Night (September 2019 update): The Night mode of the camera app.

[img]https://1.bp..com/-QN9RgUs8a4Q/Xqe6Z9AsgiI/AAAAAAAAAdQ/XFuIe4aAs7kLCVLuirqQhSd3Vk8F1V41QCLcBGAsYHQ/s1600/IMG_20200428_060731.jpg[/img]
The Galaxy s20 Ultra's score. The final score is not an average of the various sub scores.

Of course the higher the rating, the better the quality. They rate the photo quality, Video quality and the selfies.

2.0: Are their tests legit?

To answer this question squarely, it would be yes. Most reviewers just take some photos and call it a day, but DxO takes up to 1500 photos and clips of videos up to two hours long. All this in specially and carefully crafted environments meant to tackle the HDR (High dynamic range), Saturation, Hue, Distortion, Ghosting effect and contrast of the camera. To be honest the testing is actually scientific.

The tools they use are deemed industry grade, and they even have other software and services such as DxO photolab and DxO viewpoint.

3.0: Should you trust DxO for your Unbiased, Camera reviews?:

Like we said, DxO is actually scientific in testing, and the popularity speaks for itself. But how sure are we that the tests they conduct are true to life and realistic? I mean, isn't it a bit vague to judge the cameras of smartphones by some numbers that come from sub-categories?

For example, the score the Realme X2 pro at the time of writing is 95. And then the Google pixel is 90. This means the Realme x2 pro is better than the pixel……well, according to DxO that is. The anomaly here is that the Realme might have had some features like Bokeh, that tipped the scales in it's favour. Hence the reason why the scores are sometimes off. Just like DxO believes the Redmi note 8 pro is not as good as the older mi note 3. or Nokia 9 pure view is on par with Samsung a50.


[img]https://1.bp..com/-edZbfo64TWg/Xqe6gp6Wm3I/AAAAAAAAAdU/eRLudkhN_dIUjMHKu0TDW_2gQEYuYYcOACLcBGAsYHQ/s1600/PicsArt_04-28-06.00.44.jpg[/img]
The Xiaomi mi 10 pro (Top) vs the Huawei P40 pro (Bottom)


4.0: DxO is a testing company, but also a consulting company:

Yes. you read right, in as much as DxOmark is a smartphone camera analysis company, they actually work with Smartphone companies to make their cameras better. OEMs can pay to use their equipment when working before release to make their cameras better. Like a student that studies with the exam co-Ordinator before the examination. Which might mean that smartphones that did better on their ranking paid more for their equipment. And smartphones that didn't pay? Might do worse, or even might not get tested at all.


DxO actually admits that lots of manufacturers are using their services, but not Sony and apple. Sony and apple have been in the business for long and might not need their equipment to make their cameras better.


4.1: What if benchmark scorers like Antutu and Geekbench worked with OEMs before release?:

[img]https://1.bp..com/-3EEleIZ8h34/Xqe81CMrb0I/AAAAAAAAAds/ibDEMoFl5K4-q4Al_ybQDgbvxSRRa3U3ACLcBGAsYHQ/s1600/Antutu-logo.jpg[/img]

One thing about DxOmark is that they tend to loud their chart topers at any given opportunity. At the time of writing this the Huawei P40 pro is supposedly the best, and the Samsung S20 ultra arrived at sixth place. But what would happen if the popular benchmark software loud the smartphones that scored higher on their tests? What if benchmark scorers ''helped'' OEMs to get better scores?


4.2: Most smartphone companies prioritise better DxO scores, more than better cameras:

The truth is that Brands are revolving the development of their cameras around these tests. Which could come at the expense of customer satisfaction. Which might make DxO partly responsible for the bad cameras we have today. No be me talk am oh Each industry has its own one-sidedness but the fact that most OEMs are playing to the gallery is a bit worrisome. But DxO assures us that that they're independent and won't risk their image.

[img]https://1.bp..com/-d0YSXbP20dw/Xqe9lZyDpuI/AAAAAAAAAd0/OovJ2PjbvDgnol1wEFpBDSPWfk8V5vQswCLcBGAsYHQ/s1600/huawei-p40-vs-p40-pro-vs-p40-pro-plus.jpg[/img]
The Huawei p40 pro is currently DxO's Camera champion

4.3: Bottom-line:

DxO really has some solid testing, In fact not much camera testing companies come close to them. they're the major in the field. And most times the scores line up with the expectations of reviewers. But finally, you can pay attention to the scores they give and also analyse them. But don't use them like say, your holy grail of pure camera tests. When buying a smartphone and the camera is your top priority, make sure you check other places too, and not just DxOmark alone.

Thanks for reading. What do you think of DxO? Shady? Legit? Do you think their testing lives up to the hype? let us know in the comment section.

Source: https://techtemplar..com/2020/04/should-you-trust-dxo-mark-for-your.html

Cc: Mynd44
Dominique
Lalasticlala
Honsule
Dragnet

They are the best at what they do.

dboyoji:
They are the best at what they do.

Exactly..no one else comes close. That's why we should be wary. Smartphone companies might try to influence their testing to favour their releases.

CaseTemplar:
Exactly..no one else comes close. That's why we should be wary. Smartphone companies might try to influence their testing to favour their releases.

This has never happened. DXO is a very trusted company and their testing methods are legit.
If a new flagship is beaten by a previous one in camera scores it's usually because of software.

Also, same camera hardware used by different OEMs can have different scores. For example, the Mi 10 Pro and S20 Ultra have the same camera hardware but they have different algorithms that control the camera.

CaseTemplar:
[img], your holy grail

Until another suite of camera anlytics come out, they're the Holy grail of testing.

DxO is my plug

CaseTemplar:
Exactly..no one else comes close. That's why we should be wary. Smartphone companies might try to influence their testing to favour their releases.

Nothing like that for now, although there is accusation going on by apple fans that samsung influnced them not to do review on samsung galaxy S20 series because they are having issues with the camera softwares and samsung tries to delay their reviews until when they will fix the camera bug through update but DXOmark has refuted the accusation and they claimed that it was covid 19 movement restrictions that cause the dealy.

There is another allegation that phone makers over engineers camera and software of the phones they provided for them for the reviews, this gives the review phones better performance than the end user.
They also rejected this allegation by claiming the buy the phones they use for review in open market, not ordered from the producer's shop.

So far so good, no one has any evidence to backup those allegation and it is of the opinion that it is Apple fans who could not stand rating of other phones above iphones that makes those allegationa since majority of the complains are on comments sections of phones that have higher ratings than iphone.

festwiz:

This has never happened. DXO is a very trusted company and their testing methods are legit.
If a new flagship is beaten by a previous one in camera scores it's usually because of software.

Also, same camera hardware used by different OEMs can have different scores. For example, the Mi 10 Pro and S20 Ultra have the same camera hardware but they have different algorithms that control the camera.

Thanks for the comment. DxO'S testing methods are solid and legit yes, but the fact I'm trying to put across is that Dxo is also a consulting company. Which means they Actually help OEMs to score better on their tests...which is worrisome at least. Imagine if antutu and geekbench helped mobile companies get higher scores?

festwiz:


Until another suite of camera anlytics come out, they're the Holy grail of testing.

They're the best in their field yes, but what do you say of the redmi note 8 pro being worse than the older mi note 3? Or realme X2 pro better than Google pixel?

CaseTemplar:
They're the best in their field yes, but what do you say of the redmi note 8 pro being worse than the older mi note 3? Or realme X2 pro better than Google pixel?

Do you really believe the Google pixel from 2015 is better than the Realme X2 Pro? Mi Note 3 and Redmi Note 8 have different processors and such different image signal processing. A phone's SoC is just as important as the camera hardware and the algorithm that produces the image.

It's not a given that a new phone is better than the previous generation.

CaseTemplar:
Thanks for the comment. DxO'S testing methods are solid and legit yes, but the fact I'm trying to put across is that Dxo is also a consulting company. Which means they Actually help OEMs to score better on their tests...which is worrisome at least. Imagine if antutu and geekbench helped mobile companies helped higher scores?

The thing about the DXO is that though complicated their testing methods, they actually reflect real world performance.

CaseTemplar:
They're the best in their field yes, but what do you say of the redmi note 8 pro being worse than the older mi note 3? Or realme X2 pro better than Google pixel?

3 years of improved camera hardware and SoC. Bearing the Pixel branding doesn't automatically make it the best camera performer. You are being brand baised.

seunfly:


Nothing like that for now, although there is accusation going on by apple fans that samsung influnced them not to do review on samsung galaxy S20 series because they are having issues with the camera softwares and samsung tries to delay their reviews until when they will fix the camera bug through update but DXOmark has refuted the accusation and they claimed that it was covid 19 movement restrictions that cause the dealy.

There is another allegation that phone makers over engineers camera and software of the phones they provided for them for the reviews, this gives the review phones better performance than the end user.
They also rejected this allegation by claiming the buy the phones they use for review in open market, not ordered from the producer's shop.

So far so good, no one has any evidence to backup those allegation and it is of the opinion that it is Apple fans who could not stand rating of other phones above iphones that makes those allegationa since majority of the complains are on comments sections of phones that have higher ratings than iphone.

The first point is funny DxO can't obtain Samsung galaxy s20 ultra of February due to Coronavirus but they can get OnePlus 8 pro in April.

the second point is true.

There's no evidence because there isn't any need for one. DxO gives(for a price) their software to OEMs to use and develop their cameras. Any company that subscribes to DxO might do better in its tests. They even admitted that most companies patronise them except Sony and apple. Why would apple need their software and services to develop their cameras? And if you don't use their services? Then you'll get worse scores or you won't be tested

festwiz:

3 years of improved camera hardware and SoC. Bearing the Pixel branding doesn't automatically make it the best camera performer. You are being brand baised.

Truely the guy is brand biase, i remember DaxO review conclusion on realme x2 pro, DXO said the picture quality is not up to par with todays premium camera but if this was 3 years old phone, it would be considered top.

It is very insulting to the company for the guy to even contest the ranking of 2019 value for price phone with 2016 pixel, i cant believe someone can say such thing when we are not talking about tecno or dogee.
Other factors are into play such as sensors, chips, software etc i can't even believe 2016 pixel has better software not to talk of the sensor and the processor.
The guy just asumed once it is pixel, the camera automatically beats other phones. It is no so.

festwiz:

3 years of improved camera hardware and SoC. Bearing the Pixel branding doesn't automatically make it the best camera performer. You are being brand baised.

Realme x2 pro's score was 95. Google pixels score was 90. The pixel actually beat it in exposure and colour, Autofocus and noise, but where the realme won was due to it's Ultrawide camera. The space for the pixel is empty. That automatically means that a good camera can lose out to another camera if it lacks one of these cameras, or win due to an additional camera( like a depth sensor) because DxO'S "great" algorithm thinks so. Remember the scores are combinations of individual scores and not averages. How then does DxO's algorithm give out the numbers we all see on the papers?

First one is realme second is pixel

seunfly:


Truely the guy is brand biase, i remember their review on realme x2 pro, DXO said the picture quality is not up to par with todays premium camera but if this was 3 years old phone, it would ne considered top.

The guy just asumed once it is pixel, the camera automatically beats other phones. It is no so.

I'm not brand braised my good sir. I'm just pointing out flaws in DxO'S testing system. There's little anomalies when you go through their charts. Nokia 9 pure review is not better than Samsung A50. Even realme X2 pro(95) isn't better than iPhone X(97),OnePlus 6(96), Xiaomi mi 8(99). phones that were released with the pixel or closely released then. Would you say you'd pick OnePlus 6's camera over realme X2 pro?

CaseTemplar:
Realme x2 pro's score was 95. Google pixels score was 90. The pixel actually beat it in exposure and colour, Autofocus and noise, but where the realme won was due to it's Ultrawide camera. The space for the pixel is empty. That automatically means that a good camera can lose out to another camera if it lacks one of these cameras, or win due to an additional camera( like a depth sensor) because DxO'S "great" algorithm thinks so. Remember the scores are combinations of individual scores and not averages. How then does DxO's algorithm give out the numbers we all see on the papers?

First one is realme second is pixel

The problem with people is that they have perception which many times may not be correct, that was the reasons why iphone fans felt DXO favours some phone makers but DXO kept showing them those figures and challege any of them to come up with alternative.i remember DXO had to retake pictures of xaiomi note 10 pro just to prove a point in comparism.

CaseTemplar:
.

I don't get your point. The Realme came out on top. Is it a bitter pill to swallow?

seunfly:


The problem with people is that they have perception which many times may not be correct, that was the reasons why iphone fans felt DXO favours some phone makers but DXO kept showing them those figures and challege any of them to come up with alternative.i remember DXO had to retake pictures of xaiomi note 10 pro just to prove a point in comparism.

Exactly, all these backwards forwards allegations are pointless. These smartphones are scored based on real world outputs in real world scenarios. There's is no trickery here like in benchmarks where Mediatek was just caught cheating, again.

I still don't get the OPs points. For a while now, Huawei has topped camera performance and it's not just DXo scores, they have probably the best Night Mode algorithm right now. And the moment they na video capture like the iPhone 11 Pro, they'll be ahead again. There is no brand bias in things like these.

CaseTemplar:
The first point is funny DxO can't obtain Samsung galaxy s20 ultra of February due to Coronavirus but they can get OnePlus 8 pro in April.

the second point is true.

There's no evidence because there isn't any need for one. DxO gives(for a price) their software to OEMs to use and develop their cameras. Any company that subscribes to DxO might do better in its tests. They even admitted that most companies patronise them except Sony and apple. Why would apple need their software and services to develop their cameras? And if you don't use their services? Then you'll get worse scores or you won't be tested

On your first statement, they have samsung galaxy 20ultra, the only problems is that they could not review due movement restrictions that handicaps their staff, by now i think they would have done the reviews.

On your second statement, i dont think it is true, apple or sony will not get worst score because they dont patronize them, those guys are professionals. I dont even know about their software market however most people except Apple fans agreed with their score, infact most reviewers and other software firms agreed with the acore because it is what layman can even interpret.
However everybody knows Apple fans, they are in their own world and it is better to leave them there.

Imagine Apple fans claimed new iphone SE is the best thing to have ever happened to $700 smartphone range and all other smartphone within that range are trash, the new engine in old body çar is far better and more advanced than the other latest cars with the latest engine and technology.
when Apple enters your brain, u wont be able to think properly again.