Inquisitive Universe:
Oya this evening

Let's discuss a hotly contested topic. Hotly contested for those who know about it sha.

Benchmarks.

Whether we like it or not, everybody get bias. Serious biases dey. Some people try to put aside their bias and try to be objective while others want you to accept their bias as fact.

The second group I mentioned is usually more forceful in discussion and do not accept others challenging their position.

In our social media sphere, everything that is debatable is up for debate and should be up for debate.

One area where people do not seem to agree is processors. Especially mobile processors.

It is rare to find people arguing about PC processors like AMD 7000 vs Core i9 for example.

This is because many people don't really know or care about PCs that much.

But when it comes to mobile processors. The Snapdragons vs Exynos vs Mediatek sphere.

Omo! It is a bloody battlefield where everyone seems to have an opinion.

By everyone, I mean everyone. Both the people that know and the ones who don't. Everybody wan talk.

And I have read some messed up stuff in the last few years that is enough to make my eyes fall out. Like what the f"ck did I just read.

I have had people argue that the number of cores are what makes one processor better than the other. For these people, nothing else matters.

The extent of their knowledge na Octa Core and Quad core. This is what they actually check when they say that they're reading SoC specs.

Another group of people who really need to tied and flogged are those who use Clock speed to determine if one SoC is better than another one.

The only thing that they check is clock speed. Nothing more.

I had to ask one, that the Helio G90T is clocked at 2.05GHz and the Snapdragon 660 is clocked at 2.2GHz.

Which SoC is the better one? With all confidence, he replied SD660. Why? Because SD660 has a faster clockspeed.

Another set of people will tell you that it is by name that you judge an SoC. A few years back, the common opinion was that Exynos is better.

I follow my roommate for NYSC drag this thing for days. He was far older than me and used Samsung phones. To him, Exynos will always be the best.

The opinion that is held now is that Snapdragon SoCs are the best and that is what a lot of people live by.

Imagine if two people with this mindset jam.

War!!!

To these people, the idea that each SoC should be examined based on individual merit is not even remotely present in their thinking.

Snapdragon good. Everything else bad!

Exynos good. Everything else bad!

This last set of people no even join at all.

Na RAM size them take dey judge SoC ability. If you give them a Snapdragon 450 with 6GB RAM and a MediaTek Helio G96 with 4GB RAM...

They will happily pick the SD450 because of RAM

Since a lot of people don't know how to or do not want to read SoCs specs, especially the type of cores used e.g. Cortex A55, A72 etc.

There has to be a quick and easy way for people to be able to compare SoCs so as to see which ones are better than others.

A very easy and quick way (if you noticed I didn't say good) to do so is by using benchmarks.

What is a benchmark?

A benchmark is a standard or reference point against which other things are compared. It is mostly used as a tool of comparison.

For example...

If you apply for any paramilitary job like Immigration, FRSC, NSCDC etc.

They usually have a height benchmark. 170cm (5'4" for males and 165cm (5'1" for females.

If you as a guy no tall reach that 170cm, you are automatically disqualified. That height is supposed to reflect the average height of Nigerian males.

So if you pass 170cm, congratulations. If you no tall reach am, well, ���

You may be tall in other places ��‍♂️��‍♂️��‍♂️��‍♂️

In the smartphone area, benchmarks are used to compare everything from the performance of entire phones to its individual components such as cameras, SoCs (CPUs and GPUs), storage speed, battery life etc.

Why should we use benchmarks?

Simple

Benchmarks provide a fair and universal way of comparing the performance of different smartphones and their components. This is because there are many specs associated with smartphones and these are all built differently.

With benchmarks, you do not have to understand these details. You just have to look at the benchmark performance and see how good it is.

You can use it to compare the performance of the other SoC.

For example Mediatek Dimensity 900 has a score of 701 and Snapdragon 732G has a score of 560. (Geekbench single core test)

You can easily tell which one is better.

How are Benchmarks done?

A benchmark usually consists of a series of standardized tests designed to test the performance of the thing to be compared. The results of these tests are universal. They are designed to be uniform for all items being tested.

Let us use PUBG mobile as a benchmark.

To use PUBG as a benchmark, we will download PUBG into the phones to be tested and then score it based on the following criteria:

Ability to max out all settings (both graphics and fps) - 100 (A+)

Able to play the game at high settings with low fps - 60 to 69 (B)

Able to install game and play at medium settings - 50 to 59 (C)

Able to install game and play at low settings - 45 to 49 (D-)

Able to install but unable to run the game - 40 to 44 (E)

Unable to install game - 0 (F)

So basically, if we bring out my old Tecno Pouvoir 2, that can install PUBG but cannot run the game. We are going to score it E.

If I bring a Redmi 9A that can install the game and play it at low settings, we will give the 9A, a score of D-

So in other words, the Redmi 9A performance better than the Tecno Pouvoir 2.

This is how benchmarks work.

Benchmarking is usually associated with assessing the ability of a CPU and it’s ability to run software. It provides a fair way to compare the performance of two or more CPUs/SoCs.

The major generally accepted benchmarks for smartphones include:
Antutu
Geekbench
3D mark
PC mark
GFX

etc.

For a benchmark to be accepted as a standard for comparing phones. They have to fulfill certain criteria.

They have to be:

1. Relevant
2. Fair
3. Transparent
4. Verifiable or Repeatable
5. Cost effective
6. Scalable

By now, I am sure that we know what Benchmarks are and what they are used.

So why do people hate Benchmarks?

I mean something that is supposed to make life easy for all of us. Why would anyone hate it?

We will look at it from two angles.

1. From the side of phone users i.e. us

2. From the side of the benchmark companies/phone manufacturers.

Why do people hate Benchmarks?

1. They don't understand it and do not want to understand it.

2. They usually disagree with it

3. It puts their favourite brands down.

3. They distrust it

What are some of the problems with benchmarks?

1. Different interpretation: different benchmarks can sometimes put out different scores. Thus it is sometimes left to the user to interprete these scores by themselves or to rely on those who can. This situation makes it possible for people to misinterpret benchmarks.

Sometimes the score from Antutu may counter the one from Geekbench and so on and so forth.

2. Vendors can and will optimize SoCs to run better on benchmark software. These SoCs would then score high in benchmark tests but would perform poorly on other tasks.

3. Outright cheating: SoC companies have been known to add code to the firmware of their SoCs. If these codes detect a benchmark test, they are designed to overclock the SoC to reach maximum performance. This way, they can get unrealistically high benchmark scores which they would then use for advertising.

4. Overly streamlined tests that focus only on processing power and ignore other stuff like photography, videography, network speed, heating levels, power consumption, throttling etc.

5. User perception: Sometimes average day to day use does not really reflect benchmark scores. It could be possible for a phone that did well in benchmarks to flop in real life. This has led to some distrust for benchmark scores.

6. No account for CPU/OS load: Most phone are usually tested using the best possible conditions. They usually do not take into account what would happen if the storage of a phone if filled up to a certain point with several apps running in the background.

These are some of the plenty problems that one would run into when trying to use Benchmarks.

So should we stop using benchmarks then?

Well no. Absolutely not.

Despite their shortcomings, benchmarks are still the fastest and most objective way that we can compare SoCs.

Until another method is devised, they are the best that we can rely on for now so they are still very useful.

The organizations owning these benchmarks have to step up and keep their benchmarks free and fair as well as looking for ways to eliminate cheating.

So that's it on this topic from me.

As usual, contributions, additions, corrections, comments, recommendations etc are also very very much welcome.

Good evening you all. Stay safe.

https://inquisitiveuniverse.com/2021/11/24/what-is-a-benchmark/

Here for the comments

Following

The average Nigerian that are majorly tech illiterates would pick a pouvoir 2 over Redmi 9a

That said, mediatek na Baba for that coding stuff, hence why am always having reservation's about their G series

skywalker240:
The average Nigerian that are majorly tech illiterates would pick a pouvoir 2 over Redmi 9a

That said, mediatek na Baba for that coding stuff, hence why am always having reservation's about their G series

No be only you lol

How come faster Soc like the snapdragon 888 show reduced performance on phones like samsung s21 ultra and show increased performance on other phones like mi 11 ultra? Is it a software issue?

Newman20:
How come faster Soc like the snapdragon 888 show reduced performance on phones like samsung s21 ultra and show increased performance on other phones like mi 11 ultra? Is it a software issue?

Optimization is the word you're looking for.

This thread deserve to be on the front page

Newman20:
How come faster Soc like the snapdragon 888 show reduced performance on phones like samsung s21 ultra and show increased performance on other phones like mi 11 ultra? Is it a software issue?

he talked about it in the thread. And it is one of the reason why people don't trust benchmark scores

atheistandproud:


No be only you lol

As in enh

benchmark wouldn't also account for long term use

Deicide:
benchmark wouldn't also account for long term use

Bless you. It doesn't account for drop offs in performance as the CPUs age and can no longer run as fast as they used to.

I sell computers and it gets worse. One question customers always ask is " what is the speed". They'll tell you that your 10th generation core i5 laptop @1.10GHz is slow and that an old school core 2 duo laptop clocked at 3.0GHz would be faster.
The thing dey tire me seriously.

Beautiful.. Love the write up

Darkseid:
I sell computers and it gets worse. One question customers always ask is " what is the speed". They'll tell you that your 10th generation core i5 laptop @1.10GHz is slow and that an old school core 2 duo laptop clocked at 3.0GHz would be faster.
The thing dey tire me seriously.

Foto! No be juju be that!

KallmemrB0:
Beautiful.. Love the write up

Thank you

Darkseid:
I sell computers and it gets worse. One question customers always ask is " what is the speed". They'll tell you that your 10th generation core i5 laptop @1.10GHz is slow and that an old school core 2 duo laptop clocked at 3.0GHz would be faster.
The thing dey tire me seriously.

A seller showed a friend of mine two laptops, one was a 5th gen core i7, U variant with 1tb HDD and 12gb of RAM, the other was an 8th gen core i5, also U variant, with 256gb SSD with extra space for HDD and 8gb ram. Both were going for 175k, I told him the i5 was better, and he looked at me and said; "guy, you well so, you no see the numbers?". He burst into laughter and bought the i7. I wondered why he needed my opinion in the first place.

Hakeem12:
A seller showed a friend of mine two laptops, one was a 5th gen core i7, U variant with 1tb HDD and 12gb of RAM, the other was an 8th gen core i5, also U variant, with 256gb SSD with extra space for HDD and 8gb ram. Both were going for 175k, I told him the i5 was better, and he looked at me and said; "guy, you well so, you no see the numbers?". He burst into laughter and bought the i7. I wondered why he needed my opinion in the first place.

Eeeeh!

The i5 was the better option na. Ignorance!!!

atheistandproud:


Eeeeh!

The i5 was the better option na. Ignorance!!!

Yeah, but you could say that cos you understand better, my guy was looking at numbers. I'd have taken time to break it down to him, but he was laughing so hard, I felt like punching him. So I just shrugged and let him buy it, it's his money in the end. He doesn't know PC processors are even more complicated. There's generations to compare, there's variants, U, K, H and so on. Even I don't know that much about AMD processors yet.

Hakeem12:
A seller showed a friend of mine two laptops, one was a 5th gen core i7, U variant with 1tb HDD and 12gb of RAM, the other was an 8th gen core i5, also U variant, with 256gb SSD with extra space for HDD and 8gb ram. Both were going for 175k, I told him the i5 was better, and he looked at me and said; "guy, you well so, you no see the numbers?". He burst into laughter and bought the i7. I wondered why he needed my opinion in the first place.

I can totally relate to this

Hakeem12:
A seller showed a friend of mine two laptops, one was a 5th gen core i7, U variant with 1tb HDD and 12gb of RAM, the other was an 8th gen core i5, also U variant, with 256gb SSD with extra space for HDD and 8gb ram. Both were going for 175k, I told him the i5 was better, and he looked at me and said; "guy, you well so, you no see the numbers?". He burst into laughter and bought the i7. I wondered why he needed my opinion in the first place.

Please I'll like to know why the i5 is better

Thanks

atheistandproud:


Eeeeh!

The i5 was the better option na. Ignorance!!!

Reason(s) please

Thanks

OwoukoUrua:


Reason(s) please

Thanks

Newer generations of CPUs are always better than the older versions.

Newer generations have better architecture, fab sizes, production styles, newer technology, better optimization etc.

In the scenario he painted...

The i5 was the newer CPU and the distance between an old i7 and a new i5 is kinda wide. In addition, even though the SSD was smaller in size, it is way faster and would help the CPU perform even much better.

Hakeem12:
A seller showed a friend of mine two laptops, one was a 5th gen core i7, U variant with 1tb HDD and 12gb of RAM, the other was an 8th gen core i5, also U variant, with 256gb SSD with extra space for HDD and 8gb ram. Both were going for 175k, I told him the i5 was better, and he looked at me and said; "guy, you well so, you no see the numbers?". He burst into laughter and bought the i7. I wondered why he needed my opinion in the first place.

Numbers don deceive the guy. And truly it's the same with most Nigerians.

OwoukoUrua:


Reason(s) please

Thanks

If you check the benchmarks of both you'd be shocked at the gap the i5 gave the i7 though it shouldn't be surprising since it's ages newer.
Another reason is the specs the laptop itself has. SSD with an option to expand through HDD. It's too good to let i7 big name pass you by.

RuneKingThor:

Numbers don deceive the guy. And truly it's the same with most Nigerians.

This is the truth lol.

To add to this, Nigerians like simple explanations without nuance or substance hence they're easily manipulated.

atheistandproud:


Newer generations of CPUs are always better than the older versions.

Newer generations have better architecture, fab sizes, production styles, newer technology, better optimization etc.

In the scenario he painted...

The i5 was the newer CPU and the distance between an old i7 and a new i5 is kinda wide. In addition, even though the SSD was smaller in size, it is way faster and would help the CPU perform even much better.

Thank you

atheistandproud:


This is the truth lol.

To add to this, Nigerians like simple explanations without nuance or substance hence they're easily manipulated.

I really hope the narrative changes really soon.

RuneKingThor:

If you check the benchmarks of both you'd be shocked at the gap the i5 gave the i7 though it shouldn't be surprising since it's ages newer.
Another reason is the specs the laptop itself has. SSD with an option to expand through HDD. It's too good to let i7 big name pass you by.

Thanks

OwoukoUrua:


Please I'll like to know why the i5 is better

Thanks

The processors are three generations apart, the i5 being the latest one. The i5 has also has a faster ram speed so the 8gb to 12gb is negligible. Then the storage, SSD is way faster than HDD, wouldn't touch a laptop with HDD these days. Faster loading times is noticeable on SSD, so real life performance is miles apart.