Geekbench says enabling Intel BOT paints an 'unrealistic picture' of CPU performance and makes Intel chips seem faster versus the AMD competition 'than they would be in typical, real-world usage'
The company behind the popular Geekbench software tool has taken another pot shot at Intel's BOT software.
Primate Labs says that Intel BOT "only supports a handful of applications, meaning BOT-optimized benchmark results paint an unrealistic picture of how a CPU performs in practice. This makes Intel processors appear faster relative to AMD and other vendors than they would be in typical, real-world usage."
Following the announcement last week that all Geekbench results posted courtesy of Intel's new Arrow Lake Plus CPUs would receive warning flag in the online database, Primate Labs has a new blog post detailing the results of in-house testing carried out with Intel's BOT software.
Primate Labs actually carried out their testing on a Panther Lake laptop rather than one of the new Arrow Lake Plus chips. Turns out Intel's Panther Lake CPUs also support BOT, something Intel didn't initially flag.
Anywho, Primate Labs says, "Intel’s public documentation on BOT is limited, so we decided to dig in ourselves to understand how it works and what optimizations it’s applying to Geekbench."
Primate Labs claims it discovered that BOT is going well beyond Intel's characterisation of how the software operates.
"Based on the instruction counts, it’s clear BOT has performed significant changes to the HDR workload’s code. The number of total instructions is reduced by 14%. Most of that reduction comes from BOT vectorizing parts of the workload’s code, converting instructions that operate on one value into instructions that operate on eight values.
"This is a significantly more sophisticated transformation than simple code-reordering. Intel’s public documentation only discloses the simpler code-reordering techniques, not the vectorization transformations observed here," Primate Labs says.
If Primate Labs is correct, that's certainly something of a concern. To quote myself from last week, "my understanding is that what Intel's BOT is doing essentially amounts to re-ordering instructions so that they fully utilise the Arrow Lake Plus pipeline. All the actual calculations are the same. In other words, enabling BOT doesn't mean skipping any work."
I said that based on Intel's description of how BOT works, but if Primate Labs is correct, there's quite a bit more going on. Primate Labs also found that BOT adds a time penalty to application start up. "When running Geekbench 6.3 with BOT enabled, the first run has a 40-second startup delay before the program starts. Subsequent runs are faster, with a 2-second startup delay. The startup delay disappears when BOT is disabled," the blog post explains.
Ultimately, this all comes down to how many applications end up supporting BOT. My understanding is that application support will require Intel's Labs specifically doing optimisation work and adding the results of that to the tool. Quite how many apps Intel will choose to optimise is unclear, but the fact that BOT is never going to just work with any given app is a clear negative. Primate Labs probably has a point here, therefore.