After several years of development, feedback and testing iterations with customers, partners and the AI engineering community, Primate Labsthe developer of the cross-platform benchmarking tool Geekbench, has announced that its machine learning benchmark is now ready for general use and has a new name: Geekbench AI.
Those who have followed the previous preview versions will already know that Geekbench AI is a benchmark suite with a testing methodology for machine learning, deep learning, and artificial intelligence workloads.all with the same utility Multi-platform and the real-world workload accuracy that Primate Labs benchmarks are known for.
Software developers can use this tool to ensure a consistent experience for their applications across platforms, hardware engineers can use it to measure architectural improvements, and anyone can use it to measure and troubleshoot device performance with a suite of tasks based on how devices actually use AI.
Geekbench AI 1.0 is now available in download page at Primate Labs per Windows, macOS e Linuxas well as on the Google Play Store and Apple App Store for the versions for Android e iOS.
Previews of the new machine learning benchmark were previously called “Geekbench ML,” but Primate Labs explains that as companies have converged on the term “AI” for these workloads (and the marketing for them) over the past few years, to ensure that everyone from engineers to tech and benchmarking enthusiasts quickly understand what this application is for, it was a good idea to update the software’s name as well.
To account for the complexity of measuring performance in machine learning tasks, Geekbench AI provides three overall scores. That’s because, the team explains, the design of AI hardware varies across devices and silicon vendors, as does the way developers leverage that hardware. Just as CPU-bound workloads vary in how they can leverage multiple cores or threads to scale performance (making both single-core and multi-core metrics necessary in most related benchmarks), AI workloads span a range of levels of precision, depending on the task being asked, the hardware available, and the frameworks in between.
Geekbench AI provides the information a developer or hardware vendor would look for when it comes to analyzing the AI performance of a given device, showing performance in three dimensions instead of just one. This reflects the approach the company took in developing Geekbench AI, designed to measure AI performance in the real world: As the team points out, AI is complex, heterogeneous, and changes very quickly. With the Primate Labs benchmark, the company points out, it is possible to explore how different hardware approaches have been optimized for particular tasks.
Geekbench AI 1.0 includes other significant changes to improve its ability to measure real-world performance based on how applications use AI. This includes support for new frameworks, from OpenVINO on Linux and Windows to vendor-specific TensorFlow Lite proxies such as Samsung ENN, ArmNN, and Qualcomm QNN on Android, to better reflect the latest tools available to engineers and the ways developers are building their applications and services on the latest hardware.
This release also uses larger data sets that more closely reflect real-world inputs into AI use cases, and these larger, more diverse data sets also increase the power of the new accuracy ratings. All workloads in Geekbench 1.0 run for a minimum of one full second, which changes the impact of manufacturer- and vendor-specific performance tuning on scores, ensuring that devices can achieve peak performance levels during testing while still reflecting the choppy nature of real-world use cases.
Furthermore, and even more importantly according to the team, the difference in performance found in real life is taken into account; a five-year-old phone will be much slower at AI workloads than, say, a dedicated 450W AI accelerator. Some devices can be so incredibly fast at some tasks that testing them too short counterintuitively puts them at a disadvantage, underestimating their real-world performance in many real-world workloads.
More information is available on the site by Geekbench AI.
