Google Pilots Double-Blind Approach to AI Performance Verification

According to AI TIMES, Google said on August 27 that it will pilot a double-blind method for evaluating AI performance and safety, in which models and evaluation data are prevented from seeing each other's contents. Google described it as the world's first pilot, aiming to address the problem of benchmark contamination, which can inflate performance beyond reality. The key feature of the approach is that evaluating organizations cannot inspect model weights, while model developers also cannot see the private test prompts. The company said it is an attempt to improve the reliability of evaluations by applying a secure environment that can be cryptographically verified, rather than relying only on contracts or operating rules. It also said the approach matters because it could create a foundation for outside organizations to independently examine models even in areas such as sensitive safety validation, where it is difficult to disclose test questions. Google plans to use the pilot to refine its evaluation methodology and expand its AI verification framework to assess not only scores, but also how trustworthy the results are.