NeuroFusion Tops ForecastBench Preliminary Leaderboard, Proving Ability to Predict the Odds of Future Events
TECHWORLD ·
✦ AI Summary
NeuroFusion said on the 21st that its financial AI research team took first place on the preliminary leaderboard of ForecastBench, a global AI future-prediction benchmark.
ForecastBench consists of real-world prediction tasks across areas including economics, finance, politics, and technology, and asks only about future events whose results have not yet been determined at the time of submission.
NeuroFusion said that more than 530 data pipelines built over 4 years, 167 finance-specific MCPs, a refined financial data infrastructure, and finance-specific reasoning technology formed the basis of the achievement.
NeuroFusion said on the 21st that its financial AI research team took first place on the preliminary leaderboard of ForecastBench, a global AI future-prediction benchmark.
ForecastBench consists of real-world prediction tasks across areas including economics, finance, politics, and technology, and is designed to evaluate AI's ability to forecast the future. The benchmark is recognized as authoritative in evaluating AI future-prediction capabilities and is presented as a standard for proving the ability to predict the probabilities of future events.
What sets ForecastBench apart is that it asks only about future events whose results have not yet been determined at the time of submission. Unlike other benchmarks that solve problems whose answers are already known, ForecastBench makes it impossible to improve scores through pretraining on publicly available data.
The questions are automatically updated every 2 weeks. The question data draws on economic and financial time-series data from sources such as FRED and Yahoo Finance, and includes real future events from major prediction markets such as Polymarket and Metaculus.
NeuroFusion said its AI research team ranked first on the dataset category's preliminary leaderboard. The evaluation standard was a Brier Index score of 67.5, and the comparison set included a total of 464 models. The company also said that, including the team’s anonymous submitted experimental models, it took the No. 1, No. 2, No. 3, and No. 4 spots. It added that the No. 5 team scored 65.9, while Google's DeepMind's top model scored 65.0.
NeuroFusion said the achievement is based on research progress made during the process of building Demian, a 4-stage dynamic time-series graph for use in financial markets and forecasting applications.
The company cited more than 530 data pipelines built over 4 years as a factor supporting the result.
It also said that 167 finance-specific MCPs, a refined financial data infrastructure, and finance-specific reasoning technology formed the foundation of the achievement.
NeuroFusion said that after confirming in July that it outperformed the latest general-purpose AI models on its own 84-question financial evaluation set, it entered this externally public benchmark and competed on equal terms with global AI models, seeking to verify its performance through an external public benchmark.
Choi Han-cheol, CEO of NeuroFusion, said that social-science future forecasting requires inductive reasoning in the quant domain and deductive and abductive reasoning based on general-purpose LLMs, and that it also requires a separate architecture to understand the dynamics of the thoughts, cognition, and emotions of humans who participate in financial markets. He said that only about half of that architecture has been implemented so far, and that the company achieved this result through research conducted during the implementation process, expressing his pleasure at the outcome. He added that the company plans to apply the prediction capabilities validated on the global stage across Valley AI's research functions and support investors' decisions with a more sophisticated analytical foundation.
Source: TECHWORLD · Lee Gwang-jae
Original: https://www.epnc.co.kr/news/articleView.html?idxno=407178
References
This article was produced with the help of an automated content generation algorithm.
Source: TECHWORLD
View originalThis article was summarized and organized by BizCrush based on the original article from TECHWORLD. For exact quotations and full details, please refer to the original article.