Most similar is CT Open, which is also a trial outcome predictor. ForecastBench does the same but for general questions.
For 111 trials reported after the cutoff date, Opus/Sonnet/Haiku was too pessimistic (~59% expected vs. 73% observed) and was beaten by a plain regression on the registry features. Their ordering of the trials was good though, and combining with the regression got them 8% accuracy increase. Small sample size; thus, the online version
For 111 trials reported after the cutoff date, Opus/Sonnet/Haiku was too pessimistic (~59% expected vs. 73% observed) and was beaten by a plain regression on the registry features. Their ordering of the trials was good though, and combining with the regression got them 8% accuracy increase. Small sample size; thus, the online version