Orca 2 enhances small language models' reasoning by teaching diverse strategies for tasks, outperforming models up to 10x larger in complex benchmarks.
Authors: Arindam Mitra; Luciano Del Corro, work done while at Microsoft; Shweti Mahajan, work done while at Microsoft; Andres Codas, denote equal contributions; Clarisse Simoes, denote equal contributions; Sahaj Agarwal; Xuxi Chen, work done while at Microsoft;; Anastasia Razdaibiedina, work done while at Microsoft; Erik Jones, work done while at Microsoft; Kriti Aggarwal, work done while at Microsoft; Hamid Palangi; Guoqing Zheng Corby Rosset; Hamed Khanpour; Ahmed Awadall.
Eval. In Orca 2, we continue exploring how improved training signals can enhance smaller LMs’ reasoning abilities. Research on training small LMs has often relied on imitation learning to replicate the output of more capable models. We contend that excessive emphasis on imitation may restrict the potential of smaller models. We seek to teach small LMs to employ different solution strategies for different tasks, potentially different from the one used by the larger model.
United Kingdom Latest News, United Kingdom Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Smaller homes on smaller lots: Could “light touch density” help erase Colorado’s housing deficit?A right-of-center think tank, the American Enterprise Institute, came to Denver on Monday to pitch a free market solution to resolve the state’s housing deficit in under three years and gener…
Read more »
Phase 2 of plan for smaller homes on smaller lots goes before Austin City CouncilOn Thursday, the Austin City Council will hear from the public about a plan to increase density in Austin neighborhoods as a way to increase the amount of affor
Read more »
How Leaky Datasets Undermine AI Math Reasoning ClaimsQuestions over tests of AI math abilities suggest we may never know how capable intelligent machines computers can become.
Read more »
Theoretical biologists test two modes of social reasoning and find surprising truths in simplicityImagine a small village where every action someone takes, good or bad, is quietly followed by ever-attentive, nosy neighbors. An individual's reputation is built through these actions and observations, which determines how others will treat them.
Read more »
‘Massive Man’: Role, Reasoning for Atlanta Falcons Drafting Georgia DL Zion LogueThe Atlanta Falcons selected a University of Georgia player for the third time in four years under general manager Terry Fontenot. Here’s how Zion Logue ended up in Atlanta.
Read more »
Gerrit Cole Provides Airtight Reasoning for Working Out in Full PinstripesGerrit Cole faced live batting practice on Tuesday as he looks to come back from injury. He put on his full uniform for the practice session.
Read more »