tree of thoughts

A prompting framework that turns chain of thoughtchain of thought, n.: Getting an AI to write out its reasoning step by step before giving an answer, the way a student shows their working. This often makes answers to math and logic problems more accurate. from a single line of reasoning into a search over many. A problem is split into steps; at each step the language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. proposes several candidate 'thoughts' (partial solutions), the same or another model rates how promising each resulting state is, and a classical search procedure — breadth-firstbreadth-first search, n.: A way of exploring a maze or network level by level — first every place one step away, then every place two steps away, and so on — so that the shortest route is found first. or depth-first searchdepth-first search, n.: A way of exploring a maze or network by following one path as far as it goes, then backing up to the most recent fork and trying the next branch. with pruning and backtracking — chooses which branches to pursue. The model can thus look ahead and abandon a bad start, which strictly left-to-right generation cannot do. In the original experiments on the Game of 24 arithmetic puzzle, GPT-4 succeeded on 74% of problems this way, against 4% with chain-of-thought prompting. The price is many more model calls; the approach influenced later search-based and agentic reasoning, though trained reasoning modelsreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning. now do much of this exploration within a single trace.

We wrapped the planner in a tree-of-thoughts search so it could abandon bad itineraries early.