Large language models (LLMs) show promise in tackling planning problems, but there’s a balance between flexibility and complexity. While LLMs can act as zero-shot planners, they struggle with complex tasks involving multiple constraints or long-term goals.
Many frameworks that address these challenges require task-specific preparation, such as tailored examples and predefined validators, which limits their ability to adapt to different tasks.
MIT researchers have developed a “smart assistant” that enhances LLMs’ planning capabilities. Instead of modifying the model itself, this framework -LLM-Based Formalized Programming (LLMFP)—helps the LLM break down complex problems into manageable steps, mimicking human reasoning. It then uses a robust software tool to solve the problem efficiently.
Humans often tackle complex problems by narrowing them down to a few choices and selecting the best solution. Similarly, the researchers’ algorithmic solvers use this approach to handle optimization problems far beyond human capacity. However, these solvers are challenging to learn and are usually reserved for experts.
When a user describes a problem in natural language, the model transforms their input into a format suitable for an optimization solver designed to handle highly complex planning challenges.
New algorithm compresses LLM’s reams of data to increase privacy
Throughout this process, the LLM verifies each step to ensure the plan is accurately represented. If it detects an error, it corrects the flawed portion of the formulation to maintain accuracy and efficiency.
This framework was tested on nine complex challenges. It achieved an 85 percent success rate, whereas the best baseline only achieved a 39 percent success rate. This suggests its applications in various multistep planning tasks, such as scheduling airline crews or managing machine time in a factory.
The self-assessment module enables the LLM to incorporate any implicit constraints it initially overlooked. Additionally, the LLM can adjust to user preferences. For instance, if it identifies that a user prefers not to alter the time or budget for their travel plans, it can propose modifications that align better with their needs.
Journal Reference
- Yilun Hao, Yang Zhang et al. Planning Anything with Rigor: General-purpose Zero-shot Planning with LLM-based Formalized Programming. arXiv: 2410.12112v2
Source: Tech Explorist
