English | EN 
Introduction
The transition from static deep learning models to autonomous AI agents is a thrilling leap forward for medical diagnostics. Armed with iterative reasoning frameworks like ReAct (Reason + Act) (Yao et al., 2023a) or Tree of Thoughts (Yao et al., 2023b), these agents can dynamically query tools, look up patient histories, and order specialized sub-routines.
However, long-horizon clinical tasks expose a severe cognitive vulnerability in large language model (LLM)-based architectures: plan failure and goal drift. In complex medical scenarios, an agent can easily get trapped in its own reasoning loop, lose track of the primary diagnostic objective, or wander down irrelevant clinical rabbit holes. For researchers and clinicians alike, understanding and solving this structural instability is paramount to making agentic AI safe for patient care (Kim et al., 2024).
1. Anatomy of a plan failure: How agents get lost
When a human clinician encounters a complex, multi-system medical case, they maintain a rigid mental hierarchy of objectives. They prioritize stabilizing the patient and reaching a primary differential diagnosis before exploring secondary, incidental findings.
AI agents, by contrast, process information sequentially based on local context windows. When tasked with analyzing a multimodal record—such as a patient with an ambiguous pulmonary lesion, a history of autoimmune disease, and borderline lab results—agents frequently experience two types of structural failures (Schmidgall et al., 2024; Cemri et al., 2025):
- Goal Drift: The agent becomes hyper-focused on an incidental finding (e.g., an isolated, slightly abnormal lab value) and spends its computational steps querying tools related to that subset, completely forgetting to finalize the diagnosis of the primary tumor.
- Infinite Loops: The agent interprets the output of a tool incorrectly, panics, and repeatedly calls the same or a similar tool with slightly modified parameters, burning through its maximum step budget without ever reaching a conclusion.
2. The root cause: Context dilution and myopic reasoning
Why do diagnostic agents lose their way? The core issue lies in how information accumulates within the agent’s prompt context.
As an agent interacts with external tools (e.g., retrieving electronic health records, calling image segmentation pipelines, querying clinical calculators), the raw outputs of these tools are appended to its history. This rapidly bloats the context window. Critical initial information—such as the chief complaint or the primary diagnostic constraint—gets diluted by pages of raw tabular data or verbose API responses (Liu et al., 2024). The agent begins making its next decision based primarily on the most recent tool output rather than the global clinical objective. This “myopic reasoning” makes long-term planning incredibly fragile.
3. Engineering solutions: Memory structures and internal critics
To transform these chaotic loops into reliable clinical workflows, AI researchers are moving away from simple linear prompting and toward advanced agentic architectures.
Episodic memory and state tracking
Instead of dumping raw tool outputs directly into the context window, modern diagnostic agents utilize a dual-memory system (Packer et al., 2023; Sumers et al., 2024). An external state tracker explicitly maintains a structured, updating summary of what is known, what is unknown, and what needs to be answered. Before every action, the agent is forced to read this concise “clinical state dashboard,” preventing it from drifting away from the primary objective.
Monte Carlo Tree Search (MCTS) and self-correction
Rather than letting the agent execute actions blindly, researchers are implementing search algorithms like Monte Carlo Tree Search (MCTS) to let the agent simulate multiple diagnostic paths in a hidden scratchpad before executing them (Hao et al., 2023; Zhou et al., 2024).
Furthermore, integrating an internal critic(Shinn et al., 2023) —a separate, highly constrained LLM instance whose sole job is to cross-examine the main agent’s actions—creates a system of checks and balances. If the agent starts calling the same API repeatedly, the critic flags the loop and forces a backtracking routine, mimicking a senior physician redirecting a lost medical resident.
4. Moving toward deterministic boundaries
While non-deterministic, flexible reasoning is what makes agents powerful, medicine demands guardrails. The frontier of this research involves wrapping agentic loops in finite-state machines (FSMs). By forcing the agent to operate within pre-defined clinical phases (e.g., Phase 1: Information Gathering, Phase 2: Differential Expansion, Phase 3: Diagnostic Synthesis), we can mathematically guarantee that the agent cannot jump to a final conclusion or drift infinitely without fulfilling strict structural prerequisites (Wu et al., 2024).
Conclusion
Autonomous agents hold the key to truly intelligent diagnostic assistance, but their freedom must be tempered with structural discipline. Overcoming goal drift and planning failures through advanced memory management, search algorithms, and algorithmic state-tracking is not just an engineering optimization—it is an absolute prerequisite for clinical safety.
References
Cemri, M., Pan, M.Z., Yang, S. et al. (2025) ‘Why do multi-agent LLM systems fail?’, arXiv preprint, arXiv:2503.13657. Available at: https://arxiv.org/abs/2503.13657
Hao, S., Gu, Y., Ma, H., Hong, J., Wang, Z., Wang, D.Z. and Hu, Z. (2023) ‘Reasoning with language model is planning with world model’, Proceedings of EMNLP 2023, Singapore, pp. 8154–8173. Available at: https://arxiv.org/abs/2305.14992 (version ACL : https://aclanthology.org/2023.emnlp-main.507/)
Kim, Y., Park, C., Jeong, H. et al. (2024) ‘MDAgents: An adaptive collaboration of LLMs for medical decision-making’, Advances in Neural Information Processing Systems (NeurIPS), 37. Available at: https://arxiv.org/abs/2404.15155
Liu, N.F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F. and Liang, P. (2024) ‘Lost in the middle: How language models use long contexts’, Transactions of the Association for Computational Linguistics, 12, pp. 157–173. doi: 10.1162/tacl_a_00638. Available at: https://arxiv.org/abs/2307.03172
Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S.G., Stoica, I. and Gonzalez, J.E. (2023) ‘MemGPT: Towards LLMs as operating systems’, arXiv preprint, arXiv:2310.08560. Available at: https://arxiv.org/abs/2310.08560
Schmidgall, S., Ziaei, R., Harris, C. et al. (2024) ‘AgentClinic: A multimodal agent benchmark to evaluate AI in simulated clinical environments’, arXiv preprint, arXiv:2405.07960. Available at: https://arxiv.org/abs/2405.07960
Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K. and Yao, S. (2023) ‘Reflexion: Language agents with verbal reinforcement learning’, Advances in Neural Information Processing Systems (NeurIPS), 36. Available at: https://arxiv.org/abs/2303.11366
Sumers, T.R., Yao, S., Narasimhan, K. and Griffiths, T.L. (2024) ‘Cognitive architectures for language agents’, Transactions on Machine Learning Research (TMLR). Available at: https://arxiv.org/abs/2309.02427
Wu, Y., Yue, T., Zhang, S., Wang, C. and Wu, Q. (2024) ‘StateFlow: Enhancing LLM task-solving through state-driven workflows’, arXiv preprint, arXiv:2403.11322. Available at: https://arxiv.org/abs/2403.11322
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. and Cao, Y. (2023a) ‘ReAct: Synergizing reasoning and acting in language models’, International Conference on Learning Representations (ICLR). Available at: https://arxiv.org/abs/2210.03629
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T.L., Cao, Y. and Narasimhan, K. (2023b) ‘Tree of Thoughts: Deliberate problem solving with large language models’, Advances in Neural Information Processing Systems (NeurIPS), 36. Available at: https://arxiv.org/abs/2305.10601
Zhou, A., Yan, K., Shlapentokh-Rothman, M., Wang, H. and Wang, Y.-X. (2024) ‘Language Agent Tree Search unifies reasoning, acting, and planning in language models’, Proceedings of the 41st International Conference on Machine Learning (ICML). Available at: https://arxiv.org/abs/2310.04406