English | EN 
The rapid rise of cloud computing has transformed DevOps practices by enabling large-scale automation, scalability, and flexibility across distributed systems. Among the major cloud providers, Microsoft Azure stands out with its impressive range of over 200 cloud services, from virtual machines and serverless architectures to advanced governance, monitoring, and security tools. However, this growing diversity presents a significant challenge for DevOps engineers and finance teams. They must navigate a complex landscape of choices, configuring and optimizing Azure resources while balancing technical, financial, and regulatory constraints that are often in conflict.
In response to this complexity, Large Language Models (LLMs) offer a promising solution. With their ability to understand natural language, reason through complex requirements, and generate contextualized responses, LLMs pave the way for a new generation of intelligent assistants that can support cloud engineers in decision-making processes.
Where does the motivation come from?
Rapid advances in the development of various techniques leveraging LLMs have opened a promising direction for designing intelligent chatbots tailored to specific use cases. These advancements aim to assist engineers in managing configuration, compliance, cost, and architecture design more efficiently. Several approaches illustrate this trend:
- LADs (LLM-Aided DevOps Systems) 1 introduce a single-agent architecture designed to generate valid configuration files from user intentions expressed in natural language. It leverages advanced techniques such as retrieval-augmented generation (RAG), prompt chaining, and adaptive feedback to enhance accuracy and relevance.
- MOYA 2, developed by MontyCloud in collaboration with IIIT Hyderabad, proposes a modular multi-agent architecture orchestrated around a central core. Each agent, whether LLM-based or not, is responsible for automating a specific domain of CloudOps, such as cost management, security, or compliance.
- Another promising direction, led by Chavva & Veera3, explores an end-to-end pipeline for automating cloud architecture design. This approach spans the entire lifecycle, from requirements gathering to blueprint generation, integrating LLMs throughout the process to support intelligent decision-making.
These works strongly inspire the development of a chatbot that can respond to the case of optimized cloud configuration recommendations.
Challenges and our approach
Developing an Azure-centered chatbot for optimized cloud configuration recommendations presents several key challenges. Below, these obstacles are outlined, and how an LLM-based chatbot can address each of them.
Navigating the complexity of the Azure ecosystem
As highlighted in technical articles from the Microsoft Azure DevOps Blog4 and in Gartner’s 2023 reports on multi-cloud management5, the sheer diversity of Azure services often creates a cognitive overload for DevOps engineers. They are required to select and configure the most suitable services based on complex, and often conflicting, technical and business requirements. To address this, an LLM-based chatbot introduces a domain-specific intelligent assistant capable of simplifying Azure’s vast and intricate service offerings, making it easier for users to identify and utilize relevant tools.
Time-consuming navigation of documentation
The official Azure documentation is widely recognized for its depth and comprehensiveness. However, it is also frequently cited as difficult to access and apply in real-world contexts. Numerous discussions on technical forums, such as Stack Overflow and Reddit’s AZURE community, highlight the challenges of quickly finding accurate, use-case-specific information when it is needed most. By combining a structured knowledge base with Retrieval-Augmented Generation (RAG), the chatbot provides fast and contextual access to accurate documentation, reducing the time spent searching for information.
Managing cost optimization (FinOps) in real-time
Cost management (FinOps), performance, availability (SLA), and regulatory compliance (notably GDPR) are critical concerns in cloud operations. According to the “State of FinOps 2023” report published by the FinOps Foundation6, the financial management of cloud resources remains a major challenge for DevOps teams, particularly when dealing with the complexity of Azure and multi-cloud environments. To address this, cost analysis is integrated into the recommendation engine, enabling the assistant to suggest configurations that align with both performance and financial constraints.
Making informed configuration decisions
The lack of tools offering dynamic and contextualized recommendations that align with business, technical, and financial requirements has risen7. To respond to this need, the LLM-based chatbot can leverage a cross-referenced taxonomy and dynamic prompt engineering to enable the chatbot to guide users through decision-making, adapting to specific requirements and usage patterns.
Ensuring accessibility for diverse user profiles
The chatbot is designed for different user profiles, such as DevOps engineers and finance teams, each with distinct needs and interests regarding the responses they expect. To accommodate this diversity, the chatbot adapts its answers based on the user’s profile, ensuring an intuitive experience for both technical and non-technical users. It learns and fine-tunes its responses by leveraging different databases or pre-trained models adapted to each user group, enabling personalized and relevant guidance.
Conclusion
LLM-based techniques offer new opportunities to develop a chatbot adapted to the needs of optimized cloud configuration. By improving the way cloud resources are selected and configured, this chatbot can help ensure more efficient resource utilization, resulting in cost savings, reduced energy consumption, and minimized human effort.
Acknowledgement
This blog is based on the research of Achraf Jemali, an intern at SogetiLabs contributing to the AI4FinOps project. For further information, please contact achraf.jemali@sogeti.com.