Ansible Playbook Assistant: Powered by Multi-Agent AI with LangGraph

My initial attempt at using a Large Language Model (LLM) was to generate an Ansible playbook. However, the LLM's hallucination of module names and parameters hindered further progress. Moreover, my own tool's performance was not better to directly utilizing ChatGPT 4 or Meta AI. I struggled to overcome these challenges until I saw LangChain's Multi-Agent feature with LangGraph.

LangGraph supports multi-agent workflows, where each agent can be a specialized LLM or tool. These agents can work collaboratively, sharing information as needed to accomplish tasks more effectively. This design helps in dividing complex problems into manageable units handled by different agents.

Multiple agents specialize in different topics, with seven agents (Task 1, 2, 3, 5, 6, 7 and 8) assigned to specific tasks.

Tasks 1, 2, and 3 concentrate on creating a high-level playbook development plan, while Tasks 5, 6, 7 and 8 focus on developing the playbook based on the established high-level plan.

LangGraph offers a framework for agent state management, enabling data sharing across agents. Additionally, the Pydantic library helps to transform unstructured LLM output into structured data, which can then be seamlessly consumed by Python code.

The following is the workflow for generating playbooks:

Task 1: Prepare data for high level plan

It leverages the Large Language Model (LLM) to generate a list of search queries, which are then used to search the internet and collect data for the following high-level outline plan.

Task 2: Generate or Update High Level Plan

Using the data from Task 1, the LLM generates a High-Level Plan, which includes the sub-task name, description, and recommended module. If a High-Level Plan already exists, it is updated based on the comments from task 3.

Task 3: Verify the module name used in high level plan

Utilize the LLM to identify module names in the High-Level Plan, then search for them online. If a module name is not found, provide a comment for updating the High-Level Plan.

Task 4: Update High Level Plan?

If the maximum update time is reached, terminate the workflow due to failure to find a suitable Ansible module for the given topic. However, if there are comments from Task 3, loop back to Task 2. Otherwise, proceed to Task 5 if no comments are present.

Task 5: Prepare the data for playbook

Based on the high-level plan, the LLM produces a list of search queries, which are subsequently employed to search the internet and gather data for the subsequent task of creating a playbook.

Task 6: Generate Playbook

Using the high-level plan and preparation data from Task 5, the LLM generates the playbook.

Task 7: Collect the examples for the modules used in playbook

Extract playbook modules and collect online example code for each module.

Task 8: Update playbook

Using the example codes gathered in Task 7, update and refine the existing Ansible playbook.

Test Question:

Sample Ansible Playbook to take aws VM snapshot

This AI tools

ChatGPT4

Meta AI

This tool's output heavily depends on the quality of web search results at the time of query, which can lead to varying answers to the same question based on different internet data. Currently, all input data is sourced from web searches.

Furthermore, I have concerns about the tool's ability to handle complex tasks that involve complicated logic across sub-tasks, which may require more robust high-level planning agents. So far, the high-level plan agents have focused more on Ansible playbook best practices rather than analyzing and understanding the specific problem domain or topic itself.