Setting Up a Private Multi-Agent Framework From Scratch

What "private" should actually mean

A lot of products call themselves multi-agent platforms while running your agents on their servers, logging your prompts, or requiring a hosted account before anything works. None of that is private in the sense a developer usually means when searching for this. A private multi-agent framework is a library you install and run yourself, where your code and your agents' conversations never leave your machine except in the one call you make on purpose, to the model provider you chose.

That definition rules nothing in or out by vendor. Microsoft, Google and various startups all ship open source frameworks that satisfy it, and a framework can be privately run even if the company behind it also sells a managed version of the same thing. The question to ask about any option is narrower than "is it open source": does it phone home by default, and can you turn that off.

• Private means you run it, not a vendor running it for you.
• Open source and private are related but not the same thing.
• The real question is whether anything calls out by default, and how you disable it.

The frameworks worth considering

CrewAI is a lean Python framework, released under the MIT license, for orchestrating what it calls Crews, teams of autonomous agents, and Flows, event-driven workflows between them. It installs and runs entirely on your machine, with no cloud backend required to use it.

AG2 describes itself as an open-source operating system for agentic AI and is licensed under Apache 2.0. It is a continuation of Microsoft's original AutoGen project, maintained by its own community, and it keeps a separate Classic mode for people migrating old AutoGen code. It has moved away from the older OAI_CONFIG_LIST file in favor of reading standard environment variables per provider.

LangGraph is built by LangChain Inc as a lower-level orchestration runtime for long-running, stateful agents, and it can be used on its own without the rest of the LangChain ecosystem. It installs as a normal Python package and is open source.

Microsoft Agent Framework was announced on the Microsoft Foundry blog on October 1, 2025. It does not replace AutoGen or Semantic Kernel; Microsoft's own description is that it builds on both, folding AutoGen's multi-agent orchestration patterns, such as GroupChat and GraphFlow, into a single Workflow abstraction while keeping Semantic Kernel's enterprise features. Both older projects remain supported, but Microsoft says most new investment now goes into Agent Framework.

• CrewAI: MIT license, Crews and Flows, Python, runs locally.
• AG2: Apache 2.0, the open source continuation of AutoGen, env-var based keys.
• LangGraph: built by LangChain Inc, usable without LangChain itself.
• Microsoft Agent Framework: announced October 2025, unifies AutoGen and Semantic Kernel.

Building one from scratch: a CrewAI walkthrough

CrewAI is the quickest to stand up, so it is a reasonable default for a first private setup. It requires Python 3.10 or newer and below 3.14. Install the uv package manager first, with curl -LsSf https://astral.sh/uv/install.sh | sh on macOS or Linux, then install the CrewAI CLI itself with uv tool install crewai.

From there, crewai create crew your_project_name scaffolds a new project. The current default generates JSON-first configuration files, agents/*.jsonc and crew.jsonc; pass --classic to the create command if you prefer the older Python-and-YAML layout instead. Run crewai install to pull dependencies into the project, add your model API key as an environment variable rather than hardcoding it in the config, and start the crew with crewai run.

AG2 and LangGraph follow the same shape: pip install the library, define your agents in a script, set the provider's environment variable, and run the script with a normal Python interpreter. None of the three needs an account with the framework's own vendor to function.

• Install uv, then the crewai CLI, with two short shell commands.
• crewai create crew scaffolds a project; crewai run starts it.
• Set API keys as environment variables, never inside a checked-in config file.
• AG2 and LangGraph install the same way any Python package does.

Keeping it private once it is running

Installing a framework locally does not automatically mean nothing leaves your machine. CrewAI, for one, sends anonymous usage telemetry by default: software versions, crew and agent configuration shapes, and task lifecycle events, explicitly excluding prompt text, task descriptions or agent backstories. You can turn it off entirely by setting the environment variable CREWAI_DISABLE_TELEMETRY to true, or by disabling OpenTelemetry generally with OTEL_SDK_DISABLED. Check whichever framework you pick for an equivalent switch before assuming silence means nothing is sent.

If you want the model calls themselves to stay off the network too, run an open-weight model locally with a tool like Ollama instead of calling a hosted API. That trades capability for privacy; a locally run model is usually smaller and slower than the frontier models these frameworks call by default, so most teams reserve it for the agents that handle the most sensitive data and keep a hosted model for the rest.

• Check for a telemetry switch and use it; do not assume local means silent.
• CrewAI's default telemetry explicitly excludes prompt and task content, but it is still a default you can turn off.
• A local model through a tool like Ollama removes the model provider from the loop entirely, at a capability cost.

Isolating agents from each other

The word "multi" in multi-agent framework usually describes a logical separation, several agent objects inside one Python process, not a security boundary. By default, every agent in a CrewAI crew or an AG2 conversation shares the same process, the same filesystem access and the same credentials as the script that launched them.

If one agent in the group is given a tool that can read files or run shell commands, that capability is not fenced off from the other agents just because they are logically distinct. Treat the whole crew as one trust boundary unless you deliberately split it: run the agent that needs file or shell access in its own process or container, and give it only the credentials that specific job needs.

• Agents in the same script share a process and its access by default.
• A tool granted to one agent is not automatically isolated from the others.
• Split out any agent with file or shell access into its own process if the rest should not inherit that reach.

What a private framework still does not give you

None of CrewAI, AG2, LangGraph or Microsoft Agent Framework puts an operating system boundary around the code an agent runs. If an agent's tool calls a shell command, that command runs with the same permissions as the Python process, which is to say your permissions. Keeping the framework itself private and offline does not change that.

For the agent that actually executes code, pair the framework with a real sandbox, a container, a virtual machine, or a kernel-level restriction such as macOS Seatbelt, and keep secrets out of any file that process can read.

• A private framework controls where the orchestration runs, not what a tool call can touch.
• Shell and file access inside an agent still needs its own boundary.
• Pair the framework with a sandbox for any agent that executes code.

Where Forkbench fits, and where it does not

Forkbench is not a multi-agent orchestration framework, and it would be misleading to suggest it replaces CrewAI, AG2 or LangGraph. It is a desktop app for the Mac that runs coding agent CLIs, such as Claude Code or Codex, inside real terminals, one Thread per task. If your private framework's job is to run a script, that script runs the same way in a Forkbench terminal as anywhere else.

What Forkbench adds at that layer is supervision and containment for the terminal doing the work: a Thread can be locked to its folders through the macOS kernel sandbox, and its Vault lets a command use a secret from the Keychain without the value reaching the prompt or the transcript. The folder lock is opt-in and does not restrict the network, and an unpinned key is still usable by the program it was handed to, so it narrows the same risks this guide describes rather than eliminating them.

• Forkbench runs coding agent CLIs in terminals; it is not an agent-orchestration library.
• A script from CrewAI, AG2 or LangGraph runs in a Forkbench terminal the same as any other.
• Its folder lock and Vault narrow the filesystem and key exposure of that terminal, opt-in and with stated limits.