Skip to content

[BUG] crew.usage_metrics multiplies token usage by the number of agents sharing an LLM instance #7259

Description

@parthiban-sivakumar

Description

Crew.calculate_usage_metrics() loops over self.agents and adds agent.llm.get_token_usage_summary() for each one. Those counters are cumulative for the lifetime of the LLM instance, so when several agents share the same LLM object its total is added once per agent and crew.usage_metrics reports N times the real token usage.

Sharing a single LLM instance across agents is the usual way to write a crew, so this affects ordinary usage rather than an edge case. Nothing errors — the reported number is simply wrong, and the inflation grows as agents are added.

get_token_usage_summary() documents this explicitly:

The counters are cumulative for the lifetime of this instance: they grow across every call made through it, including calls issued by different agents sharing the instance.

Steps to Reproduce

  1. Create one LLM instance and give it to two agents.
  2. Run the crew.
  3. Compare crew.usage_metrics.total_tokens with llm.get_token_usage_summary().total_tokens.

Expected behavior

crew.usage_metrics should report the crew's actual token usage. Each distinct LLM instance should contribute its totals exactly once, regardless of how many agents hold a reference to it. Agents with separate LLM instances should still sum normally.

Screenshots/Code snippets

llm = LLM(model="ollama/qwen2.5:14b")
a1 = Agent(role="A", goal="Answer in one word.", backstory="B", llm=llm)
a2 = Agent(role="B", goal="Answer in one word.", backstory="B", llm=llm)  # same instance
crew = Crew(agents=[a1, a2], tasks=[t1, t2])
crew.kickoff()

llm.get_token_usage_summary().total_tokens   # 155
crew.usage_metrics.total_tokens              # 310
agents sharing one LLM instance: True
LLM instance actual total  : 155
crew.usage_metrics reports : 310
ratio                      : 2.00x

The inflation is linear in the number of agents sharing the instance. Stubbing the summary at a known 100 tokens and varying the agent count:

1 agent(s)  sharing 1 LLM (real 100) -> reported  100  (1x)
2 agent(s)  sharing 1 LLM (real 100) -> reported  200  (2x)
3 agent(s)  sharing 1 LLM (real 100) -> reported  300  (3x)
5 agent(s)  sharing 1 LLM (real 100) -> reported  500  (5x)

Operating System

Other (specify in additional context)

Python Version

3.12

crewAI Version

1.15.18

crewAI Tools Version

1.15.18

Virtual Environment

Venv

Evidence

  • lib/crewai/src/crewai/crew.py:2208-2216 — the loop adds agent.llm.get_token_usage_summary() once per agent, with no de-duplication of the underlying instance.
  • lib/crewai/src/crewai/llms/base_llm.py:1004get_token_usage_summary() returns lifetime totals for the instance and its docstring notes those include calls from other agents sharing it.
  • The same applies to manager_agent.llm at crew.py:2222-2225 when the manager shares an instance with the agents.

Reproduced both with a live crew and deterministically by stubbing the summary, so the ratio does not depend on model behaviour.

Possible Solution

Track which LLM instances have already contributed, by object identity, and skip repeats:

counted_llms: set[int] = set()

for agent in self.agents:
    if isinstance(agent.llm, BaseLLM):
        if id(agent.llm) in counted_llms:
            continue
        counted_llms.add(id(agent.llm))
        ...

with the same identity check applied to the manager agent's LLM. id() rather than model name matters here: two agents may legitimately hold separate LLM(model="...") instances with the same model, and those must still sum.

Additional context

Related but distinct: #4934 (open since 2026-03-18) fixes a different double-count in this same function, where the manager agent is counted twice because _token_process and the LLM summary are added in two independent if blocks rather than mutually exclusive branches. That change is still needed after this one; the two address different mechanisms. I've deliberately not touched the if/else restructure that PR proposes.

My OS is macOS Tahoe 26.5.2 and I'm on Python 3.13.13, neither of which is in the dropdowns. Running from a source checkout of main at commit 92eb5f9.

This issue was written with AI assistance and should carry the llm-generated label per CONTRIBUTING.md. I can't apply labels myself — could a maintainer add it?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions