Cost Tracking

Track token usage and costs per project or environment to manage budget effectively and make data-driven decisions about AI system optimization.

Effective cost management is essential for sustainable AI operations. By monitoring spending patterns, setting appropriate budgets, and analyzing cost alongside quality metrics, teams can optimize their AI systems for both performance and efficiency.

Evaligo's cost tracking provides detailed visibility into token usage, request volumes, and associated costs across different models, projects, and environments. This granular tracking enables teams to identify optimization opportunities and prevent budget overruns.

Budget Management

Define monthly budgets and alerts to prevent runaway costs. Set thresholds at the project and environment level to catch anomalies early and maintain financial discipline across your organization.

1. 1

   Set project budgets Define monthly spending limits for each project and environment to control costs.

2. 2

   Configure alerts Set up notifications when spending reaches 75%, 90%, and 100% of budget thresholds.

3. 3

   Review spending patterns Analyze historical trends to set realistic budgets and identify seasonal variations.

4. 4

   Implement cost controls Set up automatic rate limiting or circuit breakers when budgets are exceeded.

# Configure budget and alerts via API
from evaligo import Client

client = Client()

# Set monthly budget for production environment
budget = client.budgets.create(
    project_id="prod-chatbot",
    environment="production",
    monthly_limit_usd=5000,
    alerts=[
        {"threshold": 0.75, "channels": ["email", "slack"]},
        {"threshold": 0.90, "channels": ["email", "slack", "pagerduty"]},
        {"threshold": 1.0, "channels": ["email", "slack", "pagerduty"]}
    ]
)

# Track costs for specific features
client.traces.create(
    span_id="chat-response-123",
    metadata={
        "feature": "customer-support",
        "team": "product",
        "cost_center": "engineering"
    },
    model_usage={
        "model": "gpt-4",
        "input_tokens": 150,
        "output_tokens": 75,
        "cost_usd": 0.0045
    }
)

Cost Attribution

Attribute spend to features or teams using metadata tags. This enables detailed cost allocation and helps identify which parts of your system are driving the highest costs, allowing for targeted optimization efforts.

Use consistent tagging strategies across your organization to enable accurate cost attribution. Tags can include team ownership, feature flags, customer segments, or any other relevant business dimensions.

Cost Optimization

Analyze cost alongside quality and latency to make balanced tradeoffs. Identify opportunities to reduce spending without compromising user experience or system reliability.

# Analyze cost vs quality tradeoffs
cost_analysis = client.analytics.cost_quality_analysis(
    project_id="chatbot-v2",
    time_range="last_30_days",
    group_by=["model", "feature", "environment"]
)

# Identify optimization opportunities
for group in cost_analysis.groups:
    if group.cost_per_request > 0.05 and group.quality_score < 0.85:
        print(f"Optimization opportunity: {group.name}")
        print(f"  Cost per request: ${group.cost_per_request:.4f}")
        print(f"  Quality score: {group.quality_score:.2f}")
        print(f"  Suggestion: Consider model downgrade or prompt optimization")

# Export detailed cost report
report = client.reports.generate(
    type="cost_analysis",
    project_id="chatbot-v2",
    include_sections=["summary", "trends", "attribution", "recommendations"],
    format="csv"
)

# Share with stakeholders
report.share_with(["finance@company.com", "engineering-leads@company.com"])

Model Selection Impact

Different models have significantly different cost profiles. Regularly evaluate whether you're using the most cost-effective model for each use case, and consider implementing dynamic model selection based on request complexity.

Video

Cost Optimization Strategies

Learn techniques for reducing AI costs while maintaining quality

4m 20s

Reporting and Integration

Export cost reports to share with stakeholders or integrate with finance tooling. Regular reporting helps maintain organizational alignment on AI spending and supports budget planning processes.