SineFrame M3

Test MCP servers and the agents that call them in pytest

Developer ToolsArtificial IntelligenceGitHubOpen Source

▲ 0 votes1 commentsLaunched Oct 8, 2026

Weekly #148

Your MCP unit tests pass, but did the agent call the tool? M3 runs your server through real Claude Code, Codex, OpenCode or Pi and asserts on the calls they actually made, in plain pytest. Direct tests need no API key. m3 ui shows every trace and tool call; m3 ci test --upload gates PRs and keeps runs on app.m3.sineframe.com. Open source, Apache-2.0.

AI Analysis

📝 Summary

SineFrame M3 is an open-source (Apache-2.0) pytest-based tool for testing MCP servers and AI agents. It executes tests against real models like Claude, Codex, OpenCode or Pi to verify actual tool calls made, addressing the gap where unit tests pass but real agent behavior fails. Core features: no API key needed for direct tests, M3 UI for traces and tool call inspection, CI integration that uploads runs to app.m3.sineframe.com for PR gating. It solves key pain points in AI agent reliability and debugging, delivering confidence in tool integrations within a familiar testing framework.

📈 Market Timing

Favorable in 2025-2026 as AI agent and tool-calling adoption surges with maturing LLM capabilities. Developer demand for reliable testing beyond mocks is rising amid growing AI application complexity. Economic push for AI efficiency supports specialized dev tools. Excellent Timing.

✅ Feasibility

High. Builds on mature pytest and existing AI APIs with manageable technical complexity for a small dev team. Low operational costs as open-source core; hosted CI scalable via cloud. Minimal supply chain or compliance risks. Strong scalability potential for broader AI testing use cases.

🎯 Target Market

Primary segments: AI/ML engineers and developers building tool-calling agents, dev teams at tech firms using LLMs. Industries: artificial intelligence, software development. Geographic: global with concentration in US/Europe. TAM for AI dev tools ~$15B (2026), SAM for LLM testing ~$1B, SOM for this niche ~$50-100M. Pain points: unreliable agent-tool interactions. High willingness to pay for hosted CI/premium features.

⚔️ Competition

Low. Direct competitors: 1. Promptfoo (promptfoo.dev), 2. LangSmith (smith.langchain.com), 3. Helicone (helicone.ai), 4. Arize Phoenix (arize.com/phoenix). Advantages: unique pytest integration for real agent calls on MCP servers, no-API-key testing, open-source transparency. Disadvantages: newer with smaller ecosystem than LangSmith; less broad observability features.

Upgrade Pro to unlock full AI analysis