feat: Add server auth conformance tests

[pcarleton]

Summary

Adds conformance testing for MCP server OAuth authentication implementations.

Design Philosophy

The MCP authorization spec separates the Authorization Server (AS) from the Resource Server (RS/MCP server). This reflects the reality that authorization is often better handled by dedicated tools (Auth0, Keycloak, better-auth, etc.) rather than reimplemented in every SDK.

This conformance suite tests MCP server (RS) implementations, not AS implementations. SDKs are not expected to provide a full AS - they just need to:

1. Return 401 + WWW-Authenticate for unauthenticated requests
2. Serve Protected Resource Metadata (PRM) pointing to an AS
3. Validate Bearer tokens (via introspection or other means)

The conformance suite provides a fake AS for testing, so SDK authors can verify their RS implementation without setting up real auth infrastructure. Alternatively, servers can integrate with their own AS if they prefer.

What's New

CLI Commands

# Run auth tests against an MCP server
# The AS is spun up and URL passed via MCP_CONFORMANCE_AUTH_SERVER_URL environment variable. 
conformance server --suite auth --command 'npx path/to/test-server.ts'

# Run auth tests against an MCP server (with server providing AS)
conformance server --suite auth --url http://localhost:3000/mcp

# Interactive mode for servers requiring browser-based login
conformance server --suite auth --url http://localhost:3000/mcp --interactive

# Standalone fake AS for manual testing
conformance fake-auth-server [--port <port>]

New Files

• src/fake-auth-server.ts - Standalone fake OAuth AS CLI
• src/scenarios/server-auth/ - Server auth scenario directory
  • basic-dcr-flow.ts - Tests complete OAuth flow (401 → PRM → AS metadata → CIMD/DCR → Token → Auth'd request)
  • helpers/oauth-client.ts - Observation middleware and ConformanceOAuthProvider
  • spec-references.ts - RFC/spec references (2025-11-25)

Modified Files

• src/index.ts - Extended server command with --suite auth, --interactive
• src/runner/server.ts - Added runServerAuthConformanceTest() and startFakeAuthServer()
• src/scenarios/client/auth/helpers/createAuthServer.ts - Added /introspect endpoint (RFC 7662), CIMD detection
• src/types.ts - Added ClientScenarioOptions with interactive flag
• package.json - Added fake-auth-server bin entry

Interactive Mode

For servers that integrate with their own AS requiring browser-based login, use --interactive:

conformance server --suite auth --url http://localhost:3000/mcp --interactive

This starts a callback server on port 3333 and displays the authorization URL for you to open in a browser.

CIMD vs DCR

The fake AS defaults to client_id_metadata_document_supported: true, making CIMD (URL-based client IDs) the preferred method over DCR.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Conformance Suite                             │
│                                                                  │
│  ┌──────────────┐     ┌──────────────────────────────────────┐  │
│  │   Fake AS    │     │  Checker Client (typescript-sdk)      │  │
│  │  (provided)  │◄────│  - Observes HTTP requests             │  │
│  │              │     │  - Generates conformance checks       │  │
│  └──────────────┘     └──────────┬───────────────────────────┘  │
│        ▲                         │                               │
│        │ PRM points here         │ Auth'd MCP requests          │
│        │ (or to own AS)          │                               │
│        │                         ▼                               │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │              MCP Server Under Test (RS)                   │   │
│  │  - Returns 401 + WWW-Authenticate for unauth requests    │   │
│  │  - Serves PRM at /.well-known/oauth-protected-resource   │   │
│  │  - Validates Bearer tokens                                │   │
│  └──────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Conformance Checks

• auth-401-response - Server returns 401 for unauthenticated requests
• auth-www-authenticate-header - 401 includes WWW-Authenticate header with Bearer scheme
• auth-resource-metadata-param - WWW-Authenticate includes resource_metadata parameter
• auth-prm-discovery - PRM endpoint responds at .well-known/oauth-protected-resource
• auth-prm-authorization-servers - PRM contains authorization_servers array
• auth-as-metadata-discovery - AS metadata discovery works
• auth-as-metadata-fields - AS metadata includes required fields (issuer, authorization_endpoint, token_endpoint, PKCE S256)
• auth-dcr-registration - Dynamic Client Registration succeeds (when DCR used)
• auth-dcr-response - DCR response contains client credentials
• auth-token-request - Token acquisition succeeds
• auth-token-response - Token response contains access_token
• auth-authenticated-request - Authenticated MCP call succeeds
• auth-flow-completion - Complete OAuth flow succeeded

Testing Example

# Start an MCP server with OAuth support
cd ~/code/mcp/typescript-sdk
pnpm exec tsx examples/server/src/simpleStreamableHttp.ts --oauth

# Run conformance tests (uses --interactive since example uses better-auth)
conformance server --suite auth --url http://localhost:3000/mcp --interactive

Related PRs

• feat: add auth-test-server for OAuth conformance testing typescript-sdk#1384 - Adds auth-test-server.ts to typescript-sdk conformance directory

[pkg-pr-new]

Open in StackBlitz

npx https://pkg.pr.new/modelcontextprotocol/conformance/@modelcontextprotocol/conformance@105

commit: c34a019

[tobinsouth]

I think the fake AS is a great solution here and focusing on the RS is the right design choice (better than #64 !). If we do a AS specific conformance check, I think it should be separate tooling. Maybe something someone at Okta / Entra / Auth0 / Twillio / Arcade / WorkOS etc want to take on.

[maciej-kisiel]

Hi @pcarleton, I have some questions/thoughts about the server auth tests:

• What is the planned semantics of the suite flag? Will the auth suite ever be merged into active or all?
• Is the --command mode planned to be exposed also to other server conformance tests? I wonder what the recommended default is, I believe initially the server tests required the server to be started separately (as still documented in docs: add SDK integration guide and fix spec references #121), but the --command mode is really handy as it automatically starts the authorization server.
• How much do you want to pursue the "single command / single report" behavior you mentioned in examples: adjust conformance server for auth tests go-sdk#759? As you can see in the PR, our conformance script gets more complicated when we have two modes, so there would be some value in reaching a "single command" state. It could also be beneficial in the context of the GitHub Action you're providing to run the tests. As for the "single report", it might not be a big deal depending on what the tiering monitoring (more on that below) would use to calculate the score – I assume it would consume the result files. In that case running a few commands pointing to the same directory should produce full results.
• Speaking of results: I tried reading them to understand what's happening, but the fact that they are not ordered chronologically as things happen makes it difficult – there is a batch of checks done at the end of the suite that inspects the requests that were made. This is especially noticeable in the OAuth area, where the flows are quite complex. I wonder if it would be possible to reorder them to make the situation better (for example use the timestamp of the requests in each check instead of "now" and sort before saving).

More generally, is the set of conformance tests that will be applicable to the first tiering publishing identified already? Have they all been merged? Is there already a vision of how the conformance requirement for tiers will be monitored/enforced? Will there be some central system, or it will be each SDK maintainer's to set it up and monitor it? Sorry if this information is available somewhere, I started working on the SDK quite recently and may have missed some context.