API Response Validation: Beyond Status Code Checks

A 200 OK response code means the server understood your request and responded. It doesn't mean the response contains what your application expects. APIs can return 200 with empty arrays, null fields, wrong data types, missing required properties, or entirely different data structures after a breaking change. Without response validation, your monitoring gives you a false sense of security.

The Gap Between Uptime and Correctness

Consider what can go wrong while an API still returns 200:

• A database returns 0 rows instead of the expected results
• A field is null when downstream code expects a string
• A JSON object has extra fields indicating an error state
• A list is empty when it should have items
• A numeric field overflows and becomes null
• A nested object is missing expected child fields

None of these trigger a non-200 status code. All of them can break your application. Response validation bridges the gap between "the server is up" and "the API is working correctly."

JSON Schema Validation

JSON Schema is the most rigorous approach to response validation. Define the expected schema and validate every response against it:

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "title": "User API Response",
  "type": "object",
  "required": ["id", "email", "status", "created_at"],
  "properties": {
    "id": {
      "type": "string",
      "pattern": "^user_[a-z0-9]{16}$"
    },
    "email": {
      "type": "string",
      "format": "email"
    },
    "status": {
      "type": "string",
      "enum": ["active", "inactive", "pending"]
    },
    "created_at": {
      "type": "string",
      "format": "date-time"
    },
    "profile": {
      "type": "object",
      "properties": {
        "name": { "type": "string", "minLength": 1 },
        "avatar_url": { "type": "string", "format": "uri" }
      }
    }
  },
  "additionalProperties": false
}

Validate responses programmatically:

import jsonschema
import requests
import json

def validate_api_response(url, headers, expected_schema):
    """
    Fetch API response and validate against JSON schema.
    Returns validation result with details.
    """
    response = requests.get(url, headers=headers, timeout=5)
    
    if response.status_code != 200:
        return {
            "valid": False,
            "error": f"Unexpected status code: {response.status_code}"
        }
    
    try:
        data = response.json()
    except json.JSONDecodeError as e:
        return {
            "valid": False,
            "error": f"Invalid JSON: {e}"
        }
    
    validator = jsonschema.Draft7Validator(expected_schema)
    errors = list(validator.iter_errors(data))
    
    if errors:
        return {
            "valid": False,
            "errors": [
                {
                    "path": ".".join(str(p) for p in err.path),
                    "message": err.message,
                    "invalid_value": err.instance
                }
                for err in errors
            ]
        }
    
    return {"valid": True, "data": data}

# Usage
schema = json.load(open("schemas/user-response.json"))
result = validate_api_response(
    "https://api.example.com/users/test-user",
    {"Authorization": "Bearer test-token"},
    schema
)

if not result["valid"]:
    print("Schema validation failed:")
    for error in result.get("errors", []):
        print(f"  {error['path']}: {error['message']}")

JSONPath Assertions

For monitoring tools, JSONPath expressions let you write targeted assertions without full schema validation:

# Example monitoring check with JSONPath assertions
monitor:
  name: "Product Catalog API"
  url: "https://api.example.com/products?category=electronics"
  assertions:
    # Status check
    - path: "status_code"
      equals: 200
    
    # Top-level field exists and has expected type
    - path: "$.products"
      operator: is_array
    
    # Array is not empty
    - path: "$.products.length()"
      operator: greater_than
      value: 0
    
    # Specific field in first item
    - path: "$.products[0].id"
      operator: matches_regex
      value: "^prod_[0-9a-z]{8}$"
    
    # Nested field exists
    - path: "$.products[0].pricing.amount"
      operator: is_number
    
    # Metadata field
    - path: "$.pagination.total"
      operator: greater_than
      value: 0
    
    # Error field should NOT exist
    - path: "$.error"
      operator: not_exists

Business Logic Validation

Beyond structure validation, some assertions need to verify business logic:

// Custom validation logic
function validateCheckoutResponse(response) {
  const body = response.body;
  const assertions = [];
  
  // Business rule: If payment status is "captured", charge must have amount
  if (body.payment?.status === "captured") {
    assertions.push({
      test: body.payment.amount > 0,
      message: "Captured payment must have positive amount"
    });
    assertions.push({
      test: body.payment.transaction_id?.startsWith("txn_"),
      message: "Captured payment must have valid transaction ID"
    });
  }
  
  // Business rule: Tax must be calculated correctly
  if (body.order?.subtotal && body.order?.tax) {
    const expectedTax = body.order.subtotal * 0.1; // 10% tax
    const taxDifference = Math.abs(body.order.tax - expectedTax);
    assertions.push({
      test: taxDifference < 0.01,
      message: `Tax ${body.order.tax} doesn't match expected ${expectedTax}`
    });
  }
  
  // Business rule: Total = subtotal + tax + shipping
  if (body.order?.total && body.order?.subtotal) {
    const expectedTotal = body.order.subtotal + 
                         (body.order.tax || 0) + 
                         (body.order.shipping || 0);
    assertions.push({
      test: Math.abs(body.order.total - expectedTotal) < 0.01,
      message: "Order total doesn't match component sum"
    });
  }
  
  const failures = assertions.filter(a => !a.test);
  return {
    valid: failures.length === 0,
    failures: failures.map(f => f.message)
  };
}

Response Time Distribution Validation

Validate not just that a response is fast, but that the distribution is healthy:

import statistics

class ResponseTimeValidator:
    def __init__(self, samples=100):
        self.samples = []
        self.max_samples = samples
    
    def record(self, response_time_ms):
        self.samples.append(response_time_ms)
        if len(self.samples) > self.max_samples:
            self.samples.pop(0)
    
    def validate(self):
        if len(self.samples) < 10:
            return {"valid": True, "reason": "Insufficient samples"}
        
        p50 = statistics.median(self.samples)
        p95 = sorted(self.samples)[int(len(self.samples) * 0.95)]
        p99 = sorted(self.samples)[int(len(self.samples) * 0.99)]
        
        issues = []
        
        if p50 > 200:
            issues.append(f"Median latency {p50}ms exceeds 200ms threshold")
        
        if p95 > 800:
            issues.append(f"p95 latency {p95}ms exceeds 800ms threshold")
        
        if p99 > 2000:
            issues.append(f"p99 latency {p99}ms exceeds 2000ms threshold")
        
        # Check for bimodal distribution (indicates flakiness)
        if p99 / p50 > 20:
            issues.append(
                f"High p99/p50 ratio ({p99/p50:.1f}x) suggests flaky responses"
            )
        
        return {
            "valid": len(issues) == 0,
            "p50": p50,
            "p95": p95,
            "p99": p99,
            "issues": issues
        }

Content-Type and Encoding Validation

APIs sometimes return wrong content types after changes:

| Expected | Actual | Problem | |---|---|---| | application/json | text/html | Error page returned instead of API response | | application/json | application/json; charset=utf-8 | Usually fine, but worth tracking | | application/json | application/octet-stream | Binary response where JSON expected | | Gzip encoded | No encoding | Performance regression |

def validate_content_type(response, expected_content_type):
    """Validate response content type matches expectation"""
    actual = response.headers.get('Content-Type', '')
    
    # Normalize: strip parameters for comparison
    actual_base = actual.split(';')[0].strip().lower()
    expected_base = expected_content_type.split(';')[0].strip().lower()
    
    if actual_base != expected_base:
        return {
            "valid": False,
            "error": f"Expected Content-Type '{expected_base}' but got '{actual_base}'"
        }
    
    # Check encoding if expected
    if 'charset' in expected_content_type:
        expected_charset = expected_content_type.split('charset=')[1].strip().lower()
        actual_charset = actual.split('charset=')[1].strip().lower() if 'charset=' in actual else None
        
        if actual_charset != expected_charset:
            return {
                "valid": False,
                "error": f"Expected charset '{expected_charset}' but got '{actual_charset}'"
            }
    
    return {"valid": True}

Handling API Versioning in Validation

As APIs evolve, validation schemas must track versioning:

SCHEMAS = {
    "v1": {
        "user": {
            "required": ["id", "email"],
            "properties": {
                "id": {"type": "integer"},
                "email": {"type": "string"}
            }
        }
    },
    "v2": {
        "user": {
            "required": ["id", "email", "created_at"],
            "properties": {
                "id": {"type": "string"},  # Changed from int to string
                "email": {"type": "string"},
                "created_at": {"type": "string", "format": "date-time"}
            }
        }
    }
}

def get_schema(endpoint, version="v2"):
    """Get appropriate schema for endpoint version"""
    return SCHEMAS.get(version, {}).get(endpoint)

When an API version is deprecated, add deprecation detection to monitoring:

# Check for deprecation warning headers
monitor:
  url: "https://api.example.com/v1/users"
  assertions:
    - header: "Deprecation"
      operator: not_exists
      message: "v1 endpoint is returning deprecation header"
    - header: "Sunset"
      operator: not_exists
      message: "v1 endpoint has sunset date set"

Automated Schema Discovery

If you don't have schemas defined, tools can discover them from actual responses:

# Generate JSON Schema from sample responses
# Install: npm install -g @apidevtools/json-schema-faker

# Capture sample responses
curl -s https://api.example.com/users/1 > sample-user.json

# Generate schema from sample
npx json-schema-to-typescript sample-user.json > user-schema.ts

# Or use quicktype for multiple samples
quicktype --src-lang json --lang json-schema \
  sample1.json sample2.json sample3.json \
  -o user-schema.json

Run schema discovery periodically and alert when schemas change unexpectedly — it's a proxy for detecting breaking API changes.

Conclusion

Response validation is the difference between knowing your API is reachable and knowing your API is working. Status code checks are necessary but far from sufficient. By validating response structure with JSON Schema, business logic with custom assertions, and content types with header checks, you build monitoring that actually catches functional regressions. AzMonitor supports custom response assertions that let you go beyond status codes to validate the content and correctness of every API response in your monitoring suite.