OpenAPI Tooling Guide

Complete guide to validating, testing, and documenting the Bloqr Compiler API using the OpenAPI specification.

📋 Table of Contents

Overview
Validation
Documentation Generation
Contract Testing
Postman Testing
CI/CD Integration
Best Practices

Overview

The Bloqr Compiler API is fully documented using the OpenAPI 3.0.3 specification (docs/api/openapi.yaml). This specification serves as the single source of truth for:

API endpoint definitions
Request/response schemas
Authentication requirements
Error responses
Examples and documentation

Validation

Validate OpenAPI Spec

Ensure your docs/api/openapi.yaml conforms to the OpenAPI specification:

# Run validation
deno task openapi:validate

# Or directly
./scripts/validate-openapi.ts

What it checks:

✅ YAML syntax
✅ OpenAPI version compatibility
✅ Required fields (info, paths, etc.)
✅ Unique operation IDs
✅ Response definitions
✅ Schema completeness
✅ Best practices compliance

Example output:

🔍 Validating OpenAPI specification...

✅ YAML syntax is valid
✅ OpenAPI version: 3.0.3
✅ Title: Bloqr Compiler API
✅ Version: 2.0.0
✅ Servers: 2 defined
✅ Paths: 10 endpoints defined
✅ Operations: 13 total
✅ Schemas: 30 defined
✅ Security schemes: 1 defined
✅ Tags: 5 defined

📋 Checking best practices...

✅ Request examples: 2 found
✅ Contact info provided
✅ License: MIT

============================================================
VALIDATION RESULTS
============================================================

✅ OpenAPI specification is VALID!

Summary: 0 errors, 0 warnings

Pre-commit Validation

Add to your git hooks:

#!/bin/sh
deno task openapi:validate || exit 1

Documentation Generation

Generate HTML Documentation

Create beautiful, interactive API documentation using Redoc:

# Generate docs
deno task openapi:docs

# Or directly
./scripts/generate-docs.ts

Output files:

docs/api/index.html - Interactive HTML documentation (Redoc)
docs/api/README.md - Markdown reference documentation

Generate Cloudflare API Shield Schema

Generate a Cloudflare-compatible schema for use with Cloudflare’s API Shield Schema Validation:

# Generate Cloudflare schema
deno task schema:cloudflare

# Or directly
./scripts/generate-cloudflare-schema.ts

What it does:

✅ Filters out localhost servers (keeps only production/staging URLs)
✅ Removes non-standard x-* extension fields from operations
✅ Generates docs/api/cloudflare-schema.yaml ready for API Shield

Why use this: Cloudflare’s API Shield Schema Validation provides request/response validation at the edge. The generated schema is optimized for Cloudflare’s parser by removing development servers and custom ex[…]

Learn more: Cloudflare API Shield Schema Validation

CI/CD Integration: The schema generation is validated in CI to ensure it stays in sync with the main OpenAPI spec. If you update docs/api/openapi.yaml, you must regenerate the Cloudflare schema by running deno task schema:cloudflare and committing the result.

View Documentation

# Open HTML docs
open docs/api/index.html

# Or serve locally
python3 -m http.server 8000 --directory docs/api
# Then visit http://localhost:8000

Features

The generated HTML documentation includes:

🔍 Search functionality - Find endpoints quickly
📱 Responsive design - Works on mobile/tablet/desktop
🎨 Code samples - Request/response examples
📊 Schema explorer - Interactive schema browser
🔗 Deep linking - Share links to specific endpoints
📥 Download spec - Export OpenAPI YAML/JSON

Customization

Edit scripts/generate-docs.ts to customize:

Theme colors
Logo/branding
Sidebar configuration
Code sample languages

Contract Testing

Contract tests validate that your live API conforms to the OpenAPI specification.

Run Contract Tests

# Test against local server (default)
deno task test:contract

# Test against production
API_BASE_URL=https://bloqr-backend.jk-com.workers.dev deno task test:contract

# Test specific scenarios
deno test --allow-read --allow-write --allow-net --allow-env worker/openapi-contract.test.ts --filter "Contract: GET /api"

What’s Tested

Core Endpoints:

✅ GET /api - API info
✅ GET /metrics - Performance metrics
✅ POST /compile - Synchronous compilation
✅ POST /compile/stream - SSE streaming
✅ POST /compile/batch - Batch processing

Async Queue Operations (Cloudflare Queues):

✅ POST /compile/async - Queue async job
✅ POST /compile/batch/async - Queue batch job
✅ GET /queue/stats - Queue statistics
✅ GET /queue/results/{id} - Retrieve job results

Contract Validation:

✅ Response status codes match spec
✅ Response content types are correct
✅ Required fields are present
✅ Data types match schemas
✅ Headers conform to spec (X-Cache, X-Request-Deduplication)
✅ Error responses have proper structure

Async Testing with Queues

The contract tests properly validate Cloudflare Queue integration:

// Queues async compilation
const response = await apiRequest('/compile/async', {
    method: 'POST',
    body: JSON.stringify({ configuration, preFetchedContent }),
});

// Returns 202 if queues available, 500 if not configured
validateResponseStatus(response, [202, 500]);

if (response.status === 202) {
    const data = await response.json();
    // Validates requestId is returned
    validateBasicSchema(data, ['success', 'requestId', 'message']);
}

Queue Test Scenarios

Standard Priority Queue
- Tests default queue behavior
- Validates requestId generation
- Confirms job queuing
High Priority Queue
- Tests priority routing
- Validates faster processing (when implemented)
Batch Queue Operations
- Tests multiple jobs queued together
- Validates batch requestId tracking
Queue Statistics
- Validates queue depth metrics
- Confirms job status tracking
- Tests history retention

CI/CD Contract Testing

name: Contract Tests

on: [push, pull_request]

jobs:
  contract-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: denoland/setup-deno@v1
        with:
          deno-version: v2.x

      - name: Start local server
        run: deno task dev &

      - name: Wait for server
        run: sleep 5

      - name: Run contract tests
        run: deno task test:contract

Postman Testing

See POSTMAN_TESTING.md for complete Postman documentation.

Generate / Regenerate the Postman Collection

The Postman collection and environment files are auto-generated from docs/api/openapi.yaml. Do not edit them directly.

# Regenerate from the canonical OpenAPI spec
deno task postman:collection

This creates / updates:

docs/postman/postman-collection.json — all API requests with automated test assertions
docs/postman/postman-environment.json — local and production environment variables

The CI validate-postman-collection job regenerates the files and fails the build if the committed copies are out of sync with docs/api/openapi.yaml. Always run deno task postman:collection after editing the spec.

Schema Hierarchy

docs/api/openapi.yaml                 ← canonical source of truth (edit this)
docs/api/cloudflare-schema.yaml       ← auto-generated (deno task schema:cloudflare)
docs/postman/postman-collection.json  ← auto-generated (deno task postman:collection)
docs/postman/postman-environment.json ← auto-generated (deno task postman:collection)

Quick Start

# Import collection and environment into Postman
# - docs/postman/postman-collection.json
# - docs/postman/postman-environment.json

# Or use Newman CLI
npm install -g newman
newman run docs/postman/postman-collection.json -e docs/postman/postman-environment.json

Postman Features

🧪 25+ test requests
✅ Automated assertions
📊 Response validation
🔄 Dynamic variables
📈 Performance testing

CI/CD Integration

GitHub Actions

Complete pipeline for validation, testing, and documentation:

name: OpenAPI Pipeline

on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: denoland/setup-deno@v1

      - name: Validate OpenAPI spec
        run: deno task openapi:validate

  validate-cloudflare-schema:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: denoland/setup-deno@v1

      - name: Generate Cloudflare schema
        run: deno task schema:cloudflare

      - name: Check schema is up to date
        run: |
          if ! git diff --quiet docs/api/cloudflare-schema.yaml; then
            echo "❌ Cloudflare schema is out of date!"
            echo "Run 'deno task schema:cloudflare' and commit the result."
            exit 1
          fi

  generate-docs:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: denoland/setup-deno@v1

      - name: Generate documentation
        run: deno task openapi:docs

      - name: Upload docs
        uses: actions/upload-artifact@v3
        with:
          name: api-docs
          path: docs/api/

  contract-tests:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: denoland/setup-deno@v1

      - name: Start server
        run: deno task dev &

      - name: Wait for server
        run: sleep 10

      - name: Run contract tests
        run: deno task test:contract

  postman-tests:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Start server
        run: docker compose up -d

      - name: Install Newman
        run: npm install -g newman

      - name: Run Postman tests
        run: newman run docs/postman/postman-collection.json -e docs/postman/postman-environment.json --reporters cli,json

      - name: Upload results
        uses: actions/upload-artifact@v3
        with:
          name: newman-results
          path: newman/

Pre-deployment Checks

#!/bin/bash
echo "🔍 Validating OpenAPI spec..."
deno task openapi:validate || exit 1

echo "☁️  Generating Cloudflare schema..."
deno task schema:cloudflare || exit 1

echo "📚 Generating documentation..."
deno task openapi:docs || exit 1

echo "🧪 Running contract tests..."
deno task test:contract || exit 1

echo "✅ All checks passed! Ready to deploy."

Best Practices

1. Keep Spec and Code in Sync

Problem: Spec drifts from actual implementation

Solution:

Run contract tests on every PR
Use CI/CD to block deployment if tests fail
Review OpenAPI changes alongside code changes

# Add to .git/hooks/pre-push
deno task openapi:validate
deno task test:contract

2. Version Your API

Current version: 2.0.0 in docs/api/openapi.yaml

When making breaking changes:

Increment major version (2.0.0 → 3.0.0)
Update info.version in docs/api/openapi.yaml
Document changes in CHANGELOG.md
Consider API versioning in URLs

3. Document Examples

Good:

requestBody:
  content:
    application/json:
      schema:
        $ref: '#/components/schemas/CompileRequest'
      examples:
        simple:
          summary: Simple compilation
          value:
            configuration:
              name: My Filter List
              sources:
                - source: test-rules

Why: Examples improve documentation and serve as test data.

4. Use Async Queues Appropriately

When to use Cloudflare Queues:

✅ Use queues for:

Long-running compilations (>5 seconds)
Large batch operations
Background processing
Rate limit avoidance
Retry-able operations

❌ Don’t use queues for:

Quick operations (<1 second)
Real-time user interactions
Operations needing immediate feedback

Implementation:

// Queue job
const requestId = await queueCompileJob(env, configuration, preFetchedContent);

// Return immediately
return Response.json({
    success: true,
    requestId,
    message: 'Job queued for processing'
}, { status: 202 });

// Client polls for results
GET /queue/results/{requestId}

5. Test Queue Scenarios

Always test queue operations:

# Test queue availability
deno test --filter "Contract: POST /compile/async"

# Test queue stats
deno test --filter "Contract: GET /queue/stats"

# Test result retrieval
deno test --filter "Contract: GET /queue/results"

6. Monitor Queue Health

Track queue metrics:

Queue depth (pending jobs)
Processing rate (jobs/minute)
Average processing time
Failure rate
Retry rate

Access via: GET /queue/stats

7. Handle Queue Unavailability

Queues may not be configured in all environments:

if (!env.BLOQR_BACKEND_QUEUE) {
    return Response.json({
        success: false,
        error: 'Queue not available. Use synchronous endpoints instead.'
    }, { status: 500 });
}

Contract tests handle this gracefully:

validateResponseStatus(response, [202, 500]); // Both OK

Troubleshooting

Validation Fails

❌ Missing "operationId" for POST /compile

Fix: Add unique operationId to all operations in docs/api/openapi.yaml

Contract Tests Fail

Expected status 200, got 500

Fix:

Check server logs
Verify request body matches schema
Ensure queue bindings configured (for async endpoints)

Documentation Not Generating

Failed to parse YAML

Fix: Validate YAML syntax:

deno task openapi:validate

Queue Tests Always Return 500

Cause: Cloudflare Queues not configured locally

Expected: Queues are production-only. Tests accept 202 OR 500.

Fix: Deploy to Cloudflare Workers to test queue functionality.

Resources

Summary

The OpenAPI tooling provides:

Validation - Ensure spec quality (openapi:validate)
Documentation - Generate beautiful docs (openapi:docs)
Cloudflare Schema - Generate API Shield schema (schema:cloudflare)
Postman Collection - Regenerate from spec (postman:collection)
Contract Tests - Verify API compliance (test:contract)
Queue Support - Async operations via Cloudflare Queues

Schema Hierarchy

docs/api/openapi.yaml                 ← canonical source of truth (edit this)
docs/api/cloudflare-schema.yaml       ← auto-generated (deno task schema:cloudflare)
docs/postman/postman-collection.json  ← auto-generated (deno task postman:collection)
docs/postman/postman-environment.json ← auto-generated (deno task postman:collection)

All tools are designed to work together in a continuous integration pipeline, ensuring your API stays consistent, well-documented, and reliable.