Cloudflare Analytics Engine Integration
Cloudflare Analytics Engine Integration
This document describes the Analytics Engine integration for tracking metrics and telemetry data in the bloqr-backend worker.
Overview
Cloudflare Analytics Engine provides high-cardinality, real-time analytics with SQL-like querying capabilities. The bloqr-backend uses Analytics Engine to track:
- API request metrics
- Compilation success/failure rates
- Cache hit/miss ratios
- Rate limiting events
- Workflow execution metrics
- Source fetch performance
Configuration
wrangler.toml Setup
The Analytics Engine binding is already configured in wrangler.toml:
[[analytics_engine_datasets]]binding = "ANALYTICS_ENGINE"dataset = "bloqr-backend-analytics"Environment Binding
The Env interface in worker/worker.ts includes the optional Analytics Engine binding:
interface Env { // ... other bindings ANALYTICS_ENGINE?: AnalyticsEngineDataset;}The binding is optional, allowing the worker to function without Analytics Engine configured (e.g., in development).
AnalyticsService
The AnalyticsService class (src/services/AnalyticsService.ts) provides a typed interface for tracking events.
Event Types
| Event Type | Description |
|---|---|
compilation_request | A compilation request was received |
compilation_success | Compilation completed successfully |
compilation_error | Compilation failed with an error |
cache_hit | Result served from cache |
cache_miss | Cache miss, compilation required |
rate_limit_exceeded | Client exceeded rate limit |
source_fetch | External source fetch completed |
workflow_started | Workflow execution started |
workflow_completed | Workflow completed successfully |
workflow_failed | Workflow failed with an error |
api_request | Generic API request tracking |
Data Model
Analytics Engine data points consist of:
- Index (1): Event type for efficient filtering
- Doubles (up to 20): Numeric metrics
- Blobs (up to 20): String metadata
Usage Example
import { AnalyticsService } from '../src/services/AnalyticsService.ts';
// Create service instanceconst analytics = new AnalyticsService(env.ANALYTICS_ENGINE);
// Track a compilation requestanalytics.trackCompilationRequest({ requestId: 'req-123', configName: 'EasyList', sourceCount: 3,});
// Track success with metricsanalytics.trackCompilationSuccess({ requestId: 'req-123', configName: 'EasyList', sourceCount: 3, ruleCount: 50000, durationMs: 1234, cacheKey: 'cache:abc123',});
// Track errorsanalytics.trackCompilationError({ requestId: 'req-123', configName: 'EasyList', sourceCount: 3, durationMs: 500, error: 'Source fetch failed',});Utility Methods
// Hash IP addresses for privacyconst ipHash = AnalyticsService.hashIp('192.168.1.1');
// Categorize user agentsconst category = AnalyticsService.categorizeUserAgent(userAgent);// Returns: 'adguard', 'ublock', 'browser', 'curl', 'bot', 'library', 'unknown'Tracked Locations
Analytics tracking is integrated into:
Worker Endpoints (worker/worker.ts)
- Rate limiting: Tracks when clients exceed rate limits
- Cache hits/misses: Tracks cache performance on
/compile/json - Compilation requests: Tracks all compilation attempts
- Compilation results: Tracks success/failure with metrics
Workflows
All workflows track execution metrics:
| Workflow | Events Tracked |
|---|---|
CompilationWorkflow | started, completed, failed |
BatchCompilationWorkflow | started, completed, failed |
CacheWarmingWorkflow | started, completed, failed |
HealthMonitoringWorkflow | started, completed, failed |
Querying Analytics Data
Use the Cloudflare dashboard or GraphQL API to query analytics:
Dashboard
- Go to Cloudflare Dashboard > Analytics & Logs > Analytics Engine
- Select the
bloqr-backend-analyticsdataset - Use SQL queries to analyze data
Example Queries
-- Compilation success rate over last 24 hoursSELECT blob1 as event_type, COUNT(*) as countFROM `bloqr-backend-analytics`WHERE timestamp > NOW() - INTERVAL '24' HOUR AND blob1 IN ('compilation_success', 'compilation_error')GROUP BY blob1
-- Average compilation duration by configSELECT blob2 as config_name, AVG(double1) as avg_duration_ms, COUNT(*) as total_compilationsFROM `bloqr-backend-analytics`WHERE timestamp > NOW() - INTERVAL '7' DAY AND blob1 = 'compilation_success'GROUP BY blob2ORDER BY total_compilations DESC
-- Cache hit ratioSELECT SUM(CASE WHEN blob1 = 'cache_hit' THEN 1 ELSE 0 END) as hits, SUM(CASE WHEN blob1 = 'cache_miss' THEN 1 ELSE 0 END) as misses, SUM(CASE WHEN blob1 = 'cache_hit' THEN 1 ELSE 0 END) * 100.0 / COUNT(*) as hit_rate_percentFROM `bloqr-backend-analytics`WHERE timestamp > NOW() - INTERVAL '24' HOUR AND blob1 IN ('cache_hit', 'cache_miss')
-- Rate limit events by IP hashSELECT blob3 as ip_hash, COUNT(*) as limit_eventsFROM `bloqr-backend-analytics`WHERE timestamp > NOW() - INTERVAL '1' HOUR AND blob1 = 'rate_limit_exceeded'GROUP BY blob3ORDER BY limit_events DESCLIMIT 10Graceful Degradation
The AnalyticsService gracefully handles missing Analytics Engine:
constructor(dataset?: AnalyticsEngineDataset) { this.dataset = dataset; this.enabled = !!dataset;}
private writeDataPoint(event: AnalyticsEventData): void { if (!this.enabled || !this.dataset) { return; // Silently skip when not configured } // ... write data point}This ensures:
- Local development works without Analytics Engine
- No errors if binding is missing
- Easy toggle for analytics collection
Data Retention
Analytics Engine data is retained according to your Cloudflare plan:
- Free: 31 days
- Pro: 90 days
- Business: 1 year
- Enterprise: Custom
Privacy Considerations
The implementation includes privacy-conscious practices:
- IP Hashing: Client IPs are hashed before storage
- No PII: No personal identifiable information is stored
- User Agent Categorization: User agents are categorized rather than stored raw
- Request ID Tracking: Uses generated request IDs rather than user identifiers
Extending Analytics
To add new event tracking:
- Add a new event type to
AnalyticsEventType:
export type AnalyticsEventType = | 'compilation_request' // ... existing types | 'your_new_event';- Create a data interface if needed:
export interface YourEventData { requestId: string; // ... fields}- Add a tracking method to
AnalyticsService:
public trackYourEvent(data: YourEventData): void { this.writeDataPoint({ eventType: 'your_new_event', timestamp: Date.now(), doubles: [data.someNumber], blobs: [data.requestId, data.someString], });}- Call the tracking method where appropriate in the codebase.
Troubleshooting
Analytics Not Recording
- Verify the binding exists in
wrangler.toml - Check the dataset name matches
- Ensure
ANALYTICS_ENGINEis in yourEnvinterface - Check Cloudflare dashboard for the dataset
Query Returns No Results
- Verify the time range includes recent data
- Check event type names match exactly
- Ensure data is being written (check worker logs)
High Cardinality Warnings
If you see cardinality warnings:
- Avoid using raw IPs or unique identifiers in indexes
- Use categorical values in blob fields
- Consider aggregating data before writing