Registry ↔ UI Integration

← Back to API docs

This document describes the design and implementation plan for wiring the HostlistsRegistry and FiltersRegistry public indexes into the Web UI as a first-class list picker, eliminating the need for users to supply raw source URLs manually.

Background

Both upstream registries publish machine-readable JSON indexes that are ideal for populating a UI picker:

Registry	Index URL	Key Fields
HostlistsRegistry (DNS-level)	`https://adguardteam.github.io/HostlistsRegistry/assets/filters.json`	`filterKey`, `name`, `downloadUrl`, `description`
FiltersRegistry (browser-level)	`https://filters.adtidy.org/extension/chromium/filters.json`	`filterId`, `name`, `downloadUrl`, `groupId`
ServicesRegistry (service blocks)	`https://adguardteam.github.io/HostlistsRegistry/assets/services.json`	`id`, `name`, `rules[]`

Today, the CompileRequest schema requires the caller to supply raw source URLs or pre-fetched content.
No endpoint exists to surface these registries to the UI — that is what this document specifies.

Goals

UI list picker — the Web UI can present a browsable, searchable catalog of well-known filter lists without the user needing to know any URLs.
Symbolic source URIs — CompileRequest sources may reference lists by key (e.g. hostlist-registry://adguard_dns_filter) instead of raw URLs. The Worker resolves the key at compile time.
Edge caching — registry index fetches are cached at the Worker layer (KV or in-memory with TTL) so the UI never directly depends on the upstream CDN.

Architecture

flowchart TD
    subgraph UI["Web UI"]
        PICKER["Registry Picker\n(search + group filter)\n← GET /registry/hostlists\n← GET /registry/filters\n→ injects source: hostlist-registry://..."]
    end

    subgraph WORKER["Cloudflare Worker"]
        HOSTLIST["GET /registry/hostlists → HostlistRegistryProxy"]
        FILTERS["GET /registry/filters → FiltersRegistryProxy"]
        SERVICES["GET /registry/services → ServicesRegistryProxy"]
        COMPILE["POST /compile\n  └─ HostlistRegistryFetcher (resolves URIs)\n  └─ FiltersRegistryFetcher (resolves URIs)"]
    end

    PICKER -->|POST /compile| COMPILE

New Endpoints

`GET /registry/hostlists`

Returns the HostlistsRegistry index, proxied and cached by the Worker.

Response 200 application/json:

{
  "timestamp": "2026-03-10T00:00:00Z",
  "ttl": 3600,
  "lists": [
    {
      "filterKey": "adguard_dns_filter",
      "name": "AdGuard DNS Filter",
      "description": "AdGuard DNS filter is a combination of several filters...",
      "downloadUrl": "https://adguardteam.github.io/HostlistsRegistry/assets/filter_1.txt",
      "tags": ["adguard", "dns"],
      "homepage": "https://github.com/AdguardTeam/AdguardSDNSFilter"
    }
  ]
}

`GET /registry/filters`

Returns the FiltersRegistry (browser) index.

Response 200 application/json:

{
  "timestamp": "2026-03-10T00:00:00Z",
  "ttl": 3600,
  "lists": [
    {
      "filterId": 2,
      "name": "AdGuard Base Filter",
      "description": "...",
      "downloadUrl": "https://filters.adtidy.org/extension/chromium/filters/2.txt",
      "groupId": 1
    }
  ]
}

`GET /registry/services`

Returns the ServicesRegistry index (service-level blocking entries).

Symbolic Source URI Scheme

Once the fetchers are implemented, CompileRequest sources accept:

URI scheme	Example	Resolved by
`hostlist-registry://<filterKey>`	`hostlist-registry://adguard_dns_filter`	`HostlistRegistryFetcher`
`filters-registry://<filterId>`	`filters-registry://2`	`FiltersRegistryFetcher`

Example CompileRequest:

{
  "configuration": {
    "name": "My DNS Block List",
    "sources": [
      { "source": "hostlist-registry://adguard_dns_filter" },
      { "source": "hostlist-registry://goodbye_ads" }
    ],
    "transformations": ["Deduplicate", "RemoveComments"]
  }
}

The Worker expands the symbolic URI to the real downloadUrl before invoking the compiler.

Implementation Plan

Phase 1 — Registry Proxy Endpoints

Add GET /registry/hostlists handler in worker/
Add GET /registry/filters handler in worker/
Add GET /registry/services handler in worker/
Cache responses in KV (key: registry:hostlists, TTL: 1 hour)
Add OpenAPI paths + schemas to docs/api/openapi.yaml
Regenerate docs/api/README.md and docs/api/index.html via deno task openapi:docs

Phase 2 — Fetcher Implementations

HostlistRegistryFetcher — resolves hostlist-registry:// URIs
- On first use, fetches and caches the HostlistsRegistry filters.json
- Looks up filterKey → downloadUrl, then delegates to HttpFetcher
FiltersRegistryFetcher — resolves filters-registry:// URIs
Register both fetchers in CompositeFetcher chain inside WorkerCompiler
Update SourceSchema to accept the new URI schemes (Zod .refine())

Phase 3 — UI List Picker

public/ — add RegistryPicker component
- On mount: GET /registry/hostlists + GET /registry/filters
- Renders a searchable list grouped by tag / groupId
- On selection: pushes { source: "hostlist-registry://<key>" } into the sources array
Update UI CompileForm to accept symbolic sources alongside raw URLs

Phase 4 — Contract Tests & Docs

Add contract tests for GET /registry/* endpoints in worker/openapi-contract.test.ts
Add REGISTRY_UI_INTEGRATION.md link to docs/SUMMARY.md
Add entry to docs/api/OPENAPI_SUPPORT.md endpoint table

Caching Strategy

Layer	Mechanism	TTL
Worker KV	`registry:hostlists`, `registry:filters` keys	1 hour
Worker in-memory	Module-level `Map` fallback if KV unavailable	5 minutes
`Cache-Control` header	`public, max-age=3600, stale-while-revalidate=300`	1 hour

OpenAPI Schema Additions

# To be added to docs/api/openapi.yaml

/registry/hostlists:
  get:
    tags: [Registry]
    summary: List HostlistsRegistry filter lists
    operationId: getHostlistsRegistry
    responses:
      '200':
        description: Cached HostlistsRegistry index
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/RegistryIndex'

/registry/filters:
  get:
    tags: [Registry]
    summary: List FiltersRegistry filter lists
    operationId: getFiltersRegistry
    responses:
      '200':
        description: Cached FiltersRegistry index
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/RegistryIndex'

/registry/services:
  get:
    tags: [Registry]
    summary: List ServicesRegistry entries
    operationId: getServicesRegistry
    responses:
      '200':
        description: Cached ServicesRegistry index
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/RegistryIndex'

New schemas:

RegistryIndex:
  type: object
  required: [timestamp, ttl, lists]
  properties:
    timestamp:
      type: string
      format: date-time
    ttl:
      type: integer
      description: Cache TTL in seconds
    lists:
      type: array
      items:
        $ref: '#/components/schemas/RegistryListEntry'

RegistryListEntry:
  type: object
  required: [name, downloadUrl]
  properties:
    filterKey:
      type: string
    filterId:
      type: integer
    name:
      type: string
    description:
      type: string
    downloadUrl:
      type: string
      format: uri
    tags:
      type: array
      items:
        type: string
    groupId:
      type: integer
    homepage:
      type: string
      format: uri

API README
Platform Support — CompositeFetcher / HttpFetcher chaining
Zod Validation — SourceSchema URI refinement
OpenAPI Support — endpoint table and spec
Streaming API — source:start / source:complete events for registry-resolved sources

Registry ↔ UI Integration

Registry ↔ UI Integration

Background

Goals

Architecture

New Endpoints

GET /registry/hostlists

GET /registry/filters

GET /registry/services

Symbolic Source URI Scheme

Implementation Plan

Phase 1 — Registry Proxy Endpoints

Phase 2 — Fetcher Implementations

Phase 3 — UI List Picker

Phase 4 — Contract Tests & Docs

Caching Strategy

OpenAPI Schema Additions

Related Documents

`GET /registry/hostlists`

`GET /registry/filters`

`GET /registry/services`