RFC: Browser-Syntax Engine Integration

RFC: Browser-Syntax Engine Integration

Status: Core library and CLI implemented (src/); worker/ API and frontend/ UI not yet implemented — tracked as follow-up. Author: Claude (session), for jayson.knight@bloqr.dev Repos touched: bloqr-compiler (this repo). Related: bloqr-core, bloqr-csstree, adg-extcsstree.

See Browser-Syntax Engine for the current user-facing docs (CLI usage, library API, what’s implemented vs. not). This RFC is kept as the original design record; §5/§6 below are updated to reflect what has since shipped.

1. Problem

bloqr-compiler currently compiles one grammar: DNS-style / hosts-style blocklists (SourceType.Adblock and SourceType.Hosts, see src/types/index.ts). These are domain-oriented rules consumed by DNS resolvers (AdGuard Home/DNS, Pi-hole, etc.) — one rule = one blocked domain, with a small set of modifiers.

bloqr-core (and the wider AdGuard ecosystem) also needs to compile browser-syntax filter lists: network rules with URL/resource-type modifiers, cosmetic (element-hiding) rules, extended CSS selectors, scriptlet/JS injection rules — the grammar consumed by browser extensions (AdGuard Browser Extension, uBlock Origin-compatible engines). This grammar is materially different: it’s not just “domains with extra fields,” it has its own AST (selectors, scriptlet calls, $ modifier lists, exception rules with more option types, cosmetic exception scoping, etc.).

The ask: one compiler, two grammars. A single CLI/config surface that can ingest sources of either syntax, auto-detect (or be told via a flag) which grammar a source uses, run each source through the correct parser/validator/transformation pipeline for its grammar, and emit separate output artifacts per engine — never merge DNS rules and browser rules into one output file, since they’re consumed by different engines with different parsers.

2. Non-goals (this RFC)

• Rewriting the transformation pipeline to be grammar-generic in one pass. DNS transformations (Deduplicate, RemoveComments, Compress, …) mostly already operate on opaque string[] and are grammar-agnostic; browser-syntax will get its own set of transformations where semantics differ (e.g. “deduplicate” for a cosmetic rule must consider ##/#@#/#?# selector identity, not raw string identity in all cases). Building the full browser-specific transformation set is follow-up work.
• Editing worker/ routes/API surface to expose engine selection. Follow-up.
• Full AGTree-based validation of every browser rule type. The scaffold parses and round-trips generic/cosmetic/network rules; scriptlet/redirect resource validation is follow-up.

3. Chosen libraries

• @adguard/agtree — AST parser/serializer for AdGuard’s browser-syntax rule grammar (network rules, cosmetic rules, extended CSS, scriptlets, comments, !#if/preprocessor directives). This is the parser of record for browser-syntax input.
• @adguard/tsurlfilter — the matching/validation engine AGTree rules are designed to feed. We depend on it for rule validation (does this rule parse into something the real browser engine would accept?) rather than for request matching, which is out of scope for a compiler.

Both are npm packages, consumed via Deno’s npm compatibility (npm:@adguard/agtree), matching this repo’s existing pattern for npm deps (see deno.json → imports). Note: We have forged the AdGuard repo to BloqrAI/adg-filterlibs and can port and customize for our needs, and publish to JSR to keep the supply chain consistent. Need opinions on this method though. we use AdGuard’s libraries now, but can use our own fork.

3.1 Important discovery: @adguard/agtree is already integrated

This repo already depends on @adguard/agtree (deno.json → imports) and has a substantial wrapper layer around it, predating this RFC:

• src/utils/AGTreeParser.ts (~800 lines) — parses single rules and whole filter lists into AGTree ASTs, exposes RuleCategory (Comment / Cosmetic / Network / Empty), NetworkRuleType (including HostRule for hosts-file syntax vs plain NetworkRule), rule generation/serialization, and conversion helpers.
• src/utils/AGTreeWalker.ts — typed AST visitor/walker.
• src/utils/IAGTreeAdapter.ts — adapter interface.
• src/utils/RuleUtils.ts — higher-level helpers (isComment, domain/pattern extraction, ASCII conversion) built on AGTreeParser.
• src/plugins/AGTreeParserPlugin.ts — exposes AGTree as a ParserPlugin for the existing plugin system (src/plugins/PluginSystem.ts), already capable of parsing cosmetic rules, network rules, and hosts rules through one interface.

This means the AST/parsing layer for browser-syntax rules is not a gap — it exists and is well-factored. The actual gap this RFC addresses is routing and output separation: nothing today decides “this source is browser-syntax, run it through a browser-appropriate transformation set and emit it to its own file” — every source is currently assumed to funnel into the single DNS-style output. @adguard/tsurlfilter (rule validation, as opposed to AST parsing) is not yet a dependency and remains new work.

Consequently, §5 below does not add a second/duplicate rule parser. The EngineDetector scaffold is a lightweight, dependency-free line-sniffer used purely for routing (which bucket does a source’s content belong to), independent of and complementary to AGTreeParser. The follow-up BrowserSyntaxCompiler (§6) should build directly on AGTreeParser / RuleUtils / agTreeParserPlugin rather than introduce a new AGTree wrapper — e.g. RuleCategory.Cosmetic is a 100%-accurate browser-syntax signal that EngineDetector’s regex-based sniffing could eventually delegate to for higher precision than substring matching, once it’s acceptable for the detector to pay AGTree’s parse cost per sampled line (today’s regex approach is intentionally cheap enough to run on every source unconditionally).

bloqr-csstree / adg-extcsstree (forked CSS/extended-CSS grammar parsers already in the org) are the deeper selector-syntax dependency AGTree itself uses upstream for Extended CSS parsing — we do not need to depend on them directly at this layer; AGTree handles selector validation for the rule forms this compiler cares about. If/when we need standalone Extended-CSS AST manipulation beyond what AGTree exposes, those repos are the natural place to pull from and can be added as a direct dependency then.

4. Architecture

src/
  engines/                          <- new
    types.ts                        <- EngineKind, EngineSource, EngineCompilationResult
    EngineDetector.ts                <- syntax sniffing: per-line and per-source
    dns/
      index.ts                       <- thin re-export of the existing DNS pipeline
                                         (SourceCompiler, TransformationPipeline, etc.)
                                         under the engine-facing interface, so callers
                                         don't care which engine they got
    browser/
      BrowserRuleParser.ts           <- wraps @adguard/agtree: parse(line) -> AST | ParseError
      BrowserSyntaxCompiler.ts       <- source -> validated/normalized browser rules
      transformations/               <- browser-specific transformations (follow-up;
                                         scaffold ships RemoveComments + Deduplicate
                                         reimplemented against the AST)
  compiler/
    MultiEngineCompiler.ts           <- new orchestrator (see below)
    FilterCompiler.ts                <- UNCHANGED, remains the DNS-engine compiler

4.1 Syntax detection

EngineDetector.detectSource(source: ISource): EngineKind resolves in this order (first match wins):

1. Explicit: source.engine (new optional field on ISource, 'dns' | 'browser') or a CLI flag scoped to a source. Explicit always wins — never overridden by sniffing.
2. Legacy explicit: source.type === SourceType.Hosts → dns. Hosts syntax has no browser-syntax equivalent, so this is unambiguous.
3. Sniffed: sample the first N (default 200) non-comment, non-empty lines. A line counts as a browser-syntax signal if it parses as one of: a cosmetic rule (##, #@#, #?#, #$#, #%# separators), a network rule with a $ modifier list AGTree recognizes, or a scriptlet/redirect call. A line counts as a DNS signal if it’s a bare domain / ||domain^ with only DNS-relevant modifiers, or hosts-file syntax (0.0.0.0 domain). Majority vote across the sample decides; ties default to dns (today’s only behavior, so ambiguous/legacy config keeps working unchanged).
4. Config default: configuration.defaultEngine (new optional top-level config field), falling back to 'dns' if unset — preserves 100% backward compatibility for every existing config file with no engine/defaultEngine field.

Detection is a pure function over line samples — no network calls — so it’s cheap to run per-source even for sources that already declare their engine explicitly (used for a --validate warning when the declared engine disagrees with the sniffed one).

4.2 Compilation orchestration

MultiEngineCompiler (new, sits above FilterCompiler):

class MultiEngineCompiler {
  async compile(configuration: IConfiguration): Promise<MultiEngineResult> {
    const grouped = groupSourcesByEngine(configuration.sources, this.detector);
    const results: MultiEngineResult = { dns: undefined, browser: undefined };
    if (grouped.dns.length) {
      results.dns = await this.dnsCompiler.compile({ ...configuration, sources: grouped.dns });
    }
    if (grouped.browser.length) {
      results.browser = await this.browserCompiler.compile({ ...configuration, sources: grouped.browser });
    }
    return results;
  }
}

• Each engine gets its own transformation pipeline instance, its own header, and its own output file. They never share a rule array.
• configuration.sources may freely mix engines in one config file — that’s the point. output/output.dns/output.browser (or -o / --browser-output CLI flags) let the user name both files; sane defaults: <name>.txt for DNS (unchanged, backward compatible) and <name>.browser.txt for browser-syntax.
• If a configuration’s sources are all one engine (the common case, and 100% of today’s real configs), the browser path is simply never invoked — zero overhead, zero behavior change for existing users. FilterCompiler itself is untouched; MultiEngineCompiler is an additive orchestrator, not a replacement.

4.3 CLI surface (src/cli/ArgumentParser.ts, CliApp.ts)

New flags, all optional and additive:

--engine <auto|dns|browser>   Force engine selection for all sources (default: auto)
--browser-output <path>       Output path for the browser-syntax artifact
                               (default: derived from -o / config name)

auto runs MultiEngineCompiler with detection as described in 4.1. dns/browser force every source through that single engine (today’s implicit behavior is dns forced when no browser-capable sources exist — unchanged).

4.4 Output separation

Two independent output files, two independent headers/checksums, generated by the existing HeaderGenerator (DNS) and a new BrowserHeaderGenerator (adds ! Title:/! Expires: AdGuard browser-extension header conventions instead of the DNS header format). Never concatenated. --copy-to-rules (existing flag) copies both when both were produced.

5. What’s implemented (src/ and the CLI)

• src/types/index.ts: EngineKind type, optional ISource.engine, optional IConfiguration.defaultEngine.
• src/configuration/schemas.ts: EngineKindSchema, and engine/defaultEngine added to SourceSchema/ConfigurationSchema (both .strict(), so these had to be added explicitly — the initial scaffold added the TS types but not the runtime validation, which would have rejected them).
• src/engines/types.ts, src/engines/EngineDetector.ts + EngineDetector.test.ts — routing/detection logic. Dependency-free (no AGTree parse cost), safe to run on every source unconditionally.
• src/engines/browser/BrowserSyntaxCompiler.ts + test — compiles browser-syntax configurations. Built directly on SourceCompiler/TransformationPipeline/ HeaderGenerator (all already grammar-agnostic; no new AGTree wrapper needed, per §3.1). Rejects DNS-only transformations (Compress, Validate, ValidateAllowIp, InvertAllow) with a clear error when requested directly.
• src/engines/MultiEngineCompiler.ts + test — orchestrates a configuration whose sources span both engines, returning one CompilationResult per engine present. Filters a shared top-level transformations list down to the browser-safe subset for the browser bucket (rather than throwing) so one config file doesn’t need a separate transformations list per engine — see filterToBrowserSafe and its test.
• src/index.ts: all of the above re-exported from the package root.
• src/cli/CliApp.ts: --engine <auto|dns|browser> and --browser-output <file> flags. When every source resolves to the DNS engine (unaffected by this feature — still true for every pre-existing config), the original single-file code path runs unchanged. --stdout is rejected for mixed-engine configs (two streams can’t share stdout) with a clear error naming --browser-output as the alternative.
• docs/architecture/browser-syntax-engine.mdx — user-facing docs (this RFC is the design record, not the usage doc).

@adguard/agtree was already present in deno.json, reused as-is. @adguard/tsurlfilter (validation engine) is still not a dependency — only needed for the follow-up browser-syntax Validate equivalent (§6.2).

6. Follow-up work (not yet implemented)

1. worker/ API — no engine field on compile requests, no per-engine output URLs yet. The Cloudflare Worker only ever produces DNS-engine output today.
2. frontend/ Angular UI — no engine selection or browser-syntax output display.
3. Add @adguard/tsurlfilter and a browser-syntax Validate equivalent (reject rules that don’t parse into something a real browser engine would accept, rather than only rejecting DNS-only transformation names).
4. Cosmetic-AST-aware Deduplicate for the browser engine (current default is exact- string dedup — safe, but misses semantically-identical rules under different serialization).
5. src/schemas/configuration.schema.json (the JSON Schema, distinct from the Zod schema already updated in §5) — additions for source.engine / defaultEngine.
6. Consider swapping EngineDetector’s network-rule modifier heuristic for an AGTree-parse-based check (RuleCategory.Cosmetic / NetworkRuleType) once detection accuracy matters more than per-source parse cost — see §3.1.