Stop bots before they hit Meta CAPI, Google Ads, and your warehouse

Why client-side bot detection isn't enough

Most bot filtering happens after the fact, in vendor dashboards or analytics tools, after the bot conversion has already polluted Smart Bidding signals and burned your delivery budget. By the time you spot it, the damage is done.

Bot filter configuration

Rules are versioned alongside the rest of your pipeline configuration. Change them via the management UI or by committing to your blueprint repo. Every rule change is auditable.

# bot-filter.yaml
# Bot filtering is evaluated at the Signal Core, before any
# transformation or vendor delivery. Each layer contributes a
# weighted score; the thresholds decide what happens.

enabled: true
preview_mode: false      # true = nothing is dropped, scores still recorded
flag_threshold: 0.5      # score >= this  -> tagged, still delivered
block_threshold: 0.7     # score >= this  -> dropped
log_blocked_events: true # dropped events routed to the bot-events topic
server_side_bypass: true # skip for HMAC-authenticated server events

layers:
  # Identities already judged to be bots short-circuit the rest.
  identity_cache:
    enabled: true
    auto_write: true
    ttl_seconds: 86400

  # IAB/ABC spiders and bots, plus automation framework tokens.
  iab_patterns:
    enabled: true
    custom_patterns:
      - "my-internal-crawler*"

  # What a real browser reports and automation often does not.
  signal_validation:
    enabled: true
    require_screen_dimensions: true
    require_timezone: true
    require_user_agent: true
    check_tz_geo_mismatch: true

  # One question, two sources: published ranges, then the
  # autonomous system for the hosting providers they miss.
  ip_classification:
    enabled: true
    observe: false
    block_spamhaus: true
    block_tor: true
    block_datacenter_ips: true
    allowlisted_cidrs: ["203.0.113.45/32"]
    check_asn: true
    allowlisted_asns: [64496]   # your own corporate network
    blocked_asns: []

  # Per-identity sliding window: velocity and impossible travel.
  behavioural:
    enabled: true
    window_seconds: 5
    max_events_per_window: 50
    impossible_travel_enabled: true
    max_km_per_hour: 1000

allowlisted_user_agents:
  - "DataflyHealthCheck*"

What gets filtered, by detection layer

Each detection layer targets a different category of invalid traffic. They run in parallel at the Signal Core, with the first match determining the outcome.

Frequently asked questions

What is invalid traffic and why does it matter for paid media?

Invalid traffic (IVT) is bot, scraper, datacenter, and automation traffic that reaches your site without a real user behind it. Industry estimates from HUMAN Security and Pixalate put IVT at 20-50% of programmatic ad traffic. When bot conversions reach Meta CAPI, Google Ads, or TikTok server-side, they corrupt Smart Bidding, inflate ROAS calculations, and burn delivery quota that should be available for real customers.

How is server-side bot filtering different from gtag.js or Meta Pixel?

Client-side tags run inside whatever browser context loads them, including headless Chromium scrapers and Selenium-driven bots. They cannot reliably distinguish a bot from a real user because the JavaScript fingerprint is genuine, just remote-controlled. Datafly Signal filters at the Signal Core, after events arrive at your gateway, using server-observed signals (IP and autonomous system, request velocity, geography, and headers) that client-side tags do not have access to.

Will filtering bots reduce my reported conversions?

In dashboards yes, in true performance no. Reported conversion counts may drop because bot conversions stop being counted. The conversions that remain are real, which is what Smart Bidding and lookalike audiences should be optimising on. Most teams see CAPI match rates and ROAS calculations improve once bots are filtered upstream.

Can I see which events were filtered and why?

Yes. Every filtered event is logged with the rule that matched, the action taken (drop, quarantine, or tag), and the original event payload. The management UI shows bot-filtered rate per pipeline in real time and lets you drill into specific events for investigation. You can route filtered events to a quarantine warehouse table for offline analysis instead of dropping them entirely.

Can I trial a new bot rule without risking real conversions?

Yes. Every detection layer has an observe mode. In observe mode the layer still scores each event and reports what it would have filtered, but its score never counts toward the drop decision, so no real conversion is ever lost while you evaluate it. The management UI shows the would-be filter rate per layer in real time. When the numbers look right, you switch that layer from observe to enforce. New environments start fully in observe mode, so your first deployment measures bot rate before it changes any delivery.

Does Datafly Signal replace IVT detection at the vendor level?

No, it complements it. Vendors like Meta and Google have their own downstream IVT detection. Datafly Signal filters upstream so the bot conversion never reaches the vendor in the first place, saving your CAPI quota and preventing the bot signal from polluting Smart Bidding before vendor-side IVT catches it. The two layers stack.

Can I bring my own bot signature list?

Yes. You can add your own user-agent patterns, IP allowlists, and ASN allow and block lists per pipeline, and tune the velocity and impossible-travel thresholds. Network intelligence (cloud provider ranges, Tor exit nodes, Spamhaus, and autonomous system data) refreshes automatically in your own deployment without a redeploy. The user-agent signature list ships with platform releases. Every rule is version-controlled alongside your blueprint.