Anti-Bot Published Jul 9, 2026 · 4 min read · 848 words

Bypassing anti-bot protections: TLS, fingerprints, and Cloudflare

A frank guide to anti-bot defenses and how scrapers get past them: TLS/JA3 fingerprinting, Cloudflare's challenge, PerimeterX, CAPTCHAs, residential proxies, and the libraries (curl_cffi, Camoufox) that actually work in 2026.

Anti-bot has gotten good. The era when a custom User-Agent was enough is over. Modern systems like Cloudflare, PerimeterX/HUMAN, DataDome, and Akamai Bot Manager fingerprint your traffic at the TLS and HTTP/2 layers, where no header can save you. This guide explains what they actually look at and which tools work in 2026.

What anti-bot actually inspects

A bot detector builds a fingerprint from several layers, in roughly this order of importance:

The header stuff is the easy part. The TLS and HTTP/2 layers are why naive scrapers get 403s even with perfect headers.

TLS fingerprinting (JA3/JA4)

When your client opens a TLS connection, the ClientHello lists ciphers and extensions in a specific order. That order hashes to a value (JA3, and the newer JA4). Real browsers produce well-known values. Python's ssl stack produces a Python-specific one. No amount of header spoofing changes that.

The fix is a client that reproduces a browser's handshake:

# pip install curl_cffi
from curl_cffi import requests

resp = requests.get(
    "https://protected.example.com/",
    impersonate="chrome",   # reproduces Chrome's JA3 + HTTP/2 settings
    timeout=15,
)

curl_cffi wraps libcurl-impersonate. It ships JA3/JA4 and HTTP/2 fingerprints that match real Chrome, Safari, and Edge builds. For a lot of "I get 403 with requests" sites, this one swap is the fix.

HTTP/2 fingerprinting

Even with a correct JA3, the HTTP/2 SETTINGS frame (header table size, initial window, max concurrent streams) is browser-specific. curl_cffi's impersonate mode handles this too. If you're writing your own client, know that the HTTP/2 fingerprint is a second, independent check on top of JA3.

Cloudflare's challenge

Cloudflare's "Just a moment..." interstitial is a JS challenge. It proves a real browser ran JS and computed a token. Approaches, from least to most robust:

1. Solve it in a real browser. Use Playwright with stealth. Let the challenge auto-solve. Grab the cf_clearance cookie. Replay it in curl_cffi for the bulk of requests.
2. Use a managed solver. Services like ScrapingBee, ZenRows, or 2Captcha's Cloudflare module run the challenge for you. Costs money. Saves time.
3. Don't trigger it. Many sites only challenge suspicious traffic. A correct TLS fingerprint plus a residential IP plus a sane rate often skips the challenge entirely.

The cf_clearance cookie is bound to the IP and UA that solved it. Reuse the same IP and UA when you replay it.

Proxies: datacenter vs residential

A proxy fixes IP reputation. It does not fix a bad TLS fingerprint. The detector reads your fingerprint before routing by IP. Always pair proxies with a browser-accurate client (curl_cffi, Camoufox, or a stealthed Playwright).

from curl_cffi import requests

resp = requests.get(
    url,
    impersonate="chrome",
    proxies={"https": "http://user:pass@residential.proxy:4444"},
    timeout=15,
)

Rotate responsibly. One request per IP per second, not one IP per request. Aggressive rotation is itself a fingerprint.

CAPTCHAs

If you hit a CAPTCHA, something upstream already failed. You looked like a bot. Options:

• 2Captcha, CapSolver, Anti-Captcha: send the challenge, get a token back. Works for reCAPTCHA v2/v3, hCaptcha, Cloudflare Turnstile. Slow (5 to 30 seconds). Costs per solve.
• Avoid them: the cheapest CAPTCHA solver is never triggering one. Good fingerprint, residential IP, human timing. That eliminates most CAPTCHAs before they appear.

The 2026 toolkit

A note on ethics and law

Bypassing access controls can violate Terms of Service and, in some places, anti-circumvention law. This guide is for scraping your own properties, targets you have permission to scrape, or clearly public data. Read the ethics and robots.txt guide before you point any of this at someone else's site.

Key takeaways

• Detection happens at the TLS and HTTP/2 layer now, not the User-Agent.
• curl_cffi with impersonate="chrome" is the single highest-leverage swap.
• Proxies fix IP reputation. They don't fix a bad fingerprint.
• The best CAPTCHA solver is not triggering one.
• Prefer not triggering Cloudflare to solving it.

Found this useful? Cite it as: webscraping.space. “Bypassing anti-bot protections: TLS, fingerprints, and Cloudflare.” https://webscraping.space/blog/bypassing-anti-bot-protections. Published 2026-07-09.