URL(..., scheme=True) crashes on non-standard Origin header (e.g. Safari's "null")

[jonatha...@whatho.net]

Hi all,

We hit a reproducible 500 in production and wanted to check whether this is a known issue, and whether we're missing a better way to solve it than routing around URL() entirely.

py4web version: 1.20260403.2, Python 3.13.14

THE PROBLEM

URL(path, scheme=True) is documented/used for building absolute URLs (e.g. for links embedded in outbound emails). Looking at the source:

    original_url = request.environ.get("HTTP_ORIGIN") or request.url
    orig_scheme, _, domain = original_url.split("/", 3)[:3]

This unconditionally prefers the client-supplied Origin header over request.url, and unpacks the split with no validation that it actually produced three parts.

We're behind a reverse proxy (nginx, TLS terminated there, forwarded internally over plain HTTP with X-Forwarded-Proto/X-Forwarded-Host set correctly), and we started seeing intermittent 500s from real users. The traceback:

    File ".../py4web/core.py", line 1016, in URL
        orig_scheme, _, domain = original_url.split("/", 3)[:3]
    ValueError: not enough values to unpack (expected 3, got 1)

Root cause: some clients send the literal header value `Origin: null` (a standards-compliant "opaque origin" serialization per RFC 6454 — Safari/WebKit sends this in various privacy-sensitive navigation contexts, e.g. certain redirect chains, sandboxed content, some private browsing scenarios). "null" has zero slashes, so the split returns a single-element list and the unpack blows up.

REPRODUCTION

Any route that calls URL(..., scheme=True) can be crashed like this:

    curl -X POST https://yourapp.example.com/some-route \
      -H "Origin: null" \
      --data-urlencode "whatever=your-form-needs"

Confirmed 100% reproducible this way, and we confirmed it was happening for real users (traceback in our logs matched exactly, with real request IDs) before we worked around it in our own code.

OUR WORKAROUND

We stopped calling URL(scheme=True) and instead build the absolute URL ourselves from request.url (parsed with urllib.parse.urlsplit, which doesn't choke on unexpected input), since request.url already reflects X-Forwarded-Proto/X-Forwarded-Host correctly for us and isn't client-controlled:

    def absolute_url(*parts, **kwargs):
        from urllib.parse import urlsplit
        kwargs.pop("scheme", None)
        rel = URL(*parts, **kwargs)
        parsed = urlsplit(request.url)
        scheme = parsed.scheme or "https"
        return f"{scheme}://{parsed.netloc}{rel}"

That's worked fine for us so far.

QUESTIONS

1. Is this a known issue / already fixed in a more recent py4web release? We're on 1.20260403.2.

2. Is there a reason URL(scheme=True) prefers HTTP_ORIGIN over request.url at all? We're guessing it's meant to handle proxies that don't set X-Forwarded-Proto correctly, but Origin seems like a poor proxy for that given it's client-supplied and only sent on some request types (not present at all on plain GETs, for instance).

3. Is our workaround the right approach, or is there an existing, more "supported" way to get a reliable absolute URL behind a reverse proxy that we should be using instead (e.g. some config flag we've missed)?

4. Happy to submit a PR (add validation before the unpack, and/or prefer request.url over Origin) if that's welcome — wanted to check first whether there's context we're missing about why it's built this way.

Thanks!