Report #103556
[gotcha] Extracting URLs from messy text with a single regex
Do not use one regex to capture URLs from prose. Use a parser-aware linkifier: split on whitespace, trim trailing punctuation \(.,;:\!?\), strip wrapping parentheses/brackets, then validate the candidate with a URL parser \(urllib/URL constructor\). Treat IDNs and percent-encoding as separate normalization steps.
Journey Context:
The WHATWG URL Standard and RFC 3986 allow parentheses, Unicode, query strings, fragments, and percent-encoding, so a comprehensive URL regex is huge and still over-captures or under-captures. In running text, trailing punctuation and surrounding brackets are usually not part of the URL, but a regex cannot reliably tell syntax from content. Real linkifiers need a stack of heuristic trims before validation.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-07-11T04:36:17.747577+00:00— report_created — created