Cloudflare has a URL normalization tool that may be used to change the URLs of incoming requests to conform to a uniform formatting standard. Enabling URL normalization, in other words, guarantees that incoming URLs are normalized before being sent on to other Cloudflare edge features that accept URL input. Furthermore, regardless of the original incoming URL, the rule expressions responsible for filtering traffic according to URLs will perform seamlessly.
If the URL normalization option is not enabled, Cloudflare will forward the URL in its original form. This could lead to ERROR.
How does URL Normalization in CloudFlare Work?
Incoming URLs are normalized by changing encoded elements, separators, and literal bytes to ensure they follow a consistent formatting standard. We have a Firewall Rule set up to prevent all requests that match www.example.com/hello. This constraint, however, is insufficient to prevent a request with an encoded element, such as www.example.com/ percent 76ello.
Fortunately, the Cloudflare Firewall Rules expressions that use URLs are easier by normalizing the incoming URLs.
The following characters are percent decoded according to RFC-3986:
- Characters in the alphabet: a-z, A-Z.
- The following characters are not encoded or decoded: 0-9 hyphen –, period., underscore _, and tilde # [] @! $ &'() * +, ; =: /?
- Other characters, such as literal byte values, are encoded in percent.
- Remove Dot Segment protocol is used to normalize URL paths.
In addition to RFC-3986, Cloudflare can use the following additional techniques:
- Combine a series of forward slashes.
- Backslashes should be converted to forward slashes.
Configure URL Normalization in the Dashboard
The Rules app in the Cloudflare dashboard can be used to establish URL normalization settings, as shown below:
- To begin, go to the dashboard and select an account and a website.
- Then, under Rules, go to Settings.
- Then, under Configure Normalization, configure the available URL normalization options.
URL Normalization Settings
- Type of normalization:
We can select whatever type of normalization we desire here: - RFC-3986: This implements RFC-3986 URL normalization.
- Cloudflare: In addition to RFC-3986, Cloudflare provides URL normalization algorithms.
- Normalize Incoming URLs:
The URLs of all incoming traffic are configured with this parameter. When enabled, it normalizes all incoming URLs before sending them on to Cloudflare’s other services. - Normalize URLs to Origin:
This option configures the origin of URLs. It is turned off by default.