Module: Otto::Utils
Overview
Utility methods for common operations and helpers
Constant Summary collapse
- FORWARDED_FOR_HEADERS =
Forwarded-for style headers consulted (in order) when resolving the real client IP from behind a trusted proxy. Shared by IPPrivacyMiddleware and Otto::Request so the two resolvers cannot drift.
%w[ HTTP_X_FORWARDED_FOR HTTP_X_REAL_IP HTTP_X_CLIENT_IP ].freeze
- FORWARDED_AUTHORITY_HEADERS =
Forwarding metadata Rack::Request reads without consulting Otto's proxy trust verdict: host (#host/#authority), scheme (#scheme/#ssl?), and port (#port), from both the X-Forwarded-* family and RFC 7239 Forwarded (host=, proto=, and the port inside for=). IPPrivacyMiddleware deletes every one of these for a peer that failed configured proxy trust.
%w[ HTTP_FORWARDED HTTP_X_FORWARDED_HOST HTTP_X_FORWARDED_PROTO HTTP_X_FORWARDED_SCHEME HTTP_X_FORWARDED_SSL HTTP_X_FORWARDED_PORT ].freeze
- RELAY_MARKER_HEADERS =
Headers whose presence means the request was RELAYED by a proxy rather than issued directly by the peer: every forwarding carrier Otto knows — the forwarded-for family, RFC 7239 Forwarded, and the authority (host / scheme / port) carriers. The set must cover everything IPPrivacyMiddleware may DELETE on the untrusted-peer path: a carrier that is scrubbed but not counted here would let a relayed request look direct afterwards. Shared by IPPrivacyMiddleware (which records the verdict as env BEFORE the scrub) and Otto::CaddyTLS::LocalhostGuard, so the record and the guard's own fallback scan cannot drift.
(FORWARDED_FOR_HEADERS + FORWARDED_AUTHORITY_HEADERS).uniq.freeze
- CLIENT_ADDRESS_HEADERS =
Headers the client-IP resolver may read an address from: the forwarded-for family (the CIDR walk, and X-Forwarded-For in depth mode) and RFC 7239 Forwarded (depth mode with trusted_proxy_header 'Forwarded' or 'Both'). A header resolve_client_ip starts reading belongs here, so that IPPrivacyMiddleware deletes it when no client IP resolves and Otto::Testing.env_for refuses it in a request it builds as direct.
(FORWARDED_FOR_HEADERS + %w[HTTP_FORWARDED]).freeze
- SPECIAL_USE_RANGES =
Special-use IPv4/IPv6 ranges that IPAddr's #private?/#loopback?/#link_local? predicates do not cover but that should still be treated as non-public (e.g. when picking the real client out of a forwarded chain).
The documentation ranges (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24, 2001:db8::/32 and the newer 3fff::/20) are deliberately NOT listed here. The predicate answers "could this be the real client", and non-public entries are skipped as proxy hops. Those ranges are not globally routable, but they are the universal convention for an example public client in tests and docs, including this repo's own specs and guides, so treating them as non-public would make the resolver skip the example client and silently invalidate those examples. There is no security gain either: an attacker who can inject into a chain injects a routable address, and a documentation address in a real chain means a misconfigured hop, which is better surfaced than skipped.
[ IPAddr.new('0.0.0.0/8'), # "this" network / unspecified (IPv4) IPAddr.new('224.0.0.0/4'), # IPv4 multicast IPAddr.new('::/128'), # IPv6 unspecified IPAddr.new('ff00::/8'), # IPv6 multicast ].freeze
Instance Method Summary collapse
-
#forwarded_chain_for_depth(env, header_mode) ⇒ Array<String>
Positional forwarded-hop chain for depth resolution, selected by header mode.
-
#ip_in_cidrs?(ip, cidrs) ⇒ Boolean
Whether an address falls inside any of the given CIDR ranges.
-
#loopback_address?(address) ⇒ Boolean
Whether an address is on the loopback interface.
-
#normalize_ip(ip) ⇒ String?
Validate and normalize an IP address (IPv4 and IPv6).
-
#normalize_path(raw_path) ⇒ String
Canonical path normalization for literal route matching: URL-unescape, scrub invalid/undefined bytes, and strip a single trailing slash.
-
#now ⇒ Time
Current time in UTC.
-
#now_in_μs ⇒ Integer
(also: #now_in_microseconds)
Returns the current time in microseconds.
-
#private_ip?(ip) ⇒ Boolean
Whether an address is non-public: RFC1918 private, loopback, link-local, multicast, or unspecified — for both IPv4 and IPv6.
-
#relayed_request?(env) ⇒ Boolean
Whether any relay marker header is present on the request.
-
#resolve_client_ip(env, security_config) ⇒ String?
Resolve the real client IP from a Rack env, honoring forwarded headers only when the connecting peer (REMOTE_ADDR) is a trusted proxy.
-
#resolve_client_ip_by_depth(env, security_config) ⇒ String?
Resolve the client IP by trusting a fixed number of proxy hops, counted from the right of the forwarded chain (Express
trust proxy = N). -
#rfc7239_for_chain(value) ⇒ Array<String>
Extract the per-hop
for=chain from an RFC 7239 Forwarded header, preserving one position per forwarded-element. -
#rfc7239_for_value(element) ⇒ String
Pull the
for=token out of a single RFC 7239 forwarded-element. -
#routing_path(env, include_mount: false) ⇒ String
The path Otto's router matches for this request.
-
#strip_ip_port(ip) ⇒ String
Strip an optional port without corrupting IPv6 addresses.
-
#xff_chain(value) ⇒ Array<String>
Split X-Forwarded-For into raw positional entries.
-
#yes?(value) ⇒ Boolean
Determine if a value represents a "yes" or true value.
Instance Method Details
#forwarded_chain_for_depth(env, header_mode) ⇒ Array<String>
Positional forwarded-hop chain for depth resolution, selected by header mode. Each element is one hop (preserving count); values are raw — only the finally-selected entry is normalized. Mirrors OneTimeSecret's site.network.trusted_proxy.header semantics.
320 321 322 323 324 325 326 327 328 329 330 331 332 333 |
# File 'lib/otto/utils.rb', line 320 def forwarded_chain_for_depth(env, header_mode) case header_mode when 'Forwarded' rfc7239_for_chain(env['HTTP_FORWARDED']) when 'Both' # RFC 7239 wins when it carries at least one `for=`; otherwise fall back # to X-Forwarded-For. The chains are NOT merged (matches OTS's # `extract_rfc7239_forwarded(env) || extract_x_forwarded_for(env)`). forwarded = rfc7239_for_chain(env['HTTP_FORWARDED']) forwarded.any? { |entry| !entry.empty? } ? forwarded : xff_chain(env['HTTP_X_FORWARDED_FOR']) else xff_chain(env['HTTP_X_FORWARDED_FOR']) end end |
#ip_in_cidrs?(ip, cidrs) ⇒ Boolean
Whether an address falls inside any of the given CIDR ranges.
The general-purpose CIDR-set matcher (allowlists, denylists, network zones), sharing the semantics of the trusted-proxy matcher: the client address is normalized (port stripped, validated) and folded via IPAddr#native so an IPv4-mapped IPv6 peer (::ffff:203.0.113.7) matches an IPv4 range; ranges of the other address family are skipped rather than raising.
Ranges are folded through #native too, so the fold is symmetric: a mapped-IPv6 CIDR (::ffff:10.0.0.0/104) matches a plain IPv4 client just as a mapped client matches a plain IPv4 range. Folding only one side made the family check reject the pair and silently drop the entry — wrong verdict, not a raise. #native returns self for ranges that are not IPv4-mapped/compatible, so ordinary IPv4 and IPv6 CIDRs are untouched, and it returns a new IPAddr rather than mutating, so pre-parsed entries in a caller's configuration array stay intact.
The fold needs the prefix to cover the mapped marker — /96 or longer. ::ffff:10.0.0.0/104 folds to 10.0.0.0/8; ::ffff:10.0.0.0/64 does not fold at all, because masking zeroes the ffff marker, and so matches neither an IPv4 client nor a mapped one. Write mapped ranges at /96+, or just write the IPv4 CIDR. ::ffff:0:0/96 is the whole mapped space and therefore matches every IPv4 address. Deprecated IPv4-compatible notation (::a.b.c.d) folds on the same terms, on both sides.
Asymmetric strictness, on purpose:
ipis runtime data — nil, blank, or malformed input returns false (fail-closed for allowlist callers).cidrsentries are configuration — an invalid CIDR string raises IPAddr::InvalidAddressError, because silently skipping an entry narrows an allowlist or widens a denylist. Validate entries at write/boot time; pre-parsed IPAddr entries skip re-parsing here.
451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 |
# File 'lib/otto/utils.rb', line 451 def ip_in_cidrs?(ip, cidrs) return false if cidrs.nil? client = if ip.is_a?(IPAddr) ip.native else candidate = normalize_ip(ip&.to_s) return false unless candidate IPAddr.new(candidate).native end cidrs.any? do |entry| range = entry.is_a?(IPAddr) ? entry : IPAddr.new(entry.to_s) # IPAddr#native builds its result with #clone, which carries frozen # state over and then fails to mutate it. Callers who freeze their # range configuration (or pass it through Ractor.make_shareable) would # hit FrozenError, so hand #native an unfrozen receiver. Gating the dup # on a foldable-range predicate would cost more than it saves: # #ipv4_compat? is deprecated and warns under -w, and #native already # short-circuits to self for anything that does not fold. range = range.dup if range.frozen? range = range.native range.family == client.family && range.include?(client) end end |
#loopback_address?(address) ⇒ Boolean
Whether an address is on the loopback interface.
This is the RAW SOCKET PEER test used to authenticate a direct local call (Otto::CaddyTLS::LocalhostGuard) — and, because IPPrivacyMiddleware now runs outermost and rewrites REMOTE_ADDR, the same test IPPrivacyMiddleware applies to the original peer and records as the leak-free boolean env. Shared here so the pre-masking record and the guard's own fallback cannot drift.
Fails closed: a blank or unparseable value is non-loopback rather than raising on the hot path.
#native folds IPv4-mapped IPv6 (::ffff:127.0.0.1, which dual-stack servers commonly present) so it is recognized as loopback; plain IPAddr#loopback? returns false for the mapped form.
Deliberately does NOT strip a ':port' suffix (unlike #private_ip?): a conforming Rack server reports the peer port in REMOTE_PORT, so an unexpected format means something upstream is non-standard and denying is safer than coercing.
514 515 516 517 518 519 520 521 |
# File 'lib/otto/utils.rb', line 514 def loopback_address?(address) addr = address.to_s.strip return false if addr.empty? IPAddr.new(addr).native.loopback? rescue IPAddr::InvalidAddressError, IPAddr::AddressFamilyError false end |
#normalize_ip(ip) ⇒ String?
Validate and normalize an IP address (IPv4 and IPv6).
Strips an optional port (IPv6-safe), validates with IPAddr, and returns the cleaned address string, or nil if the input is blank or malformed.
190 191 192 193 194 195 196 197 198 199 200 201 |
# File 'lib/otto/utils.rb', line 190 def normalize_ip(ip) return nil if ip.nil? || ip.empty? candidate = strip_ip_port(ip.strip) return nil if candidate.nil? || candidate.empty? # IPAddr validates both IPv4 and IPv6; raises for malformed input IPAddr.new(candidate) candidate rescue IPAddr::InvalidAddressError, IPAddr::AddressFamilyError nil end |
#normalize_path(raw_path) ⇒ String
Canonical path normalization for literal route matching: URL-unescape, scrub invalid/undefined bytes, and strip a single trailing slash.
This is the SINGLE SOURCE OF TRUTH shared by the router (Otto::Core::Router#handle_request, through #routing_path) and every guard that compares a request path or a configured path against what the router dispatches (Otto::CaddyTLS::LocalhostGuard, Otto::MCP.endpoint_path?). For a request, call #routing_path rather than passing PATH_INFO here yourself. Guard and router MUST normalize identically: if a crafted path — a trailing slash, a percent-encoded byte, an invalid UTF-8 byte — normalized differently in the guard than in the router, the router could dispatch a request the guard let through. One implementation makes that drift impossible.
Robust to invalid input. Rack::Utils.unescape raises ArgumentError on a malformed escape (%zz, a trailing %) and on an invalid byte in a UTF-8-tagged string (a raw \xFF); either way the raw string is kept. Invalid UTF-8 is scrubbed after that, so a raw \xFF and a percent-encoded %FF normalize alike. The method itself does not raise.
129 130 131 132 133 134 135 136 137 138 139 140 141 |
# File 'lib/otto/utils.rb', line 129 def normalize_path(raw_path) raw = raw_path.to_s decoded = begin Rack::Utils.unescape(raw) rescue ArgumentError raw end decoded = '/' if decoded.empty? decoded .encode('UTF-8', invalid: :replace, undef: :replace, replace: '') .gsub(%r{/$}, '') end |
#now ⇒ Time
Returns Current time in UTC.
79 80 81 |
# File 'lib/otto/utils.rb', line 79 def now Time.now.utc end |
#now_in_μs ⇒ Integer Also known as: now_in_microseconds
Returns the current time in microseconds. This is used to measure the duration of Database commands.
Alias: now_in_microseconds
89 90 91 |
# File 'lib/otto/utils.rb', line 89 def now_in_μs Process.clock_gettime(Process::CLOCK_MONOTONIC, :microsecond) end |
#private_ip?(ip) ⇒ Boolean
Whether an address is non-public: RFC1918 private, loopback, link-local, multicast, or unspecified — for both IPv4 and IPv6.
Uses IPAddr's family-aware predicates (which also fold IPv4-mapped IPv6 via #native) plus an explicit set of special-use ranges that the predicates don't cover (IPv4 0.0.0.0/8 and 224.0.0.0/4, IPv6 ::/128 and ff00::/8). Returns false for malformed input rather than raising.
397 398 399 400 401 402 403 404 405 406 407 408 409 |
# File 'lib/otto/utils.rb', line 397 def private_ip?(ip) return false if ip.nil? return false if ip.respond_to?(:empty?) && ip.empty? addr = ip.is_a?(IPAddr) ? ip : IPAddr.new(strip_ip_port(ip.to_s.strip)) addr = addr.native # fold IPv4-mapped IPv6 (::ffff:a.b.c.d) to IPv4 return true if addr.private? || addr.loopback? || addr.link_local? SPECIAL_USE_RANGES.any? { |range| range.family == addr.family && range.include?(addr) } rescue IPAddr::InvalidAddressError, IPAddr::AddressFamilyError false end |
#relayed_request?(env) ⇒ Boolean
Whether any relay marker header is present on the request.
Call this only from code that runs BEFORE forwarding carriers may be deleted (IPPrivacyMiddleware's untrusted-peer scrub); downstream code should prefer the recorded env boolean.
487 488 489 |
# File 'lib/otto/utils.rb', line 487 def relayed_request?(env) RELAY_MARKER_HEADERS.any? { |header| !env[header].to_s.strip.empty? } end |
#resolve_client_ip(env, security_config) ⇒ String?
Resolve the real client IP from a Rack env, honoring forwarded headers only when the connecting peer (REMOTE_ADDR) is a trusted proxy.
This is the single canonical resolver shared by IPPrivacyMiddleware ("resolve once") and Otto::Request#client_ipaddress (its no-middleware fallback), so both paths agree on which headers to trust and how to walk a proxy chain. It walks the forwarded chain left-to-right and returns the first address that is not itself a trusted proxy; if the peer is not a trusted proxy (or there is no config) it returns REMOTE_ADDR unchanged.
235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 |
# File 'lib/otto/utils.rb', line 235 def resolve_client_ip(env, security_config) remote_addr = env['REMOTE_ADDR'] # Count-based ("trust the last N hops") mode for non-enumerable proxy # tiers (Fly, cloud load balancers, dynamic reverse proxies) where the # CIDR-walk below has no enumerable proxy IPs to match. Mirrors Express # `trust proxy = N`. Takes precedence over CIDR-walk; the two modes are # mutually exclusive (enforced at config freeze). return resolve_client_ip_by_depth(env, security_config) if security_config&.trusted_proxy_depth_mode? # No config, or the peer is a direct (untrusted) connection: REMOTE_ADDR # is the client. Don't honor forwarded headers from untrusted sources. return remote_addr unless security_config&.trusted_proxy?(remote_addr) forwarded_ips = FORWARDED_FOR_HEADERS .filter_map { |header| env[header] } .flat_map { |value| value.split(/,\s*/) } forwarded_ips.each do |candidate| clean_ip = normalize_ip(candidate.strip) next unless clean_ip # First address in the chain that isn't a known proxy is the client. return clean_ip unless security_config.trusted_proxy?(clean_ip) end # Whole chain was trusted proxies (or empty): fall back to the peer. remote_addr end |
#resolve_client_ip_by_depth(env, security_config) ⇒ String?
Resolve the client IP by trusting a fixed number of proxy hops, counted
from the right of the forwarded chain (Express trust proxy = N). Used
when the proxy tier's addresses cannot be enumerated as CIDRs.
The chain is the configured forwarded header (leftmost = client .. rightmost = nearest proxy) plus REMOTE_ADDR (the direct peer). With depth N the client is chain — exactly N trusted hops from the right, equivalent to Express's addrs. This is robust to forwarded-header padding: a forged leftmost entry is never reached.
SECURITY: depth trust ASSUMES ORIGIN LOCKDOWN — the app must be unreachable except through the proxy tier. Without it, a direct client could pad the forwarded header to land a forged value at the target index. This is the inherent trade vs CIDR-walk (a fixed hop count instead of enumerable proxy addresses).
The forwarded chain is selected by security_config.trusted_proxy_header:
'X-Forwarded-For' (default), 'Forwarded' (RFC 7239), or 'Both' (RFC 7239
when it carries a for=, otherwise X-Forwarded-For — mirrors
OneTimeSecret's site.network.trusted_proxy.header). X-Real-IP / X-Client-IP
are single-value and cannot express a hop chain, so they are never
consulted in depth mode. Positions are counted raw (never dropped), so junk
padding cannot shift the index; only the selected entry is validated. If
the chain is shorter than N+1 (a request that may have bypassed the proxy
tier) or the selected entry is invalid, REMOTE_ADDR is returned rather than
a spoofable forwarded value.
295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 |
# File 'lib/otto/utils.rb', line 295 def resolve_client_ip_by_depth(env, security_config) remote_addr = env['REMOTE_ADDR'] depth = security_config.trusted_proxy_depth.to_i # Build the positional hop chain from the configured header, keeping every # position (junk/empty entries included) so the client can be located by # counting from the right; dropping entries would let padding shift the # index. REMOTE_ADDR (the direct peer) is the rightmost hop. forwarded = forwarded_chain_for_depth(env, security_config.trusted_proxy_header) chain = forwarded + [remote_addr] index = chain.length - (depth + 1) return remote_addr if index.negative? # chain shorter than depth + 1 normalize_ip(chain[index].to_s.strip) || remote_addr end |
#rfc7239_for_chain(value) ⇒ Array<String>
Extract the per-hop for= chain from an RFC 7239 Forwarded header,
preserving one position per forwarded-element. Elements without a for=
parameter yield a blank placeholder so they still occupy a hop position
(raw position counting). The extracted token is only unquoted here; port
and IPv6 brackets are left for normalize_ip when the entry is selected.
Obfuscated (for=_hidden) and for=unknown identifiers are preserved as
positions but normalize to nil (→ REMOTE_ADDR fallback if selected).
Commas separate forwarded-elements (and join multiple Forwarded headers).
A nil/blank header splits to [] (not ['']), so an absent Forwarded header
yields an empty chain and depth's explicit short-chain guard returns
REMOTE_ADDR — symmetric with xff_chain.
358 359 360 |
# File 'lib/otto/utils.rb', line 358 def rfc7239_for_chain(value) value.to_s.split(',', -1).map { |element| rfc7239_for_value(element) } end |
#rfc7239_for_value(element) ⇒ String
Pull the for= token out of a single RFC 7239 forwarded-element. The value
is a quoted-string (which may itself legally contain ';') or an unquoted
token ending at the next ';'. The quoted form is matched first so a ';'
inside DQUOTEs is NOT treated as a parameter separator — otherwise a value
like for="1.2.3.4;junk" would be truncated to a valid-looking IP instead of
being rejected. Only DQUOTE wrappers are stripped: RFC 7239 quoted-strings
use DQUOTE exclusively, so a value like for='1.2.3.4' keeps its quotes,
fails normalize_ip, and safely falls back to REMOTE_ADDR rather than being
permissively accepted. This is deliberately stricter than OTS (which strips
both ['"]), consistent with depth's other intentionally-not-reconciled-down
safety properties. The raw value (port / IPv6 brackets intact) is left for
normalize_ip when the entry is selected. Returns '' when the element carries
no for= parameter, preserving the hop position. The for= pair may be the
element's first pair or follow a ';'; leading whitespace (e.g. after a comma
split) is tolerated.
380 381 382 383 384 385 |
# File 'lib/otto/utils.rb', line 380 def rfc7239_for_value(element) match = element.match(/(?:\A|;)\s*for=(?:"([^"]*)"|([^;]+))/i) return '' unless match (match[1] || match[2]).strip end |
#routing_path(env, include_mount: false) ⇒ String
The path Otto's router matches for this request. Use it in any code that judges a request by its path before the router sees it: guards, throttles, session skips, audit filters.
The router does not match raw PATH_INFO. It matches this value, and Otto::Core::Router#handle_request calls this method to get it, so a guard that reads routing_path sees the path the router dispatches on. A guard that reads anything else can see a different path, and when it matches less than the router does the difference is a bypass: GET /%63olonel is '/%63olonel' as raw PATH_INFO and '/colonel' to the router.
By default the result is mount-relative. Rack::URLMap
(map '/api' { run otto }) moves the mount prefix into SCRIPT_NAME and
leaves the remainder in PATH_INFO, which is all the router sees; this is
the form to compare against paths as written in a routes file.
With include_mount: true SCRIPT_NAME and PATH_INFO are joined and then
normalized as one string, giving the request's full path. That is
the form for middleware shared by several mounted apps and configured
with external URLs: inside an app mounted at /api/v2, '/status' is the
mount-relative path and '/api/v2/status' the mounted one, and matching
the mount-relative form would also match every other app's /status.
The value is normalize_path output, so root is '' (the router's literal table keys root the same way) and a configured path must go through normalize_path before an exact comparison. Never raises: a malformed escape such as %zz is kept as written.
Not memoized: middleware may rewrite PATH_INFO or SCRIPT_NAME, and the router must see the value as it stands at dispatch.
177 178 179 180 181 |
# File 'lib/otto/utils.rb', line 177 def routing_path(env, include_mount: false) path = env['PATH_INFO'] path = "#{env['SCRIPT_NAME']}#{path}" if include_mount normalize_path(path) end |
#strip_ip_port(ip) ⇒ String
Strip an optional port without corrupting IPv6 addresses.
Handles bracketed IPv6 with a port ([2001:db8::1]:443) and IPv4
host:port (203.0.113.5:443). A bare IPv6 address (multiple colons,
no brackets) is returned unchanged.
211 212 213 214 215 216 217 218 219 220 |
# File 'lib/otto/utils.rb', line 211 def strip_ip_port(ip) if ip.start_with?('[') inner = ip[/\A\[([^\]]+)\]/, 1] return inner if inner end return ip.split(':', 2).first if ip.count(':') == 1 ip end |
#xff_chain(value) ⇒ Array<String>
Split X-Forwarded-For into raw positional entries. -1 keeps trailing
empty fields so a malformed/empty hop still counts as a position.
340 341 342 |
# File 'lib/otto/utils.rb', line 340 def xff_chain(value) value.to_s.split(',', -1) end |
#yes?(value) ⇒ Boolean
Determine if a value represents a "yes" or true value
Examples: yes?('true') # => true yes?('yes') # => true yes?('1') # => true
103 104 105 |
# File 'lib/otto/utils.rb', line 103 def yes?(value) !value.to_s.empty? && %w[true yes 1].include?(value.to_s.downcase) end |