Module: Mongoid::Matcher::RegexpBudget Private

Defined in:: lib/mongoid/matcher/regexp_budget.rb

Overview

This module is part of a private API. You should avoid using this module if possible, as it may be removed or be changed in the future.

Bounds the time spent executing regular expressions while evaluating a single in-memory match operation.

A query condition can carry an application-supplied pattern, and the in-memory matcher compiles and runs that pattern in the caller's thread. Both the cost of one match and the number of matches performed are under the control of whoever supplied the condition, so the limit is cumulative over an entire operation rather than per match.

The budget is held in thread- or fiber-local storage, so concurrent queries are accounted for independently.

Defined Under Namespace

Classes: Budget, TimedOut

Constant Summary collapse

PER_REGEXP_TIMEOUT =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Whether a per-Regexp timeout can be relied on to reach Regexp.new.

MRI added them in 3.2. JRuby 10.0.6 defines Regexp::TimeoutError, reports Ruby 3.4, and does honour a timeout that reaches it, but its Regexp.new accepts the keyword only for the first couple of calls through a given call site and raises ArgumentError from then on. Because that breakage is per call site, no load-time probe can predict it: a probe at its own call site reports a capability that the call in Budget#compile does not have. So non-MRI engines are excluded outright and use the Timeout fallback, which does interrupt a Joni match already under way. Worth revisiting if JRuby fixes the keyword handling.

if RUBY_ENGINE == 'ruby' && defined?(::Regexp::TimeoutError)
  true
else
  false
end

TIMEOUT_ERROR =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

The exception raised by a per-Regexp timeout. Tied to the constant rather than to the probe, so that a timeout set some other way (an application assigning Regexp.timeout, say) is still translated. On Rubies with no such constant, a class that is never raised stands in.

defined?(::Regexp::TimeoutError) ? ::Regexp::TimeoutError : Class.new(StandardError)

Class Method Summary collapse

.coerce(condition) ⇒ Regexp private
Returns the condition as a Regexp, without a timeout.
.limit_for(selector) ⇒ Float | nil private
The limit a scope evaluating this selector would be bounded by.
.match?(value, condition) ⇒ Integer | nil private
Matches a value against a regular expression condition, charging the time it takes against the open budget.
.open(selector, &block) ⇒ Object private
Opens a budget scope for the duration of the block.
.open_with(limit, &block) ⇒ Object private
Opens a budget scope for a limit the caller has already decided on.
.protect(&block) ⇒ Object private
Runs the block without letting a scope timeout tear it in half.
.remaining ⇒ Float | nil private
The time left in the open budget, or nil when no budget is open.

Class Method Details

.coerce(condition) ⇒ `Regexp`

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the condition as a Regexp, without a timeout.

Parameters:

condition (Regexp | BSON::Regexp::Raw) —
The condition.

Returns:

(Regexp) —
The pattern.

# File 'lib/mongoid/matcher/regexp_budget.rb', line 317

def coerce(condition)
  case condition
  when ::Regexp then condition
  when BSON::Regexp::Raw then condition.compile
  else raise ArgumentError, "Not a regular expression: #{condition.inspect}"
  end
end

.limit_for(selector) ⇒ `Float | nil`

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

The limit a scope evaluating this selector would be bounded by.

A caller that has to rearrange its work to make the scan interruptible -- loading documents up front so that nothing is mutated before the scan finishes, say -- can ask this first and skip the rearrangement, and whatever it costs, when there is no pattern to bound. It then passes what it got to .open_with, so that the decision it acted on is the one the scope is opened with. Callers with nothing to rearrange should just call .open, which asks this itself.

Parameters:

selector (Hash) —
The selector about to be evaluated.

Returns:

(Float | nil) —
The limit, or nil where there is nothing to bound.

# File 'lib/mongoid/matcher/regexp_budget.rb', line 239

def limit_for(selector)
  # nil.to_f is 0.0, so an unset limit and a limit of zero or less are
  # the same thing here: no limit. Zero is a common way to spell
  # "disabled", and taking it literally would mean a budget that is
  # spent before the first match and a query that can never run.
  limit = Mongoid::Config.in_memory_regexp_time_limit.to_f
  return nil unless limit.positive?

  limit if contains_regexp?(selector)
end

.match?(value, condition) ⇒ `Integer | nil`

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Matches a value against a regular expression condition, charging the time it takes against the open budget.

Parameters:

value (Object) —
The value to match.
condition (Regexp | BSON::Regexp::Raw) —
The condition.

Returns:

(Integer | nil) —
The offset of the match, or nil.

Raises:

(Errors::InMemoryRegexpTimeout) —
if the budget is exhausted.

# File 'lib/mongoid/matcher/regexp_budget.rb', line 259

def match?(value, condition)
  budget = current
  return value =~ coerce(condition) unless budget

  started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  pattern = nil
  begin
    raise timeout_error(budget) if budget.exhausted?

    # Compiling is charged too. It is not free, and for a pattern with
    # very many branches it costs far more than running the pattern
    # does, so leaving it out would leave a way to spend unbounded time
    # without the budget ever noticing.
    pattern = budget.compile(condition)
    value =~ pattern
  rescue TIMEOUT_ERROR
    # Name the limit that actually fired, which is not always the
    # budget's. A baked pattern carries it: the smaller of the budget's
    # limit and any global Regexp.timeout the application has set.
    # Where nothing was baked -- an engine that raises this error but
    # will not take a per-Regexp timeout, which is JRuby -- the global
    # is the only thing that can have fired, and the pattern reports
    # nil. Naming the budget's limit in either case would state a time
    # that was never spent and send the reader after a setting that is
    # not the one in the way.
    #
    # Both readers arrived together with Regexp::TimeoutError, so every
    # engine that can reach this rescue at all has them.
    raise timeout_error(budget, pattern&.timeout || ::Regexp.timeout || budget.limit)
  ensure
    budget.charge(Process.clock_gettime(Process::CLOCK_MONOTONIC) - started)
  end
end

.open(selector, &block) ⇒ `Object`

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Opens a budget scope for the duration of the block.

Nested calls join the enclosing budget instead of starting a new one, which is what lets a scan over many documents share a single limit. It also keeps the recursion in Expression.matches? (through $elemMatch, $and, $or and $nor) from resetting the budget.

No budget is opened for a selector that carries no regular expression. There would be nothing for it to bound, and on the Timeout path it would put a deadline on in-memory work that has nothing to do with regular expressions.

The scope covers everything nested inside the block, a selector other than this one included: a nested call joins the scope rather than deciding for itself, which is what keeps a scan from walking the selector once per document. Where the scope has nothing to bound, that means nested selectors are not bounded either -- so do not open one around work that can run application code. Loading documents runs find callbacks, and a query in one of those brings its own selector.

Where a selector does carry one, the Timeout path still measures the whole scope rather than the matching alone, so a long scan can trip the limit with a cheap pattern. That imprecision is accepted: the alternative is a Timeout around each individual match, which was measured at about nine seconds per million matches, and the limit exists to bound a scan of exactly that size. The error message is worded to hold either way, and Rubies with per-Regexp timeouts -- every supported MRI from 3.2 on -- do not take this path at all.

Code inside the block that mutates state should be wrapped in .protect, since on Rubies without a per-Regexp timeout the budget is enforced with an asynchronous exception that can land anywhere.

Parameters:

selector (Hash) —
The selector about to be evaluated.

Returns:

(Object) —
The value of the block.

# File 'lib/mongoid/matcher/regexp_budget.rb', line 172

def open(selector, &block)
  # The key is present but nil where an enclosing scope found nothing
  # to bound, so that a nested call does not scan the selector again.
  # Deciding before this check, in a default argument say, would walk
  # the selector once per document on a scan.
  return yield if Threaded.has?(Threaded::REGEXP_BUDGET_KEY)

  open_with(limit_for(selector), &block)
end

.open_with(limit, &block) ⇒ `Object`

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Opens a budget scope for a limit the caller has already decided on.

A caller that rearranges its work around the decision -- loading documents up front so that nothing is mutated before the scan finishes, say -- has to make it before it can act on it, and must not then make it a second time. Asking .limit_for and letting .open ask again reads the configured limit twice, and the two reads can differ: a limit that becomes positive in between would establish a budget in the branch that was chosen for not needing one, and on the Timeout path that arms a deadline over work the branch never made interruptible.

See .open for what the scope does and does not bound, and for the note about mutating state inside it.

Parameters:

limit (Float | nil) —
The seconds the scope may spend, or nil for a scope with nothing to bound.

Returns:

(Object) —
The value of the block.

# File 'lib/mongoid/matcher/regexp_budget.rb', line 201

def open_with(limit, &block)
  return yield if Threaded.has?(Threaded::REGEXP_BUDGET_KEY)

  budget = Budget.new(limit) if limit&.positive?

  begin
    # Set inside the begin so that an asynchronous exception from an
    # enclosing timeout cannot leave the key behind on a pooled thread.
    Threaded.set(Threaded::REGEXP_BUDGET_KEY, budget)

    if budget.nil? || PER_REGEXP_TIMEOUT
      yield
    else
      begin
        Timeout.timeout(budget.limit, TimedOut, &block)
      rescue TimedOut
        raise timeout_error(budget)
      end
    end
  ensure
    Threaded.delete(Threaded::REGEXP_BUDGET_KEY)
  end
end

.protect(&block) ⇒ `Object`

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Runs the block without letting a scope timeout tear it in half.

Where the budget is enforced with Timeout, the exception is raised asynchronously and can arrive at any point. Wrapping a mutation in this holds the exception back until the block has finished, so the interruption is deferred rather than given up.

Returns:

(Object) —
The value of the block.



301
302
303

# File 'lib/mongoid/matcher/regexp_budget.rb', line 301

def protect(&block)
  Thread.handle_interrupt(TimedOut => :never, &block)
end

.remaining ⇒ `Float | nil`

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

The time left in the open budget, or nil when no budget is open.

Returns:

(Float | nil) —
The remaining seconds.



308
309
310

# File 'lib/mongoid/matcher/regexp_budget.rb', line 308

def remaining
  current&.remaining
end

Module: Mongoid::Matcher::RegexpBudget Private

Overview

Defined Under Namespace

Constant Summary collapse

Class Method Summary collapse

Class Method Details

.coerce(condition) ⇒ Regexp

.limit_for(selector) ⇒ Float | nil

.match?(value, condition) ⇒ Integer | nil

.open(selector, &block) ⇒ Object

.open_with(limit, &block) ⇒ Object

.protect(&block) ⇒ Object

.remaining ⇒ Float | nil

.coerce(condition) ⇒ `Regexp`

.limit_for(selector) ⇒ `Float | nil`

.match?(value, condition) ⇒ `Integer | nil`

.open(selector, &block) ⇒ `Object`

.open_with(limit, &block) ⇒ `Object`

.protect(&block) ⇒ `Object`

.remaining ⇒ `Float | nil`