Class: RailsAiGateway::Proxy Private

Inherits:
Object
  • Object
show all
Defined in:
lib/rails_ai_gateway/proxy.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

Internal Rack endpoint implementing OpenAI-compatible proxy behavior.

Defined Under Namespace

Classes: Failure, Stream

Constant Summary collapse

RETRY_STATUSES =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future. 

[429, 500, 502, 503, 504].freeze
ERROR_TYPES =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future. 

{
  400 => ["invalid_request_error", "invalid_request"],
  401 => ["authentication_error", "invalid_api_key"],
  403 => ["permission_error", "permission_denied"],
  404 => ["invalid_request_error", "model_not_found"],
  409 => ["conflict_error", "conflict"],
  422 => ["invalid_request_error", "unprocessable_entity"],
  429 => ["rate_limit_error", "rate_limit_exceeded"]
}.freeze
PRIVATE_RANGES =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future. 

%w[0.0.0.0/8 10.0.0.0/8 100.64.0.0/10 127.0.0.0/8 169.254.0.0/16
172.16.0.0/12 192.0.0.0/24 192.0.2.0/24 192.168.0.0/16 198.18.0.0/15
198.51.100.0/24 203.0.113.0/24 224.0.0.0/4 240.0.0.0/4 ::/128 ::1/128
fc00::/7 fe80::/10 ff00::/8 2001:db8::/32].map { |range| IPAddr.new(range) }.freeze

Instance Method Summary collapse

Constructor Details

#initialize(endpoint) ⇒ Proxy

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Proxy.



58
59
60
 
# File 'lib/rails_ai_gateway/proxy.rb', line 58

def initialize(endpoint)
  @endpoint = endpoint
end

Instance Method Details

#call(env) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/rails_ai_gateway/proxy.rb', line 62

def call(env)
  request = Rack::Request.new(env)
  token = request.get_header("HTTP_AUTHORIZATION").to_s[/\ABearer (\S+)\z/i, 1]
  key = GatewayKey.authenticate(token)
  return error("Invalid or expired gateway key", 401) unless key

  if @endpoint == "models"
    names = ModelRoute.available.distinct.pluck(:name).select { |name| key.allows?(name) }.sort
    capabilities = ModelRoute.available.where(name: names).group_by(&:name).transform_values { |routes| routes.flat_map(&:capabilities).uniq }
    return json(200, object: "list", data: names.map { |name| { id: name, object: "model", created: 0, owned_by: "rails_ai_gateway", capabilities: capabilities.fetch(name, []) } })
  end

  payload = parse_request(request)
  return error("Model is not allowed", 403) unless key.allows?(payload["model"])
  routes = ModelRoute.ranked_for_query(name: payload["model"], query: query_text(payload), capabilities: required_capabilities(payload)).first(config.max_attempts)
  return error("No route for requested model", 404) if routes.empty?

  log = RequestLog.create!(request_id: SecureRandom.uuid, gateway_key: key, model: payload["model"], endpoint: @endpoint)
  if payload["stream"]
    stream_response(routes, payload, log.id, log.request_id)
  else
    perform(routes, payload, log.id, log.request_id)
  end
rescue JSON::ParserError
  error("Invalid JSON", 400)
rescue Failure => exception
  error(exception.message, exception.status)
end