Module: RubyLLM::Providers::GPUStack::Transcription

Included in:
ChatCompletions
Defined in:
lib/ruby_llm/providers/gpustack/transcription.rb

Overview

GPUStack's vLLM transcription stream uses Chat Completions deltas.

Class Method Summary collapse

Class Method Details

.build_transcription_chunk(data) ⇒ Object 



14
15
16
17
18
19
20
21
22
23
24
25
 
# File 'lib/ruby_llm/providers/gpustack/transcription.rb', line 14

def build_transcription_chunk(data)
  return super unless data.key?('choices')

  choice = data['choices'].first || {}
  finished = choice['finish_reason'] || data['choices'].empty?

  RubyLLM::TranscriptionChunk.new(
    type: finished ? RubyLLM::TranscriptionChunk::DONE : RubyLLM::TranscriptionChunk::DELTA,
    delta: choice.dig('delta', 'content'),
    raw: data
  )
end

.stream_transcription(payload, model:) ⇒ Object 



10
11
12
 
# File 'lib/ruby_llm/providers/gpustack/transcription.rb', line 10

def stream_transcription(payload, model:, &)
  super({ stream_include_usage: 'true' }.merge(payload), model:, &)
end