Module: RubyLLM::Protocols::Cohere::Media

Defined in:
lib/ruby_llm/protocols/cohere/media.rb

Overview

Handles formatting of media content for the Cohere v2 API

Constant Summary collapse

URL_IMAGE_EXTENSIONS =

Cohere resolves a remote image by its extension and rejects a URL without one, so those are inlined from the bytes already fetched.

%w[png jpeg jpg gif webp].freeze

Class Method Summary collapse

Class Method Details

.documents?(messages) ⇒ Boolean

Returns:

  • (Boolean) 


71
72
73
 
# File 'lib/ruby_llm/protocols/cohere/media.rb', line 71

def documents?(messages)
  messages.any? { |message| message.attachments.any?(&:text?) }
end

.format_content(content, attachments = [], citations: false) ⇒ Object

Cohere user messages accept only text and image_url blocks. Text attachments become citable documents when citations are on, so they are pulled out of the message and left out of the content blocks.



17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
 
# File 'lib/ruby_llm/protocols/cohere/media.rb', line 17

def format_content(content, attachments = [], citations: false)
  parts = []
  parts << format_text(content) if content

  attachments.each do |attachment|
    case attachment.type
    when :image
      parts << format_image(attachment)
    when :text
      parts << format_text(attachment.for_llm) unless citations
    else
      raise UnsupportedAttachmentError, attachment.mime_type
    end
  end

  parts
end

.format_documents(messages) ⇒ Object

Text attachments across the conversation become the request's top-level documents array, which is what Cohere cites against.



57
58
59
60
61
62
63
64
65
66
67
68
69
 
# File 'lib/ruby_llm/protocols/cohere/media.rb', line 57

def format_documents(messages)
  index = -1

  messages.flat_map do |message|
    message.attachments.select(&:text?).map do |attachment|
      index += 1
      {
        id: "doc:#{index}",
        data: { title: attachment.filename, text: attachment.content }.compact
      }
    end
  end
end

.format_image(image) ⇒ Object 



42
43
44
45
46
47
 
# File 'lib/ruby_llm/protocols/cohere/media.rb', line 42

def format_image(image)
  {
    type: 'image_url',
    image_url: { url: image_source(image) }
  }
end

.format_text(text) ⇒ Object 



35
36
37
38
39
40
 
# File 'lib/ruby_llm/protocols/cohere/media.rb', line 35

def format_text(text)
  {
    type: 'text',
    text: text
  }
end

.image_source(image) ⇒ Object 



49
50
51
52
53
 
# File 'lib/ruby_llm/protocols/cohere/media.rb', line 49

def image_source(image)
  return image.source.to_s if image.url? && URL_IMAGE_EXTENSIONS.include?(image.extension)

  "data:#{image.mime_type};base64,#{image.encoded}"
end