Method: Boxcars::Cohere#max_tokens_for_prompt

Defined in:
lib/boxcars/engine/cohere.rb

#max_tokens_for_prompt(prompt_text) ⇒ Integer

Calculate the maximum number of tokens possible to generate for a prompt.



137
138
139
140
141
142
143
 
# File 'lib/boxcars/engine/cohere.rb', line 137

def max_tokens_for_prompt(prompt_text)
  num_tokens = get_num_tokens(prompt_text)

  # get max context size for model by name
  max_size = modelname_to_contextsize(model_name)
  max_size - num_tokens
end