Class: Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation

Inherits:

Object

• Object
• Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation

Extended by:

Protobuf::MessageExts::ClassMethods

Includes:

Protobuf::MessageExts

Defined in:

proto_docs/google/cloud/aiplatform/v1/endpoint.rb

Overview

N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.

Instance Attribute Summary

• #ngram_size ⇒ ::Integer

  The number of last N input tokens used as ngram to search/match against the previous prompt sequence.

Instance Attribute Details

#ngram_size ⇒ ::Integer

# File 'proto_docs/google/cloud/aiplatform/v1/endpoint.rb', line 450

class NgramSpeculation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end