Class: Ai4r::Classifiers::SupportVectorMachine

Inherits:
Classifier
  • Object
show all
Defined in:
lib/ai4r/classifiers/support_vector_machine.rb

Overview

A lightweight linear SVM classifier trained via gradient descent. Only two classes are supported. Predictions return the same class labels used in the training data.

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Data::Parameterizable

#get_parameters, included, #set_parameters

Constructor Details

#initialize ⇒ SupportVectorMachine

Returns a new instance of SupportVectorMachine.



28
29
30
31
32
33
34
35
36
 
# File 'lib/ai4r/classifiers/support_vector_machine.rb', line 28

def initialize
  super()
  @learning_rate = 0.01
  @iterations = 1000
  @c = 1.0
  @weights = []
  @bias = 0.0
  @classes = []
end

Instance Attribute Details

#bias ⇒ Object (readonly)

Returns the value of attribute bias.



22
23
24
 
# File 'lib/ai4r/classifiers/support_vector_machine.rb', line 22

def bias
  @bias
end

#classes ⇒ Object (readonly)

Returns the value of attribute classes.



22
23
24
 
# File 'lib/ai4r/classifiers/support_vector_machine.rb', line 22

def classes
  @classes
end

#weights ⇒ Object (readonly)

Returns the value of attribute weights.



22
23
24
 
# File 'lib/ai4r/classifiers/support_vector_machine.rb', line 22

def weights
  @weights
end

Instance Method Details

#build(data_set) ⇒ Object

Train the SVM using the provided DataSet. Only numeric attributes and exactly two classes are supported.

Raises:

  • (ArgumentError) 


40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/ai4r/classifiers/support_vector_machine.rb', line 40

def build(data_set)
  data_set.check_not_empty
  @classes = data_set.build_domains.last.to_a
  raise ArgumentError, 'SVM only supports two classes' unless @classes.size == 2

  num_features = data_set.data_labels.length - 1
  @weights = Array.new(num_features, 0.0)
  @bias = 0.0

  samples = data_set.data_items.map do |row|
    [row[0...-1].map(&:to_f), row.last]
  end

  @iterations.times do
    samples.each do |features, label|
      y = label == @classes[0] ? 1.0 : -1.0
      prediction = dot(@weights, features) + @bias
      if y * prediction < 1
        @weights.map!.with_index do |w, i|
          w + (@learning_rate * ((@c * y * features[i]) - (2 * w)))
        end
        @bias += @learning_rate * @c * y
      else
        @weights.map!.with_index { |w, _i| w - (@learning_rate * 2 * w) }
      end
    end
  end
  self
end

#eval(data) ⇒ Object

Predict the class for the given numeric feature vector.



71
72
73
74
 
# File 'lib/ai4r/classifiers/support_vector_machine.rb', line 71

def eval(data)
  score = dot(@weights, data.map(&:to_f)) + @bias
  score >= 0 ? @classes[0] : @classes[1]
end

#get_rules ⇒ Object

Support Vector Machine classifiers cannot generate human readable rules. This method returns a string indicating rule extraction is unsupported.



78
79
80
 
# File 'lib/ai4r/classifiers/support_vector_machine.rb', line 78

def get_rules
  'SupportVectorMachine does not support rule extraction.'
end