Class: Ai4r::Classifiers::LogisticRegression

Inherits:
Classifier
  • Object
show all
Defined in:
lib/ai4r/classifiers/logistic_regression.rb

Overview

Implementation of binary Logistic Regression using gradient descent.

Training data must have numeric attributes with the last attribute being the class label (0 or 1). Parameters can be adjusted with Data::Parameterizable#set_parameters.

Example:

data = Ai4r::Data::DataSet.new(:data_items => [[0.2, 1], [0.4, 0]])
classifier = LogisticRegression.new.build(data)
classifier.eval([0.3])

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Data::Parameterizable

#get_parameters, included, #set_parameters

Constructor Details

#initialize ⇒ LogisticRegression

Returns a new instance of LogisticRegression.



33
34
35
36
37
38
 
# File 'lib/ai4r/classifiers/logistic_regression.rb', line 33

def initialize
  super()
  @learning_rate = 0.1
  @iterations = 1000
  @weights = nil
end

Instance Attribute Details

#weights ⇒ Object (readonly)

Returns the value of attribute weights.



28
29
30
 
# File 'lib/ai4r/classifiers/logistic_regression.rb', line 28

def weights
  @weights
end

Instance Method Details

#build(data_set) ⇒ Object

Train the logistic regression classifier using the provided dataset.



41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
 
# File 'lib/ai4r/classifiers/logistic_regression.rb', line 41

def build(data_set)
  raise 'Error instance must be passed' unless data_set.is_a?(Ai4r::Data::DataSet)

  data_set.check_not_empty

  x = data_set.data_items.map { |item| item[0...-1].map(&:to_f) }
  y = data_set.data_items.map { |item| item.last.to_f }
  m = x.length
  n = x.first.length
  @weights = Array.new(n + 1, 0.0) # last value is bias

  @iterations.times do
    predictions = x.map do |row|
      z = row.each_with_index.inject(@weights.last) { |s, (v, j)| s + (v * @weights[j]) }
      1.0 / (1.0 + Math.exp(-z))
    end
    errors = predictions.zip(y).map { |p, label| p - label }

    n.times do |j|
      grad = (0...m).inject(0.0) { |sum, i| sum + (errors[i] * x[i][j]) } / m
      @weights[j] -= @learning_rate * grad
    end
    bias_grad = errors.sum / m
    @weights[n] -= @learning_rate * bias_grad
  end
  self
end

#eval(data) ⇒ Object

Predict the class (0 or 1) for the given data array.



70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
 
# File 'lib/ai4r/classifiers/logistic_regression.rb', line 70

def eval(data)
  raise 'Model not trained' unless @weights

  expected_size = @weights.length - 1
  if data.length != expected_size
    raise ArgumentError,
          "Wrong number of inputs. Expected: #{expected_size}, " \
          "received: #{data.length}."
  end

  z = data.each_with_index.inject(@weights.last) do |s, (v, j)|
    s + (v.to_f * @weights[j])
  end
  prob = 1.0 / (1.0 + Math.exp(-z))
  prob >= 0.5 ? 1 : 0
end

#get_rules ⇒ Object

Logistic Regression classifiers cannot generate human readable rules.

This method returns a string explaining that rule extraction is not supported for this algorithm.



91
92
93
 
# File 'lib/ai4r/classifiers/logistic_regression.rb', line 91

def get_rules
  'LogisticRegression does not support rule extraction.'
end