Class: RubyLinearRegression

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby_linear_regression.rb

Overview

RubyLinearRegression

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ RubyLinearRegression

Returns a new instance of RubyLinearRegression.



8
9
10
11
 
# File 'lib/ruby_linear_regression.rb', line 8

def initialize
  @mu = 0
  @sigma = 1
end

Instance Attribute Details

#lambda ⇒ Object (readonly)

Returns the value of attribute lambda.



6
7
8
 
# File 'lib/ruby_linear_regression.rb', line 6

def lambda
  @lambda
end

#mu ⇒ Object (readonly)

Returns the value of attribute mu.



6
7
8
 
# File 'lib/ruby_linear_regression.rb', line 6

def mu
  @mu
end

#normalize ⇒ Object (readonly)

Returns the value of attribute normalize.



6
7
8
 
# File 'lib/ruby_linear_regression.rb', line 6

def normalize
  @normalize
end

#sigma ⇒ Object (readonly)

Returns the value of attribute sigma.



6
7
8
 
# File 'lib/ruby_linear_regression.rb', line 6

def sigma
  @sigma
end

#theta ⇒ Object (readonly)

Returns the value of attribute theta.



6
7
8
 
# File 'lib/ruby_linear_regression.rb', line 6

def theta
  @theta
end

#x ⇒ Object (readonly)

Returns the value of attribute x.



6
7
8
 
# File 'lib/ruby_linear_regression.rb', line 6

def x
  @x
end

#y ⇒ Object (readonly)

Returns the value of attribute y.



6
7
8
 
# File 'lib/ruby_linear_regression.rb', line 6

def y
  @y
end

Instance Method Details

#compute_cost(test_x = nil, test_y = nil) ⇒ Object

Compute the mean squared cost / error function



35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
 
# File 'lib/ruby_linear_regression.rb', line 35

def compute_cost test_x = nil, test_y = nil

  if not test_x.nil?
    test_x.each_index do |row|
      test_x[row].each_index do |i|
        test_x[row][i] = (test_x[row][i] - @mu[i]) / @sigma[i].to_f
      end
    end if @normalize
    test_x = test_x.map { |r| [1].concat(r) }
  end

  # per default use training data to compute cost if no data is given
  cost_x = test_x.nil? ? @x : Matrix.rows( test_x )
  cost_y = test_y.nil? ? @y : Matrix.rows( test_y.collect { |e| [e] } )

  # First use matrix multiplication and vector subtracton to find errors
  errors = (cost_x * @theta) - cost_y

  # Then square all errors
  errors = errors.map { |e| (e.to_f**2)  }

  # Find the mean of the square errors
  mean_square_error = 0.5 * (errors.inject{ |sum, e| sum + e }.to_f / errors.row_size)

  return mean_square_error
end

#load_training_data(x_data, y_data, normalize = true) ⇒ Object

Loads and normalizes the training data, must be called prior to training. Arguments:

x_data: (Two dimensiolnal array with the independent variables of your training data)
y_data: (Array with the dependent variables of your training data)



17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
 
# File 'lib/ruby_linear_regression.rb', line 17

def load_training_data x_data, y_data, normalize = true

      @normalize = normalize

      # normalize the x_data
      x_data = normalize_data( x_data ) if @normalize

      # add 1 column to our data
      x_data = x_data.map { |r| [1].concat(r) }

      # build our x Matrix & y Vector
      @x = Matrix.rows( x_data )
      @y = Matrix.rows( y_data.collect { |e| [e] } )

      @theta = Matrix.zero(@x.column_count, 1)
end

#predict(data) ⇒ Object

Makes a prediction based on your trained model. train_normal_equation must be called prior to making a prediction.

Arguments:

data: (Array of independent variables to base your prediction on)



102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/ruby_linear_regression.rb', line 102

def predict data

  # normalize
  data.each_index do |i|
    data[i] = (data[i] - @mu[i]) / @sigma[i].to_f
  end if @normalize

  # add 1 column to prediction data
  data = [1].concat( data )

  # perform prediction
  prediction = (Matrix[data] * @theta)[0,0].to_f

  return prediction

end

#train_gradient_descent(alpha = 0.01, iterations = 500, verbose = false) ⇒ Object

Calculate optimal theta using gradient descent Arguments:

alpha: Learning rate
iterations: Number of iterations to run gradient descent
verbose: If true will output cost after each iteration, can be used to find optimal learning rate (alpha) and iteration



82
83
84
85
86
87
88
89
90
91
92
93
94
95
 
# File 'lib/ruby_linear_regression.rb', line 82

def train_gradient_descent( alpha = 0.01, iterations = 500, verbose = false )

  0.upto( iterations ) do |i|
    @temp_theta = Array.new(@theta.row_size)
    0.upto(@theta.row_size-1) do |row|
      @temp_theta[row] = @theta[row,0] - alpha * compute_gradient(row)
    end

    @theta = Matrix.columns([@temp_theta])

    puts "Cost after #{i} iterations = #{compute_cost}" if verbose
  end

end

#train_normal_equation(l = 0) ⇒ Object

Calculate the optimal theta using the normal equation



63
64
65
66
67
68
69
70
71
72
73
74
75
 
# File 'lib/ruby_linear_regression.rb', line 63

def train_normal_equation l = 0

  @lambda = l
  lambda_matrix = Matrix.build(@theta.row_size,@theta.row_size) do |c,r|
      (( c == 0 && r == 0) || c != r) ? 0 : 1;
    end

  # Calculate the optimal theta using the normal equation
  # theta = ( X' * X )^1 * X' * y
  @theta = (@x.transpose * @x + @lambda * lambda_matrix ).inverse * @x.transpose * @y

  return @theta
end