Class: SVMKit::KernelMachine::KernelSVC

Inherits:
Object
  • Object
show all
Includes:
Base::BaseEstimator, Base::Classifier
Defined in:
lib/svmkit/kernel_machine/kernel_svc.rb

Overview

KernelSVC is a class that implements (Nonlinear) Kernel Support Vector Classifier with stochastic gradient descent (SGD) optimization. For multiclass classification problem, it uses one-vs-the-rest strategy.

Reference

    1. Shalev-Shwartz, Y. Singer, N. Srebro, and A. Cotter, “Pegasos: Primal Estimated sub-GrAdient SOlver for SVM,” Mathematical Programming, vol. 127 (1), pp. 3–30, 2011.

Examples:

training_kernel_matrix = SVMKit::PairwiseMetric::rbf_kernel(training_samples)
estimator =
  SVMKit::KernelMachine::KernelSVC.new(reg_param: 1.0, max_iter: 1000, random_seed: 1)
estimator.fit(training_kernel_matrix, traininig_labels)
testing_kernel_matrix = SVMKit::PairwiseMetric::rbf_kernel(testing_samples, training_samples)
results = estimator.predict(testing_kernel_matrix)

Instance Attribute Summary collapse

Attributes included from Base::BaseEstimator

#params

Instance Method Summary collapse

Constructor Details

#initialize(reg_param: 1.0, max_iter: 1000, random_seed: nil) ⇒ KernelSVC

Create a new classifier with Kernel Support Vector Machine by the SGD optimization.

Parameters:

  • reg_param (Float) (defaults to: 1.0) —

    The regularization parameter.

  • max_iter (Integer) (defaults to: 1000) —

    The maximum number of iterations.

  • random_seed (Integer) (defaults to: nil) —

    The seed value using to initialize the random generator.



44
45
46
47
48
49
50
51
52
53
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 44

def initialize(reg_param: 1.0, max_iter: 1000, random_seed: nil)
  @params = {}
  @params[:reg_param] = reg_param
  @params[:max_iter] = max_iter
  @params[:random_seed] = random_seed
  @params[:random_seed] ||= srand
  @weight_vec = nil
  @classes
  @rng = Random.new(@params[:random_seed])
end

Instance Attribute Details

#classes ⇒ Numo::Int32 (readonly)

Return the class labels.

Returns:

  • (Numo::Int32) —

    (shape: [n_classes])



33
34
35
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 33

def classes
  @classes
end

#rng ⇒ Random (readonly)

Return the random generator for performing random sampling.

Returns:

  • (Random) 


37
38
39
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 37

def rng
  @rng
end

#weight_vec ⇒ Numo::DFloat (readonly)

Return the weight vector for Kernel SVC.

Returns:

  • (Numo::DFloat) —

    (shape: [n_classes, n_trainig_sample])



29
30
31
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 29

def weight_vec
  @weight_vec
end

Instance Method Details

#decision_function(x) ⇒ Numo::DFloat

Calculate confidence scores for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_testing_samples, n_training_samples]) The kernel matrix between testing samples and training samples to compute the scores.

Returns:

  • (Numo::DFloat) —

    (shape: [n_testing_samples, n_classes]) Confidence score per sample.



86
87
88
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 86

def decision_function(x)
  x.dot(@weight_vec.transpose)
end

#fit(x, y) ⇒ KernelSVC

Fit the model with given training data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_training_samples, n_training_samples]) The kernel matrix of the training data to be used for fitting the model.

  • y (Numo::Int32) —

    (shape: [n_training_samples]) The labels to be used for fitting the model.

Returns:

  • (KernelSVC) —

    The learned classifier itself.



61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 61

def fit(x, y)
  @classes = Numo::Int32[*y.to_a.uniq.sort]
  n_classes = @classes.size
  _n_samples, n_features = x.shape

  if n_classes > 2
    @weight_vec = Numo::DFloat.zeros(n_classes, n_features)
    n_classes.times do |n|
      bin_y = Numo::Int32.cast(y.eq(@classes[n])) * 2 - 1
      @weight_vec[n, true] = binary_fit(x, bin_y)
    end
  else
    negative_label = y.to_a.uniq.sort.first
    bin_y = Numo::Int32.cast(y.ne(negative_label)) * 2 - 1
    @weight_vec = binary_fit(x, bin_y)
  end

  self
end

#marshal_dump ⇒ Hash

Dump marshal data.

Returns:

  • (Hash) —

    The marshal data about KernelSVC.



115
116
117
118
119
120
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 115

def marshal_dump
  { params: @params,
    weight_vec: @weight_vec,
    classes: @classes,
    rng: @rng }
end

#marshal_load(obj) ⇒ nil

Load marshal data.

Returns:

  • (nil) 


124
125
126
127
128
129
130
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 124

def marshal_load(obj)
  @params = obj[:params]
  @weight_vec = obj[:weight_vec]
  @classes = obj[:classes]
  @rng = obj[:rng]
  nil
end

#predict(x) ⇒ Numo::Int32

Predict class labels for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_testing_samples, n_training_samples]) The kernel matrix between testing samples and training samples to predict the labels.

Returns:

  • (Numo::Int32) —

    (shape: [n_testing_samples]) Predicted class label per sample.



95
96
97
98
99
100
101
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 95

def predict(x)
  return Numo::Int32.cast(decision_function(x).ge(0.0)) * 2 - 1 if @classes.size <= 2

  n_samples, = x.shape
  decision_values = decision_function(x)
  Numo::Int32.asarray(Array.new(n_samples) { |n| @classes[decision_values[n, true].max_index] })
end

#score(x, y) ⇒ Float

Claculate the mean accuracy of the given testing data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_testing_samples, n_training_samples]) The kernel matrix between testing samples and training samples.

  • y (Numo::Int32) —

    (shape: [n_testing_samples]) True labels for testing data.

Returns:

  • (Float) —

    Mean accuracy



109
110
111
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 109

def score(x, y)
  super
end