Class: SVMKit::KernelMachine::KernelSVC

Inherits:
Object
  • Object
show all
Includes:
Base::BaseEstimator, Base::Classifier
Defined in:
lib/svmkit/kernel_machine/kernel_svc.rb

Overview

KernelSVC is a class that implements (Nonlinear) Kernel Support Vector Classifier with the Pegasos algorithm.

Reference

    1. Shalev-Shwartz, Y. Singer, N. Srebro, and A. Cotter, “Pegasos: Primal Estimated sub-GrAdient SOlver for SVM,” Mathematical Programming, vol. 127 (1), pp. 3–30, 2011.

Examples:

training_kernel_matrix = SVMKit::PairwiseMetric::rbf_kernel(training_samples)
estimator =
  SVMKit::KernelMachine::KernelSVC.new(reg_param: 1.0, max_iter: 1000, random_seed: 1)
estimator.fit(training_kernel_matrix, traininig_labels)
testing_kernel_matrix = SVMKit::PairwiseMetric::rbf_kernel(testing_samples, training_samples)
results = estimator.predict(testing_kernel_matrix)

Instance Attribute Summary collapse

Attributes included from Base::BaseEstimator

#params

Instance Method Summary collapse

Constructor Details

#initialize(reg_param: 1.0, max_iter: 1000, random_seed: nil) ⇒ KernelSVC

Create a new classifier with Kernel Support Vector Machine by the Pegasos algorithm.

Parameters:

  • reg_param (Float) (defaults to: 1.0) —

    The regularization parameter.

  • max_iter (Integer) (defaults to: 1000) —

    The maximum number of iterations.

  • random_seed (Integer) (defaults to: nil) —

    The seed value using to initialize the random generator.



36
37
38
39
40
41
42
43
44
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 36

def initialize(reg_param: 1.0, max_iter: 1000, random_seed: nil)
  @params = {}
  @params[:reg_param] = reg_param
  @params[:max_iter] = max_iter
  @params[:random_seed] = random_seed
  @params[:random_seed] ||= srand
  @weight_vec = nil
  @rng = Random.new(@params[:random_seed])
end

Instance Attribute Details

#rng ⇒ Random (readonly)

Return the random generator for performing random sampling in the Pegasos algorithm.

Returns:

  • (Random) 


29
30
31
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 29

def rng
  @rng
end

#weight_vec ⇒ Numo::DFloat (readonly)

Return the weight vector for Kernel SVC.

Returns:

  • (Numo::DFloat) —

    (shape: [n_trainig_sample])



25
26
27
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 25

def weight_vec
  @weight_vec
end

Instance Method Details

#decision_function(x) ⇒ Numo::DFloat

Calculate confidence scores for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_testing_samples, n_training_samples]) The kernel matrix between testing samples and training samples to compute the scores.

Returns:

  • (Numo::DFloat) —

    (shape: [n_testing_samples]) Confidence score per sample.



80
81
82
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 80

def decision_function(x)
  @weight_vec.dot(x.transpose)
end

#fit(x, y) ⇒ KernelSVC

Fit the model with given training data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_training_samples, n_training_samples]) The kernel matrix of the training data to be used for fitting the model.

  • y (Numo::Int32) —

    (shape: [n_training_samples]) The labels to be used for fitting the model.

Returns:

  • (KernelSVC) —

    The learned classifier itself.



52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 52

def fit(x, y)
  # Generate binary labels
  negative_label = y.to_a.uniq.sort.shift
  bin_y = y.to_a.map { |l| l != negative_label ? 1 : -1 }
  # Initialize some variables.
  n_training_samples = x.shape[0]
  rand_ids = []
  weight_vec = Numo::DFloat.zeros(n_training_samples)
  # Start optimization.
  @params[:max_iter].times do |t|
    # random sampling
    rand_ids = [*0...n_training_samples].shuffle(random: @rng) if rand_ids.empty?
    target_id = rand_ids.shift
    # update the weight vector
    func = (weight_vec * bin_y[target_id]).dot(x[target_id, true].transpose).to_f
    func *= bin_y[target_id] / (@params[:reg_param] * (t + 1))
    weight_vec[target_id] += 1.0 if func < 1.0
  end
  # Store the learned model.
  @weight_vec = weight_vec * Numo::DFloat.asarray(bin_y)
  self
end

#marshal_dump ⇒ Hash

Dump marshal data.

Returns:

  • (Hash) —

    The marshal data about KernelSVC.



107
108
109
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 107

def marshal_dump
  { params: @params, weight_vec: @weight_vec, rng: @rng }
end

#marshal_load(obj) ⇒ nil

Load marshal data.

Returns:

  • (nil) 


113
114
115
116
117
118
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 113

def marshal_load(obj)
  @params = obj[:params]
  @weight_vec = obj[:weight_vec]
  @rng = obj[:rng]
  nil
end

#predict(x) ⇒ Numo::Int32

Predict class labels for samples.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_testing_samples, n_training_samples]) The kernel matrix between testing samples and training samples to predict the labels.

Returns:

  • (Numo::Int32) —

    (shape: [n_testing_samples]) Predicted class label per sample.



89
90
91
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 89

def predict(x)
  Numo::Int32.cast(decision_function(x).map { |v| v >= 0 ? 1 : -1 })
end

#score(x, y) ⇒ Float

Claculate the mean accuracy of the given testing data.

Parameters:

  • x (Numo::DFloat) —

    (shape: [n_testing_samples, n_training_samples]) The kernel matrix between testing samples and training samples.

  • y (Numo::Int32) —

    (shape: [n_testing_samples]) True labels for testing data.

Returns:

  • (Float) —

    Mean accuracy



99
100
101
102
103
 
# File 'lib/svmkit/kernel_machine/kernel_svc.rb', line 99

def score(x, y)
  p = predict(x)
  n_hits = (y.to_a.map.with_index { |l, n| l == p[n] ? 1 : 0 }).inject(:+)
  n_hits / y.size.to_f
end