Class: Rumale::LinearModel::SVC
- Inherits:
-
BaseLinearModel
- Object
- BaseLinearModel
- Rumale::LinearModel::SVC
- Includes:
- Base::Classifier
- Defined in:
- lib/rumale/linear_model/svc.rb
Overview
SVC is a class that implements Support Vector Classifier with mini-batch stochastic gradient descent optimization. For multiclass classification problem, it uses one-vs-the-rest strategy.
Reference
-
Shalev-Shwartz and Y. Singer, “Pegasos: Primal Estimated sub-GrAdient SOlver for SVM,” Proc. ICML’07, pp. 807–814, 2007.
-
Instance Attribute Summary collapse
-
#bias_term ⇒ Numo::DFloat
readonly
Return the bias term (a.k.a. intercept) for SVC.
-
#classes ⇒ Numo::Int32
readonly
Return the class labels.
-
#rng ⇒ Random
readonly
Return the random generator for performing random sampling.
-
#weight_vec ⇒ Numo::DFloat
readonly
Return the weight vector for SVC.
Attributes included from Base::BaseEstimator
Instance Method Summary collapse
-
#decision_function(x) ⇒ Numo::DFloat
Calculate confidence scores for samples.
-
#fit(x, y) ⇒ SVC
Fit the model with given training data.
-
#initialize(reg_param: 1.0, fit_bias: false, bias_scale: 1.0, max_iter: 1000, batch_size: 20, probability: false, optimizer: nil, random_seed: nil) ⇒ SVC
constructor
Create a new classifier with Support Vector Machine by the SGD optimization.
-
#marshal_dump ⇒ Hash
Dump marshal data.
-
#marshal_load(obj) ⇒ nil
Load marshal data.
-
#predict(x) ⇒ Numo::Int32
Predict class labels for samples.
-
#predict_proba(x) ⇒ Numo::DFloat
Predict probability for samples.
Methods included from Base::Classifier
Constructor Details
#initialize(reg_param: 1.0, fit_bias: false, bias_scale: 1.0, max_iter: 1000, batch_size: 20, probability: false, optimizer: nil, random_seed: nil) ⇒ SVC
Create a new classifier with Support Vector Machine by the SGD optimization.
52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
# File 'lib/rumale/linear_model/svc.rb', line 52 def initialize(reg_param: 1.0, fit_bias: false, bias_scale: 1.0, max_iter: 1000, batch_size: 20, probability: false, optimizer: nil, random_seed: nil) check_params_float(reg_param: reg_param, bias_scale: bias_scale) check_params_integer(max_iter: max_iter, batch_size: batch_size) check_params_boolean(fit_bias: fit_bias, probability: probability) check_params_type_or_nil(Integer, random_seed: random_seed) check_params_positive(reg_param: reg_param, bias_scale: bias_scale, max_iter: max_iter, batch_size: batch_size) keywd_args = method(:initialize).parameters.map { |_t, arg| [arg, binding.local_variable_get(arg)] }.to_h keywd_args.delete(:probability) super(keywd_args) @params[:probability] = probability @prob_param = nil @classes = nil end |
Instance Attribute Details
#bias_term ⇒ Numo::DFloat (readonly)
Return the bias term (a.k.a. intercept) for SVC.
31 32 33 |
# File 'lib/rumale/linear_model/svc.rb', line 31 def bias_term @bias_term end |
#classes ⇒ Numo::Int32 (readonly)
Return the class labels.
35 36 37 |
# File 'lib/rumale/linear_model/svc.rb', line 35 def classes @classes end |
#rng ⇒ Random (readonly)
Return the random generator for performing random sampling.
39 40 41 |
# File 'lib/rumale/linear_model/svc.rb', line 39 def rng @rng end |
#weight_vec ⇒ Numo::DFloat (readonly)
Return the weight vector for SVC.
27 28 29 |
# File 'lib/rumale/linear_model/svc.rb', line 27 def weight_vec @weight_vec end |
Instance Method Details
#decision_function(x) ⇒ Numo::DFloat
Calculate confidence scores for samples.
112 113 114 115 |
# File 'lib/rumale/linear_model/svc.rb', line 112 def decision_function(x) check_sample_array(x) x.dot(@weight_vec.transpose) + @bias_term end |
#fit(x, y) ⇒ SVC
Fit the model with given training data.
72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
# File 'lib/rumale/linear_model/svc.rb', line 72 def fit(x, y) check_sample_array(x) check_label_array(y) check_sample_label_size(x, y) @classes = Numo::Int32[*y.to_a.uniq.sort] n_classes = @classes.size n_features = x.shape[1] if n_classes > 2 @weight_vec = Numo::DFloat.zeros(n_classes, n_features) @bias_term = Numo::DFloat.zeros(n_classes) @prob_param = Numo::DFloat.zeros(n_classes, 2) n_classes.times do |n| bin_y = Numo::Int32.cast(y.eq(@classes[n])) * 2 - 1 @weight_vec[n, true], @bias_term[n] = partial_fit(x, bin_y) @prob_param[n, true] = if @params[:probability] Rumale::ProbabilisticOutput.fit_sigmoid(x.dot(@weight_vec[n, true].transpose) + @bias_term[n], bin_y) else Numo::DFloat[1, 0] end end else negative_label = y.to_a.uniq.min bin_y = Numo::Int32.cast(y.ne(negative_label)) * 2 - 1 @weight_vec, @bias_term = partial_fit(x, bin_y) @prob_param = if @params[:probability] Rumale::ProbabilisticOutput.fit_sigmoid(x.dot(@weight_vec.transpose) + @bias_term, bin_y) else Numo::DFloat[1, 0] end end self end |
#marshal_dump ⇒ Hash
Dump marshal data.
152 153 154 155 156 157 158 159 |
# File 'lib/rumale/linear_model/svc.rb', line 152 def marshal_dump { params: @params, weight_vec: @weight_vec, bias_term: @bias_term, prob_param: @prob_param, classes: @classes, rng: @rng } end |
#marshal_load(obj) ⇒ nil
Load marshal data.
163 164 165 166 167 168 169 170 171 |
# File 'lib/rumale/linear_model/svc.rb', line 163 def marshal_load(obj) @params = obj[:params] @weight_vec = obj[:weight_vec] @bias_term = obj[:bias_term] @prob_param = obj[:prob_param] @classes = obj[:classes] @rng = obj[:rng] nil end |
#predict(x) ⇒ Numo::Int32
Predict class labels for samples.
121 122 123 124 125 126 127 128 129 |
# File 'lib/rumale/linear_model/svc.rb', line 121 def predict(x) check_sample_array(x) return Numo::Int32.cast(decision_function(x).ge(0.0)) * 2 - 1 if @classes.size <= 2 n_samples, = x.shape decision_values = decision_function(x) Numo::Int32.asarray(Array.new(n_samples) { |n| @classes[decision_values[n, true].max_index] }) end |
#predict_proba(x) ⇒ Numo::DFloat
Predict probability for samples.
135 136 137 138 139 140 141 142 143 144 145 146 147 148 |
# File 'lib/rumale/linear_model/svc.rb', line 135 def predict_proba(x) check_sample_array(x) if @classes.size > 2 probs = 1.0 / (Numo::NMath.exp(@prob_param[true, 0] * decision_function(x) + @prob_param[true, 1]) + 1.0) return (probs.transpose / probs.sum(axis: 1)).transpose end n_samples, = x.shape probs = Numo::DFloat.zeros(n_samples, 2) probs[true, 1] = 1.0 / (Numo::NMath.exp(@prob_param[0] * decision_function(x) + @prob_param[1]) + 1.0) probs[true, 0] = 1.0 - probs[true, 1] probs end |