Class: SvmToolkit::Evaluator

Inherits:
Object
  • Object
show all
Defined in:
lib/svm_toolkit/evaluators.rb

Overview

The Evaluator classes provides some classes and methods to construct classes for evaluating the performance of a model against a dataset.

Different evaluators measure different kinds of performance.

Evaluator classes are accessed by name, with an optional positive label name. For example:

Evaluator::OverallAccuracy # => class evaluates overall accuracy
Evaluator::ClassPrecision(label) # => class evaluates precision for class "label"

Evaluators are wrapped around confusion matrices, outputting the required statistical measure, and support the following methods:

add_result(actual, prediction)

called to add information about each instance when testing a model.

value

retrieves the appropriate measure of performance, based on the class name.

to_s

returns a string naming the evaluator and giving its value.

Direct Known Subclasses

GeometricMean, OverallAccuracy

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Evaluator

Creates a new Evaluator, with a confusion matrix to store results.



112
113
114
 
# File 'lib/svm_toolkit/evaluators.rb', line 112

def initialize
  @cm = ConfusionMatrix.new
end

Class Method Details

.ClassPrecision(label) ⇒ Object

Defines an Evaluator returning the value of precision for given class label.



26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
 
# File 'lib/svm_toolkit/evaluators.rb', line 26

def Evaluator.ClassPrecision label
  Class.new(Evaluator) do 
    @@label = label

    # Returns the precision.

    #

    def value
      @cm.precision(@@label)
    end

    def to_s # :nodoc:

      "Precision for label #{@@label}: #{value}"
    end
  end
end

.ClassRecall(label) ⇒ Object

Defines an Evaluator returning the value of recall for given class label.



45
46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'lib/svm_toolkit/evaluators.rb', line 45

def Evaluator.ClassRecall label
  Class.new(Evaluator) do 
    @@label = label

    def value # :nodoc:

      @cm.recall(@@label)
    end

    def to_s # :nodoc:

      "Recall for label #{@@label}: #{value}"
    end
  end
end

.FMeasure(label) ⇒ Object

Defines an Evaluator returning the value of the F-measure for given class label.



62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'lib/svm_toolkit/evaluators.rb', line 62

def Evaluator.FMeasure label
  Class.new(Evaluator) do 
    @@label = label

    def value # :nodoc:

      @cm.f_measure(@@label)
    end

    def to_s # :nodoc:

      "F-measure for label #{@@label}: #{value}"
    end
  end
end

.Kappa(label) ⇒ Object

Defines an Evaluator returning the value of Cohen’s Kappa statistics for given class label.



79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/svm_toolkit/evaluators.rb', line 79

def Evaluator.Kappa label
  Class.new(Evaluator) do 
    @@label = label

    def value # :nodoc:

      @cm.kappa(@@label)
    end

    def to_s # :nodoc:

      "Kappa for label #{@@label}: #{value}"
    end
  end
end

.MatthewsCorrelationCoefficient(label) ⇒ Object

Defines an Evaluator returning the value of the Matthews Correlation Coefficient for given class label.



96
97
98
99
100
101
102
103
104
105
106
107
108
 
# File 'lib/svm_toolkit/evaluators.rb', line 96

def Evaluator.MatthewsCorrelationCoefficient label
  Class.new(Evaluator) do 
    @@label = label

    def value # :nodoc:

      @cm.matthews_correlation(@@label)
    end

    def to_s # :nodoc:

      "Matthews correlation coefficient: #{value}"
    end
  end
end

Instance Method Details

#add_result(actual, prediction) ⇒ Object

Adds result to the underlying confusion matrix.



118
119
120
 
# File 'lib/svm_toolkit/evaluators.rb', line 118

def add_result(actual, prediction)
  @cm.add_for(actual, prediction)
end

#better_than?(other) ⇒ Boolean

This object is better than given object, if the given object is an instance of nil, or the value of this object is better.



125
126
127
 
# File 'lib/svm_toolkit/evaluators.rb', line 125

def better_than? other
  other.nil? or self.value > other.value
end

#display ⇒ Object

Prints the confusion matrix.



131
132
133
 
# File 'lib/svm_toolkit/evaluators.rb', line 131

def display
  puts @cm
end