Class: Rumale::Ensemble::ExtraTreesRegressor

Inherits:
RandomForestRegressor show all
Defined in:
lib/rumale/ensemble/extra_trees_regressor.rb

Overview

ExtraTreesRegressor is a class that implements extremely randomized trees for regression The algorithm of extremely randomized trees is similar to random forest. The features of the algorithm of extremely randomized trees are not to apply the bagging procedure and to randomly select the threshold for splitting feature space.

Reference

    1. Geurts, D. Ernst, and L. Wehenkel, “Extremely randomized trees,” Machine Learning, vol. 63 (1), pp. 3–42, 2006.

Examples:

estimator =
  Rumale::Ensemble::ExtraTreesRegressor.new(
    n_estimators: 10, criterion: 'mse', max_depth: 3, max_leaf_nodes: 10, min_samples_leaf: 5, random_seed: 1)
estimator.fit(training_samples, traininig_values)
results = estimator.predict(testing_samples)

Instance Attribute Summary collapse

Attributes included from Base::BaseEstimator

#params

Instance Method Summary collapse

Methods included from Base::Regressor

#score

Constructor Details

#initialize(n_estimators: 10, criterion: 'mse', max_depth: nil, max_leaf_nodes: nil, min_samples_leaf: 1, max_features: nil, random_seed: nil) ⇒ ExtraTreesRegressor

Create a new regressor with extremely randomized trees.



48
49
50
51
52
53
54
55
56
57
58
59
 
# File 'lib/rumale/ensemble/extra_trees_regressor.rb', line 48

def initialize(n_estimators: 10,
               criterion: 'mse', max_depth: nil, max_leaf_nodes: nil, min_samples_leaf: 1,
               max_features: nil, random_seed: nil)
  check_params_type_or_nil(Integer, max_depth: max_depth, max_leaf_nodes: max_leaf_nodes,
                                    max_features: max_features, random_seed: random_seed)
  check_params_integer(n_estimators: n_estimators, min_samples_leaf: min_samples_leaf)
  check_params_string(criterion: criterion)
  check_params_positive(n_estimators: n_estimators, max_depth: max_depth,
                        max_leaf_nodes: max_leaf_nodes, min_samples_leaf: min_samples_leaf,
                        max_features: max_features)
  super
end

Instance Attribute Details

#estimators ⇒ Array<ExtraTreeRegressor> (readonly)

Return the set of estimators.



25
26
27
 
# File 'lib/rumale/ensemble/extra_trees_regressor.rb', line 25

def estimators
  @estimators
end

#feature_importances ⇒ Numo::DFloat (readonly)

Return the importance for each feature.



29
30
31
 
# File 'lib/rumale/ensemble/extra_trees_regressor.rb', line 29

def feature_importances
  @feature_importances
end

#rng ⇒ Random (readonly)

Return the random generator for random selection of feature index.



33
34
35
 
# File 'lib/rumale/ensemble/extra_trees_regressor.rb', line 33

def rng
  @rng
end

Instance Method Details

#apply(x) ⇒ Numo::Int32

Return the index of the leaf that each sample reached.



103
104
105
106
 
# File 'lib/rumale/ensemble/extra_trees_regressor.rb', line 103

def apply(x)
  check_sample_array(x)
  super
end

#fit(x, y) ⇒ ExtraTreesRegressor

Fit the model with given training data.



66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
 
# File 'lib/rumale/ensemble/extra_trees_regressor.rb', line 66

def fit(x, y)
  check_sample_array(x)
  check_tvalue_array(y)
  check_sample_tvalue_size(x, y)
  # Initialize some variables.
  n_features = x.shape[1]
  @params[:max_features] = Math.sqrt(n_features).to_i unless @params[:max_features].is_a?(Integer)
  @params[:max_features] = [[1, @params[:max_features]].max, n_features].min
  @feature_importances = Numo::DFloat.zeros(n_features)
  # Construct forest.
  @estimators = Array.new(@params[:n_estimators]) do
    tree = Tree::ExtraTreeRegressor.new(
      criterion: @params[:criterion], max_depth: @params[:max_depth],
      max_leaf_nodes: @params[:max_leaf_nodes], min_samples_leaf: @params[:min_samples_leaf],
      max_features: @params[:max_features], random_seed: @rng.rand(Rumale::Values.int_max)
    )
    tree.fit(x, y)
    @feature_importances += tree.feature_importances
    tree
  end
  @feature_importances /= @feature_importances.sum
  self
end

#marshal_dump ⇒ Hash

Dump marshal data.



110
111
112
 
# File 'lib/rumale/ensemble/extra_trees_regressor.rb', line 110

def marshal_dump
  super
end

#marshal_load(obj) ⇒ nil

Load marshal data.



116
117
118
 
# File 'lib/rumale/ensemble/extra_trees_regressor.rb', line 116

def marshal_load(obj)
  super
end

#predict(x) ⇒ Numo::DFloat

Predict values for samples.



94
95
96
97
 
# File 'lib/rumale/ensemble/extra_trees_regressor.rb', line 94

def predict(x)
  check_sample_array(x)
  super
end