Class: DNN::Layers::Connection

Inherits:
HasParamLayer show all
Defined in:
lib/dnn/core/layers.rb

Overview

It is a superclass of all connection layers.

Direct Known Subclasses

Conv2D, Dense, RNN

Instance Attribute Summary collapse

Attributes inherited from HasParamLayer

#params, #trainable

Attributes inherited from Layer

#input_shape

Instance Method Summary collapse

Methods inherited from HasParamLayer

#build, #update

Methods inherited from Layer

#backward, #build, #built?, #forward, #output_shape

Constructor Details

#initialize(weight_initializer: Initializers::RandomNormal.new, bias_initializer: Initializers::Zeros.new, l1_lambda: 0, l2_lambda: 0, use_bias: true) ⇒ Connection

Returns a new instance of Connection.

Parameters:

  • weight_initializer (DNN::Initializers) (defaults to: Initializers::RandomNormal.new) —

    weight initializer.

  • bias_initializer (DNN::Initializers) (defaults to: Initializers::Zeros.new) —

    bias initializer.

  • l1_lambda (Float) (defaults to: 0) —

    L1 regularization

  • l2_lambda (Float) (defaults to: 0) —

    L2 regularization

  • use_bias (Bool) (defaults to: true) —

    whether to use bias.



126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
 
# File 'lib/dnn/core/layers.rb', line 126

def initialize(weight_initializer: Initializers::RandomNormal.new,
               bias_initializer: Initializers::Zeros.new,
               l1_lambda: 0,
               l2_lambda: 0,
               use_bias: true)
  super()
  @weight_initializer = weight_initializer
  @bias_initializer = bias_initializer
  @l1_lambda = l1_lambda
  @l2_lambda = l2_lambda
  @params[:weight] = @weight = Param.new
  # For compatibility on or before with v0.9.3, setting use_bias to nil use bias.
  # Therefore, setting use_bias to nil is deprecated.
  if use_bias || use_bias == nil
    @params[:bias] = @bias = Param.new
  else
    @params[:bias] = @bias = nil
  end
end

Instance Attribute Details

#bias_initializer ⇒ DNN::Initializers (readonly)

Returns bias initializer.

Returns:



115
116
117
 
# File 'lib/dnn/core/layers.rb', line 115

def bias_initializer
  @bias_initializer
end

#l1_lambda ⇒ Float (readonly)

Returns L1 regularization.

Returns:

  • (Float) —

    L1 regularization.



117
118
119
 
# File 'lib/dnn/core/layers.rb', line 117

def l1_lambda
  @l1_lambda
end

#l2_lambda ⇒ Float (readonly)

Returns L2 regularization.

Returns:

  • (Float) —

    L2 regularization.



119
120
121
 
# File 'lib/dnn/core/layers.rb', line 119

def l2_lambda
  @l2_lambda
end

#weight_initializer ⇒ DNN::Initializers (readonly)

Returns weight initializer.

Returns:



113
114
115
 
# File 'lib/dnn/core/layers.rb', line 113

def weight_initializer
  @weight_initializer
end

Instance Method Details

#regularizers ⇒ Object 



146
147
148
149
150
151
 
# File 'lib/dnn/core/layers.rb', line 146

def regularizers
  regularizers = []
  regularizers << Lasso.new(@l1_lambda, @weight) if @l1_lambda > 0
  regularizers << Ridge.new(@l2_lambda, @weight) if @l2_lambda > 0
  regularizers
end

#to_hash(merge_hash) ⇒ Object 



158
159
160
161
162
163
 
# File 'lib/dnn/core/layers.rb', line 158

def to_hash(merge_hash)
  super({weight_initializer: @weight_initializer.to_hash,
         bias_initializer: @bias_initializer.to_hash,
         l1_lambda: @l1_lambda,
         l2_lambda: @l2_lambda}.merge(merge_hash))
end

#use_bias ⇒ Bool

Return whether to use bias.

Returns:

  • (Bool) —

    Return whether to use bias.



154
155
156
 
# File 'lib/dnn/core/layers.rb', line 154

def use_bias
  @bias ? true : false
end