Class: TensorStream::Train::RMSPropOptimizer

Inherits:
Optimizer
  • Object
show all
Includes:
OpHelper
Defined in:
lib/tensor_stream/train/rmsprop_optimizer.rb

Overview

High Level implementation of the RMSProp algorithm This is a straight port from TensorFlows rmsprop.py

Instance Attribute Summary collapse

Attributes inherited from Optimizer

#name

Instance Method Summary collapse

Methods included from OpHelper

#_op, #cons, #format_source, #fp_type?, #i_cons, #i_op, #i_var, #int_type?, #reduced_shape, #shape_eval, #shape_full_specified, #shapes_fully_specified_and_equal

Methods inherited from Optimizer

#apply_gradients, #compute_gradients, #get_slot, #get_slot_names, #minimize

Methods included from SlotCreator

#create_slot, #create_slot_var, #create_slot_with_initializer, #create_zeros_slot

Methods included from Utils

#__v_scope_name, #apply_data_type_coercion, #assign, #check_allowed_types, #check_data_types, #check_if_dense, #colocate_with, #constant, #control_dependencies, #convert_to_tensor, #device, #disable_eager_execution, #dynamic_stitch, #enable_eager_execution, #executing_eagerly?, #float32, #get_collection, #get_default_graph, #get_variable, #get_variable_scope, #global_variables_initializer, #graph, #group, #image, #layers, #list_local_devices, #math, #name_scope, #placeholder, #program, #reset_default_graph, #session, #set_random_seed, #train, #trainable_variables, #variable, #variable_scope

Constructor Details

#initialize(learning_rate, decay = 0.9, momentum = 0.0, epsilon = 1e-10, centered: false, use_locking: false, name: "RMSProp") ⇒ RMSPropOptimizer

Optimizer that implements the RMSProp algorithm.

[paper](www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf).



14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
 
# File 'lib/tensor_stream/train/rmsprop_optimizer.rb', line 14

def initialize(learning_rate, decay = 0.9, momentum = 0.0, epsilon = 1e-10, centered: false,
  use_locking: false, name: "RMSProp")
  @learning_rate = learning_rate
  @decay = decay
  @momentum = momentum
  @epsilon = epsilon
  @centered = centered

  # Tensor versions of the constructor arguments, created in _prepare().
  @learning_rate_tensor = nil
  @decay_tensor = nil
  @momentum_tensor = nil
  @epsilon_tensor = nil

  super(name: name, use_locking: use_locking)
end

Instance Attribute Details

#learning_rate ⇒ Object

Returns the value of attribute learning_rate.



8
9
10
 
# File 'lib/tensor_stream/train/rmsprop_optimizer.rb', line 8

def learning_rate
  @learning_rate
end