Class: TensorStream::Train::MomentumOptimizer

Inherits:

Optimizer

• Object
• Optimizer
• TensorStream::Train::MomentumOptimizer

Includes:

OpHelper

Defined in:

lib/tensor_stream/train/momentum_optimizer.rb

Overview

Optimizer that implements the Momentum algorithm. loosely based on the tensorflow implementation.

Instance Attribute Summary

Attributes inherited from Optimizer

#name

Instance Method Summary

• #initialize(learning_rate, momentum, name: "momentum", use_nesterov: false, use_locking: false) ⇒ MomentumOptimizer constructor

  Construct a new Momentum optimizer.

Methods included from OpHelper

#_op, #cons, #format_source, #fp_type?, #i_cons, #i_op, #i_var, #int_type?, #reduced_shape, #shape_eval, #shape_full_specified, #shapes_fully_specified_and_equal

Methods inherited from Optimizer

#apply_gradients, #compute_gradients, #get_slot, #get_slot_names, #minimize

Methods included from SlotCreator

#create_slot, #create_slot_var, #create_slot_with_initializer, #create_zeros_slot

Methods included from Utils

#__v_scope_name, #apply_data_type_coercion, #assign, #check_allowed_types, #check_data_types, #check_if_dense, #colocate_with, #constant, #control_dependencies, #convert_to_tensor, #device, #disable_eager_execution, #dynamic_stitch, #enable_eager_execution, #executing_eagerly?, #float32, #get_collection, #get_default_graph, #get_variable, #get_variable_scope, #global_variables_initializer, #graph, #group, #image, #layers, #list_local_devices, #math, #name_scope, #placeholder, #program, #reset_default_graph, #session, #set_random_seed, #train, #trainable_variables, #variable, #variable_scope

Constructor Details

#initialize(learning_rate, momentum, name: "momentum", use_nesterov: false, use_locking: false) ⇒ MomentumOptimizer

Construct a new Momentum optimizer.

Args:

learning_rate: A Tensor or a floating point value that indicates the learning rate
momentum: A Tensor or a floating point value for the momentum
name: Optional name prefix
use_nesterov: boolean - Flag that indicates if nesterov momentum is to be used. http://jmlr.org/proceedings/papers/v28/sutskever13.pdf
use_locking: boolean - filler argument for compatibility, not used at the moment

# File 'lib/tensor_stream/train/momentum_optimizer.rb', line 16

def initialize(learning_rate, momentum, name: "momentum", use_nesterov: false, use_locking: false)
  @learning_rate = learning_rate
  @momentum = momentum
  @use_nesterov = use_nesterov
  super(name: name, use_locking: use_locking)
end