Class: MachineLearningWorkbench::NeuralNetwork::Recurrent

Inherits:
Base
  • Object
show all
Defined in:
lib/machine_learning_workbench/neural_network/recurrent.rb

Overview

Recurrent Neural Network

Instance Attribute Summary

Attributes inherited from Base

#act_fn, #layers, #state, #struct

Instance Method Summary collapse

Methods inherited from Base

act_fn, #activate, #bias, #deep_reset, #init_random, #initialize, #interface_methods, #layer_col_sizes, #layer_shapes, lecun_hyperbolic, #load_weights, logistic, #nlayers, #nneurs, #nweights, #nweights_per_layer, #out, #reset_state, sigmoid, #weights

Constructor Details

This class inherits a constructor from MachineLearningWorkbench::NeuralNetwork::Base

Instance Method Details

#activate_layer(nlay) ⇒ Object

Activates a layer of the network. Bit more complex since it has to copy the layer’s activation on last input to its own inputs, for recursion.

Parameters:

  • i (Integer) —

    the layer to activate, zero-indexed



21
22
23
24
25
26
27
28
29
30
31
32
 
# File 'lib/machine_learning_workbench/neural_network/recurrent.rb', line 21

def activate_layer nlay #_layer
  # NOTE: current layer index corresponds to index of next state!
  previous = nlay     # index of previous layer (inputs)
  current = nlay + 1  # index of current layer (outputs)
  # Copy the level's last-time activation to the input (previous state)
  # NOTE: ranges in NMatrix#[] not reliable! gotta loop :(
  nneurs(current).times do |i| # for each activations to copy
    # Copy output from last-time activation to recurrency in previous state
    @state[previous][0, nneurs(previous) + i] = state[current][0, i]
  end
  act_fn.call state[previous].dot layers[nlay]
end

#layer_row_sizes ⇒ Array<Integer>

Calculate the size of each row in a layer’s weight matrix. Each row holds the inputs for the next level: previous level’s activations (or inputs), this level’s last activations (recursion) and bias.

Returns:

  • (Array<Integer>) —

    per-layer row sizes



11
12
13
14
15
 
# File 'lib/machine_learning_workbench/neural_network/recurrent.rb', line 11

def layer_row_sizes
  @layer_row_sizes ||= struct.each_cons(2).collect do |prev, rec|
    prev + rec + 1
  end
end