Supporting data structures and routines

 

The following variables and data structures are used in the pseudocode:

 

columns List of all columns.

 

input(t,j) The input to this level at time t. input(t, j) is 1 if the j'th input is on.

 

overlap(c) The spatial pooler overlap of column c with a particular input pattern.

 

activeColumns(t) List of column indices that are winners due to bottom-up input.

 

desiredLocalActivity A parameter controlling the number of columns that will be winners after the inhibition step.

 

inhibitionRadius Average connected receptive field size of the columns.

 

neighbors(c) A list of all the columns that are within inhibitionRadius of column c.

 

minOverlap A minimum number of inputs that must be active for a column to be considered during the inhibition step.

 

boost(c) The boost value for column c as computed during learning - used to increase the overlap value for inactive columns.

 

synapse A data structure representing a synapse - contains a permanence value and the source input index.

 

connectedPerm If the permanence value for a synapse is greater than this value, it is said to be connected.

 

potentialSynapses(c) The list of potential synapses and their permanence values.

 

connectedSynapses(c) A subset of potentialSynapses(c) where the permanence value is greater than connectedPerm. These are the

bottom-up inputs that are currently connected to column c.

 

permanenceInc Amount permanence values of synapses are incremented during learning.

 

permanenceDec Amount permanence values of synapses are decremented during learning.

 

activeDutyCycle(c) A sliding average representing how often column c has been active after inhibition (e.g. over the last 1000 iterations).

overlapDutyCycle(c) A sliding average representing how often column c has had significant overlap (i.e. greater than minOverlap) with its inputs (e.g. over the last 1000 iterations).

 

minDutyCycle(c) A variable representing the minimum desired firing rate for a cell. If a cell's firing rate falls below this value, it will be boosted. This value is calculated as 1% of the maximum firing rate of its neighbors.

 

 

The following supporting routines are used in the above code.

 

kthScore(cols, k)

Given the list of columns, return the k'th highest overlap value.

 

 

updateActiveDutyCycle(c)

Computes a moving average of how often column c has been active after inhibition.

 

updateOverlapDutyCycle(c)

Computes a moving average of how often column c has overlap greater than minOverlap.

 

averageReceptiveFieldSize()

The radius of the average connected receptive field size of all the columns. The connected receptive field size of a column includes only the connected synapses (those with permanence values >= connectedPerm). This is used to determine the extent of lateral inhibition between columns.

 

maxDutyCycle(cols)

Returns the maximum active duty cycle of the columns in the given list of columns.

 

increasePermanences(c, s)

Increase the permanence value of every synapse in column c by a scale factor s.

 

boostFunction(c)

Returns the boost value of a column. The boost value is a scalar >= 1. If activeDutyCyle(c) is above minDutyCycle(c), the boost value is 1. The boost increases linearly once the column's activeDutyCyle starts falling below its minDutyCycle.

 

 

Chapter 4: Temporal Pooling Implementation and Pseudocode

 

This chapter contains the detailed pseudocode for a first implementation of the temporal pooler function. The input to this code is activeColumns(t), as computed by the spatial pooler. The code computes the active and predictive state for each

cell at the current timestep, t. The boolean OR of the active and predictive states for each cell forms the output of the temporal pooler for the next level.

The pseudocode is split into three distinct phases that occur in sequence: Phase 1: compute the active state, activeState(t), for each cell

Phase 2: compute the predicted state, predictiveState(t), for each cell

Phase 3: update synapses

 

Phase 3 is only required for learning. However, unlike spatial pooling, Phases 1 and

2 contain some learning-specific operations when learning is turned on. Since temporal pooling is significantly more complicated than spatial pooling, we first list

the inference-only version of the temporal pooler, followed by a version that

combines inference and learning. A description of some of the implementation details, terminology, and supporting routines are at the end of the chapter, after the pseudocode.

Temporal pooler pseudocode: inference alone