Initialization

 

Prior to receiving any inputs, the region is initialized by computing a list of initial potential synapses for each column. This consists of a random set of inputs selected from the input space. Each input is represented by a synapse and assigned a

random permanence value. The random permanence values are chosen with two criteria. First, the values are chosen to be in a small range around connectedPerm

(the minimum permanence value at which a synapse is considered "connected").

This enables potential synapses to become connected (or disconnected) after a small number of training iterations. Second, each column has a natural center over the input region, and the permanence values have a bias towards this center (they have higher values near the center).

Phase 1: Overlap

 

Given an input vector, the first phase calculates the overlap of each column with that vector. The overlap for each column is simply the number of connected synapses with active inputs, multiplied by its boost. If this value is below minOverlap, we set the overlap score to zero.

 

 

1. for c in columns

2.

3. overlap(c) = 0

4. for s in connectedSynapses(c)

5. overlap(c) = overlap(c) + input(t, s.sourceInput)

6.

7. if overlap(c) < minOverlap then

8. overlap(c) = 0

Else

10. overlap(c) = overlap(c) * boost(c)

 

Phase 2: Inhibition

 

The second phase calculates which columns remain as winners after the inhibition step. desiredLocalActivity is a parameter that controls the number of columns that end up winning. For example, if desiredLocalActivity is 10, a column will be a winner if its overlap score is greater than the score of the 10'th highest column within its inhibition radius.

 

 

11. for c in columns

12.

13. minLocalActivity = kthScore(neighbors(c), desiredLocalActivity)

14.

15. if overlap(c) > 0 and overlap(c) ≥ minLocalActivity then

16. activeColumns(t).append(c)

17.

Phase 3: Learning

 

The third phase performs learning; it updates the permanence values of all synapses as necessary, as well as the boost and inhibition radius.

 

The main learning rule is implemented in lines 20-26. For winning columns, if a synapse is active, its permanence value is incremented, otherwise it is decremented. Permanence values are constrained to be between 0 and 1.

 

Lines 28-36 implement boosting. There are two separate boosting mechanisms in place to help a column learn connections. If a column does not win often enough (as measured by activeDutyCycle), its overall boost value is increased (line 30-32). Alternatively, if a column's connected synapses do not overlap well with any inputs often enough (as measured by overlapDutyCycle), its permanence values are boosted (line 34-36). Note: once learning is turned off, boost(c) is frozen.

 

Finally, at the end of Phase 3 the inhibition radius is recomputed (line 38).

 

 

18. for c in activeColumns(t)

19.

20. for s in potentialSynapses(c)

21. if active(s) then

22. s.permanence += permanenceInc

23. s.permanence = min(1.0, s.permanence)

Else

25. s.permanence -= permanenceDec

26. s.permanence = max(0.0, s.permanence)

27.

28. for c in columns:

29.

30. minDutyCycle(c) = 0.01 * maxDutyCycle(neighbors(c))

31. activeDutyCycle(c) = updateActiveDutyCycle(c)

32. boost(c) = boostFunction(activeDutyCycle(c), minDutyCycle(c))

33.

34. overlapDutyCycle(c) = updateOverlapDutyCycle(c)

35. if overlapDutyCycle(c) < minDutyCycle(c) then

36. increasePermanences(c, 0.1*connectedPerm)

37.

38. inhibitionRadius = averageReceptiveFieldSize()

39.