Request for Shen help (lengthy)

Antti Ylikoski

unread,

Sep 13, 2016, 2:52:57 AM9/13/16

to Shen

I wrote this to ask from the Shen group for some help with a "mystic

behavior" problem. It concerns the error backpropagation program.

I attach a sample run. In the sample run, the inputs of the Neurons

"A" and "S" have been set at 0.66 at the beginning of the computation;

those values have been set with the (make-pctron ...) function calls

in the pctronml.shen source file.

I decided to use the Shen Ruby for this purpose, since it is rather

"efficient", and running 600 training rounds, which might happen in

an industrial application, does not demand very much CPU time.

In the sample run, I execute and time 600 training rounds for the ANN.

After that, the sample run contains some (describe-neuron ...) calls.

Those calls show that after the computation, the inputs of Neurons "A"

and "S" still have their originally set values 0.66. This ought to be

impossible.

Those input values should have been reset thousands of times. It

should be impossible for them to still remain at the numerical value

of 0.66, after the computation.

I have been wondering about this "mystic behavior" problem for several

days. Otherwise, the error backpropagation program is very close to

being finished and operative.

I would be very grateful for the Shen group if they could help a bit

about this problem, as experienced Shen professionals, which I'm not.

yours, Dr Antti J Ylikoski

HELSINKI

Finland, the E.U.

mystic.sample.run.txt

pctronml.shen

Antti Ylikoski

unread,

Sep 16, 2016, 1:14:48 AM9/16/16

to Shen

I bought for myself a new laptop, a used IBM Thinkpad, and discovered

that the error backpropagation program is so big, that it will exceed

the capabilities of the 32-bit Linux Ruby system.

The attempt to load the backpropagation program with the 32-bit Linux

Ruby will crash the Ruby system at a stack overflow, as seen below.

The backpropagation program will load with the CL Shen, and it will

load with the 64-bit Linux Shen Ruby.

This made me think that maybe the "mystical" error I discussed, might

be caused by too heavily straining the capabilities of the underlying

language or hardware systems, on which ones the backpropagation

program has been constructed.

I'm essentially sure that the OS Shen system is perfectly OK and must

not be blamed.

------------------------------------------------------------

antti@antti-ThinkPad-T43:~/pctronm$ srrepl

Loading.... Completed in 8.86 seconds.

www.shenlanguage.org, Shen 18.1

running under Ruby, implementation: ruby 1.9.3

port 0.15.1 ported by Greg Spurrier

(0-) (tc +)

true

(1+) (load "maths-lib1.shen")

sign : (number --> number)

abs : (number --> number)

floor : (number --> number)

maths.floor-pos : (number --> number)

maths.floor-neg : (number --> number)

maths.floor-h : (number --> (number --> (number --> number)))

ceiling : (number --> number)

trunc : (number --> number)

maths-round0 : (number --> number)

maths.round-down? : (number --> boolean)

int-part : (number --> number)

frac-part : (number --> number)

modf : (number --> (number * number))

maths-round' : (number --> (number --> number))

maths.pow-2 : (number --> (number --> number))

fmod : (number --> (number --> number))

frexp : (number --> (number * number))

maths.frexp-neg : (number --> (number * number))

maths.frexp-pos : (number --> (number * number))

maths.mult-2 : (number --> (number --> (number * number)))

maths.div-2 : (number --> (number --> (number * number)))

ldexp : (number --> (number --> number))

square : (number --> number)

maths.power-pos : (number --> (number --> number))

power : (number --> (number --> number))

maths.type#global : symbol

1.0e-15 : number

2.7182818284590455 : number

2.302585092994046 : number

0.6931471805599454 : number

3.141592653589793 : number

1.5707963267948966 : number

0.7853981633974484 : number

6.283185307179586 : number

0.31830988618379075 : number

0.6366197723675814 : number

1.4426950408889634 : number

0.43429448190325187 : number

1.4142135623730951 : number

0.7071067811865476 : number

1.1283791670955126 : number

0.017453292519943295 : number

57.29577951308232 : number

maths.small-enough? : (number --> boolean)

rad->degs : (number --> number)

degs->rad : (number --> number)

dms->degs : ((list number) --> number)

maths.dms->degs-pos : ((list number) --> number)

degs->dms : (number --> (list number))

maths.range-ok? : (number --> boolean)

exp : (number --> number)

maths.exp-large : (number --> number)

maths.exp-h : (number --> number)

maths.exp-sum : (number --> (number --> (number --> (number --> number))))

sinh : (number --> number)

cosh : (number --> number)

tanh : (number --> number)

expt : (number --> (number --> number))

sqrt : (number --> number)

maths.sqrt-scale : (number --> (number --> number))

maths.sqrt-h : (number --> number)

maths.sqrt-iter : (number --> (number --> (number --> number)))

maths.mean : (number --> (number --> number))

log : (number --> number)

maths.log-scale : (number --> (number --> number))

maths.log-h : (number --> number)

maths.log-sum : (number --> (number --> (number --> (number --> number))))

log10 : (number --> number)

log2 : (number --> number)

log' : (number --> (number --> number))

sin : (number --> number)

maths.sin-h : (number --> number)

maths.sin-sum : (number --> (number --> (number --> (number --> (number --> number)))))

cos : (number --> number)

tan : (number --> number)

asin : (number --> number)

maths.asin-h : (number --> number)

maths.asin-sum : (number --> (number --> (number --> (number --> number))))

acos : (number --> number)

atan : (number --> number)

maths.atan-h : (number --> number)

maths.atan-sum : (number --> (number --> (number --> (number --> (number --> number)))))

maths.atan-lt1 : (number --> number)

maths.atan-gt1 : (number --> number)

maths.atan-transf : (number --> number)

atan2 : (number --> (number --> number))

even? : (number --> boolean)

odd? : (number --> boolean)

natural? : (number --> boolean)

positive? : (number --> boolean)

negative? : (number --> boolean)

zero? : (number --> boolean)

maths.rsh : (number --> number)

maths.rsh-h : (number --> (number --> number))

maths./-pos : (number --> (number --> (number * number)))

maths.div-w : (number --> (number --> (number --> (number --> (number * number)))))

maths.pow-2div : (number --> (number --> (number --> number)))

divisible-by? : (number --> (number --> boolean))

maths.mod-pos : (number --> (number --> number))

maths.div-ww : (number --> (number --> (number --> number)))

/mod : (number --> (number --> (number * number)))

maths./mod-h : (number --> (number --> (number --> (number --> (number --> (number * number))))))

div : (number --> (number --> number))

maths.div-h : (number --> (number --> (number --> (number --> number))))

mod : (number --> (number --> number))

maths.mod-h : (number --> (number --> (number --> (number --> number))))

/rem : (number --> (number --> (number * number)))

maths./rem-h : (number --> (number --> (number --> (number --> (number * number)))))

trunc-div : (number --> (number --> number))

rem : (number --> (number --> number))

/% : (number --> (number --> (number * number)))

maths./%-h : (number --> (number --> (number --> (number --> (number --> (number * number))))))

div-eucl : (number --> (number --> number))

maths.adjust-Q : (number --> (number --> (number --> number)))

% : (number --> (number --> number))

gcd : (number --> (number --> number))

lcm : (number --> (number --> number))

!= : (A --> (A --> boolean))

max : (number --> (number --> number))

min : (number --> (number --> number))

run time: 28.3385009765625 secs

typechecked in 30115 inferences

loaded : symbol

(2+) (load "pctronml.shen")

type#pctron : symbol

make-pctron : (string --> (string --> ((list (number * string)) --> ((list (number * string)) --> ((list (number * string)) --> ((list (number * string)) --> (number --> (number --> (number --> ((list string) --> pctron))))))))))

pctronNeuronName : (pctron --> string)

pctronDocumentation : (pctron --> string)

pctronInputs : (pctron --> (list (number * string)))

pctronWeights : (pctron --> (list (number * string)))

pctronDeltas : (pctron --> (list (number * string)))

pctronSumDeltas : (pctron --> (list (number * string)))

pctronActivationLevel : (pctron --> number)

pctronNeuronOutput : (pctron --> number)

pctronBeta : (pctron --> number)

pctronOutputs : (pctron --> (list string))

setNeuronName : (pctron --> (string --> pctron))

setDocumentation : (pctron --> (string --> pctron))

setInputs : (pctron --> ((list (number * string)) --> pctron))

setWeights : (pctron --> ((list (number * string)) --> pctron))

setDeltas : (pctron --> ((list (number * string)) --> pctron))

setSumDeltas : (pctron --> ((list (number * string)) --> pctron))

setActivationLevel : (pctron --> (number --> pctron))

setNeuronOutput : (pctron --> (number --> pctron))

setBeta : (pctron --> (number --> pctron))

setOutputs : (pctron --> ((list string) --> pctron))

type#params : symbol

0.2 : number

2.0 : number

0.1 : number

type#oblist : symbol

[] : (list pctron)

add-neuron-to-oblist : (pctron --> (list A))

type#neurons-levels : symbol

[[]] : (list (list pctron))

subst-o : (pctron --> (list A))

subst-o-h : (pctron --> ((list pctron) --> (list pctron)))

subst-n : (pctron --> (list A))

subst-strata : (pctron --> ((list (list pctron)) --> (list (list pctron))))

subst-stratum : (pctron --> ((list pctron) --> (list pctron)))

subst-pctron : (pctron --> (pctron --> pctron))

update-neuron : (pctron --> (list A))

transfer-function : (pctron --> pctron)

last : ((list A) --> A)

activation-level : (pctron --> number)

activation-level-h : ((list (number * string)) --> ((list (number * string)) --> (number --> number)))

threshold : (number --> number)

fire-synapses : (pctron --> (list A))

fire-synapses-h : (pctron --> (number --> ((list string) --> (list A))))

fire-synapse-aux : (pctron --> (number --> (pctron --> (list A))))

update-inputs : ((list (number * string)) --> (pctron --> ((number * string) --> (pctron --> (list (number * string))))))

update-pair : ((number * string) --> (pctron --> ((number * string) --> (pctron --> (number * string)))))

find-neuron : (string --> pctron)

find-neuron-h : (string --> ((list pctron) --> pctron))

type#neurons : symbol

["H1" "Neuron H1, (Winston, 1992)" [(@p 0.55 "1") (@p 0.55 "2") (@p 0.55 "3") (@p 1.0 "bias")] [(@p 0.55 "1") (@p 0.55 "2") (@p 0.55 "3") (@p 1.0 "bias")] [(@p 0.55 "1") (@p 0.55 "2") (@p 0.55 "3") (@p 1.0 "bias")] [(@p 0.55 "1") (@p 0.55 "2") (@p 0.55 "3") (@p 1.0 "bias")] 0.2 0.2 0.1 ["A" "S"]] : pctron

["H2" "Neuron H2, (Winston, 1992)" [(@p 0.55 "1") (@p 0.55 "2") (@p 0.55 "3") (@p 1.0 "bias")] [(@p 0.55 "1") (@p 0.55 "2") (@p 0.55 "3") (@p 0.55 "bias")] [(@p 0.55 "1") (@p 0.55 "2") (@p 0.55 "3") (@p 1.0 "bias")] [(@p 0.55 "1") (@p 0.55 "2") (@p 0.55 "3") (@p 1.0 "bias")] 0.2 0.2 0.1 ["A" "S"]] : pctron

["A" "Neuron Acquaintances, (Winston, 1992)" [(@p 0.6600000000000001 "H1") (@p 0.6600000000000001 "H2") (@p 1.0 "bias")] [(@p 0.6600000000000001 "H1") (@p 0.6600000000000001 "H2") (@p 0.2 "bias")] [(@p 0.6600000000000001 "H1") (@p 0.6600000000000001 "H2") (@p 0.2 "bias")] [(@p 0.6600000000000001 "H1") (@p 0.6600000000000001 "H2") (@p 0.2 "bias")] 0.2 0.2 0.1 []] : pctron

["S" "Neuron Siblings, (Winston, 1992)" [(@p 0.6600000000000001 "H1") (@p 0.6600000000000001 "H2") (@p 1.0 "bias")] [(@p 0.6600000000000001 "H1") (@p 0.6600000000000001 "H2") (@p 0.2 "bias")] [(@p 0.6600000000000001 "H1") (@p 0.6600000000000001 "H2") (@p 0.2 "bias")] [(@p 0.6600000000000001 "H1") (@p 0.6600000000000001 "H2") (@p 0.2 "bias")] 0.2 0.2 0.1 []] : pctron

type#training-samples : symbol

record-samples : ((list (list (list number))) --> (list A))

[] : (list A)

init-layers : (--> (list A))

append-layers : ((list pctron) --> (list A))

[] : (list A)

bpropm : (number --> (list A))

bpropm-aux : (number --> (number --> (list A)))

bpropm-iterate : (--> (list A))

bpropm-iterate1 : ((list (list (list number))) --> (list A))

butlast : ((list A) --> (list A))

assign-inputs : ((list (list number)) --> (list A))

updateAssign : ((list pctron) --> ((list (list number)) --> (list A)))

modify-inputs : (pctron --> ((list number) --> (list (number * string))))

modify-inputs-h : ((list (number * string)) --> ((list number) --> (list (number * string))))

launch-ann : (--> (list A))

launch-ann-h : ((list (list pctron)) --> (list A))

launch-stratum : ((list pctron) --> (list A))

beta-output-layer : ((list (list number)) --> (list A))

last-stratum : ((list (list pctron)) --> (list pctron))

last-sublist : ((list (list number)) --> (list number))

beta-output-layer-aux : ((list pctron) --> ((list number) --> (list A)))

beta-layers : (--> (list A))

beta-layers-h : ((list (list pctron)) --> (number --> (list A)))

beta-stratum : ((list pctron) --> (list A))

beta-neuron : (pctron --> (list A))

beta-sum : (pctron --> ((list string) --> number))

beta-sum-h : (pctron --> ((list string) --> (number --> number)))

find-weight : (pctron --> (string --> number))

find-weight-h : ((list (number * string)) --> (string --> number))

deltas-1st : ((list (list number)) --> (list A))

iterate-1st-stratum : ((list pctron) --> ((list (list number)) --> (list A)))

deltas-1st-pctron : (pctron --> ((list number) --> (list A)))

deltas-1st-aux : ((list (number * string)) --> ((list number) --> (number --> (number --> (number --> (number --> (number --> (list (number * string)))))))))

deltas-2nd : (--> (list A))

iterate-2-n-strata : ((list (list pctron)) --> (list A))

iterate-2-n-stratum : ((list pctron) --> (list A))

find-d : ((list (number * string)) --> (number --> number))

deltas-2-n-pctron : (pctron --> (list A))

deltas-2nd-aux : ((list (number * string)) --> ((list (number * string)) --> (number --> (number --> (number --> (number --> (number --> (list (number * string)))))))))

find-1 : ((list (number * string)) --> (number --> (number * string)))

zero-weight-changes : (--> (list A))

zero-weight-changes-aux : ((list pctron) --> (list A))

zero-weight-changes-pctron : (pctron --> pctron)

null-changes : ((list (number * string)) --> (list (number * string)))

add-up-weight-changes : (--> (list A))

add-up-aux : ((list pctron) --> (list A))

add-up-pctron : (pctron --> (list A))

sumDeltas1 : ((list (number * string)) --> ((list (number * string)) --> (list (number * string))))

sumDeltas-aux : ((list (number * string)) --> ((list (number * string)) --> ((list (number * string)) --> (list (number * string)))))

subst-n-s : ((list (number * string)) --> ((number * string) --> (list (number * string))))

find-s : ((list (number * string)) --> (string --> number))

change-weights : (--> (list A))

change-weights-aux : ((list pctron) --> (list A))

change-weights-pctron : (pctron --> (list A))

sumIntoWeights : ((list (number * string)) --> ((list (number * string)) --> (list (number * string))))

sumIntoWeights-aux : ((list (number * string)) --> ((list (number * string)) --> ((list (number * string)) --> (list (number * string)))))

Stack overflow

(3+)

------------------------------------------------------------

yours, Dr A. J. Y.

HELSINKI

Finland, the E.U.

Willi Riha

unread,

Sep 17, 2016, 12:20:54 AM9/17/16

to Shen

The "stack overflow" error occurs, as I myself have experienced many times, when you try to load too much code at the same time.

Mark knows about it, and has promised to do something about it.

In the meantime, we have to load the files in "instalments".

W

Antti Ylikoski

unread,

Sep 18, 2016, 9:02:22 AM9/18/16

to Shen

Dear Willi:

With which hardware/software configurations have you got the Stack

overflow error message?

I only have seen it with the 32-bit Linux Ruby Shen. The 32-bit CLisp

OS Shen will load the pctronml.shen program impeccably. So will the

64-bit Linux Ruby.

I was hoping that one of the individuals who wish to carry out work

concerning ANNs and Shen, might want to see the pctronml.shen code, in

order to learn techniques and tips for their own Shen ANN work, and

this would uncover the "mystical" problem with the multilevel

perceptron program.

We shall see about that.

yours, AJY

HELSINKI

Finland, the E.U.

Willi Riha

unread,

Sep 18, 2016, 11:11:14 PM9/18/16

to Shen

I am using the Lisp Platform (SBCL).

W

On Tuesday, September 13, 2016 at 7:52:57 AM UTC+1, Antti Ylikoski wrote:

Antti Ylikoski

unread,

Sep 20, 2016, 8:07:35 AM9/20/16

to Shen

When using freeware LISPs, one can see that there will be errors in

the underlying systems. I have written this to speculate that the

"mystical" error I mentioned in this group might be a CL

implementation issue, or something analogous to it.

The file obj.lisp is as follows. It comes from Paul Graham's book On

Lisp:

------------------------------------------------------------

;;; Function to create objects for the OO paradigm

;;;

;;; AJY 2016-09-19

;;;

;;; Multiple inheritance AJY 2016-09-19

;;;

(defun get-ancestors (obj)

(labels ((getall (x)

(append (list x)

(mapcan #'getall

(gethash 'parents x)))))

(stable-sort (delete-duplicates (getall obj))

#'(lambda (x y)

(member y (gethash 'parents x))))))

(defun some2 (fn lst)

(if (atom lst)

nil

(multiple-value-bind (val win) (funcall fn (car lst))

(if (or val win)

(values val win)

(some2 fn (cdr lst))))))

(defun obj (&rest parents)

(let ((obj (make-hash-table)))

(setf (gethash 'parents obj) parents)

(ancestors obj)

obj))

(defun ancestors (obj)

(or (gethash 'ancestors obj)

(setf (gethash 'ancestors obj) (get-ancestors obj))))

(defun rget (obj prop)

(some2 #'(lambda (a) (gethash prop a))

(ancestors obj)))

------------------------------------------------------------

It will crash the CLisp:

------------------------------------------------------------

antti@antti-ThinkPad-T43:~/obj$ clisp

i i i i i i i ooooo o ooooooo ooooo ooooo

I I I I I I I 8 8 8 8 8 o 8 8

I \ `+' / I 8 8 8 8 8 8

\ `-+-' / 8 8 8 ooooo 8oooo

`-__|__-' 8 8 8 8 8

| 8 o 8 8 o 8 8

------+------ ooooo 8oooooo ooo8ooo ooooo 8

Welcome to GNU CLISP 2.49 (2010-07-07) <http://clisp.cons.org/>

Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998

Type :h and hit Enter for context help.

[1]> (load "obj.lisp")

;; Loading file obj.lisp ...

;; Loaded file obj.lisp

T

[2]> (obj)

*** - Lisp stack overflow. RESET

[3]>

------------------------------------------------------------

but it will work with the SBCL:

------------------------------------------------------------

antti@antti-ThinkPad-T43:~/obj$ sbcl

This is SBCL 1.1.14.debian, an implementation of ANSI Common Lisp.

More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.

It is mostly in the public domain; some portions are provided under

BSD-style licenses. See the CREDITS and COPYING files in the

distribution for more information.

* (load "obj.lisp")

; file: /home/antti/obj/obj.lisp

; in: DEFUN OBJ

; (ANCESTORS OBJ)

;

; caught STYLE-WARNING:

; undefined function: ANCESTORS

;

; compilation unit finished

; Undefined function:

; ANCESTORS

; caught 1 STYLE-WARNING condition

T

* (obj)

#<HASH-TABLE :TEST EQL :COUNT 2 {ACC1579}>

*

------------------------------------------------------------

and it also will work with the ABCL, the Armed Bear Common LISP:

------------------------------------------------------------

antti@antti-ThinkPad-T43:~/obj$ java -jar abcl.jar

Armed Bear Common Lisp 1.3.3

Java 1.7.0_111 Oracle Corporation

OpenJDK Client VM

Low-level initialization completed in 0.934 seconds.

Startup completed in 3.631 seconds.

Type ":help" for a list of available commands.

CL-USER(1): (load "obj.lisp")

T

CL-USER(2): (obj)

#<EQL HASH-TABLE 2 entries, 11 buckets {1734786}>

CL-USER(3):

------------------------------------------------------------

yours, Dr AJY

Helsinki

Finland, the EU

Antti Ylikoski

unread,

Oct 4, 2016, 7:38:44 AM10/4/16

to Shen

I am using the CLisp Shen, plus the ELisp Shen.

Mark Tarver

unread,

Oct 4, 2016, 9:12:20 AM10/4/16

to Shen

Antti,

I can't comment on what you are doing but you know that certain platforms are more prone to overflows than others. Shen Ruby might be an example. AFAIK CLisp is about 4x slower than SBCL.

Mark

Reply all

Reply to author

Forward