New Training Algorithm
Thorsten Kiefer
I'm currently working on a new Training Algorithm for Feed Forward Neural
Nets.
My experiment is like this :
1. convert the neural net into a SAT instance
return e.g. : input = [[1,2,3,4],[5,6,7,8],[9,10,11,12]]
weights = [[24,27,30,35],[38,40,43,45],........
output = [[1024,1030,1034,1039],......
I.e. I use 4 bits per input in fixed point representation,
Variables 1,2,3,4 contain the four bits of input[0],
Variables 5,6,7,8 contain the four bits of input[1]...........
Variables 24,27,30,35 contain the four bits of weight[0],
Variables 38,40,43,45 contain the four bits of weight[1],..........
Variables 1024,1030,1034,1039 contain the four bits of ouput[0],........
2. e.g. input[] = {0.5,0.25,0.75] = {1000,0100,1100}
so we insert the Maxterms into the SAT instance :
(1),(-2),(-3),(-4),
(-5),(6),(-7),(-8),
(9),(10),(-11),(-12)
3. e.g. ouput[] = {0.125} = {0010}
so we insert the Maxterms : (-1024),(-1030),(1034),(-1039)
4. run your favorite SAT solver, and after it found the solution,
extract the values for the weights.
It maybe that there is no exact solution so my hope is that MaxSat is
a good approximation.
If MaxSat is still too slow, we use Schöning's Randomized algorithm, wich
converges against the solution - I don't know, if Schöning's algorithm
converges against MaxSAT.
I also hope that Schöning's algorithm is also suitable for online training.
Has anyone ever tried this ?
Any comments ?
Best Regards
Thorsten
Greg
> Hi,
> I'm currently working on a new Training Algorithm for Feed Forward Neural
> Nets.
> My experiment is like this :
> 1. convert the neural net into a SAT instance
What is that?
Greg