--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To post to this group, send email to sy...@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy.
For more options, visit https://groups.google.com/groups/opt_out.
Hello,Theano/Sympy questionsFred, in terms of replacing sympy.Pieceiwse with a Theano equivalent, since sympy.Piecewise attempts each condition (from ExprCondPair) until one is valid, I would think that the closest equivalent would be a recursive theano ifelse (ie. sympy.Piecewise((expr1,cond1),(expr2,cond2)) ~ theano.ifelse(cond1,expr1,theano.ifelse(cond2,expr2,None))) statement so that the expressions are not evaluated until the condition is achieved. This might be too much of a patch though and I am not sure how to implement it with the same argument structure as sympy.Pieceiwse.
With regards to using an externally defined theano graph or op (theano wrapped sympy implementation), how would one pass them to theano_function? It feels as though it would be analogous to the autowrap helpers argument, or would this be an issue of merging graphs before calling theano.function? If I can understand this step, I feel as though both of my problems would be solvable (a wrapping of a piecewise along with a wrapping of an interpolated function)
In all of my simplified test cases (no pieceiwse or interpolations, 9 heavily-linked non-linear ODEs), I also show that the lambdify function is faster to evaluate than my created theano function. When passing a list of expressions, if there is a commons subexpression that exists in two expressions (not twice in the same expression), is it treated as such during theano compilation or is each expression handled separately?
Thanks Guy and Jason for starting this conversation and thanks Fred for jumping in with Theano expertise.
I'm out camping in Wisconsin this week but should be available starting Thursday. I'm excited to help contribute to this machinery.
Theano will merge duplicated subexpression automatically by default. What is your function? If it is too "simple", for example just working on scalar, Theano have by default much overhead. As said for the other benchmark, there is tricks to remove a big part or even all of it, but it ask for some work on the developer.
Ok, Just went through this thread again.It should be simple to translate SymPy.Piecewise to a recursive Theano.switch (after translating SymPy.LT to theano.lt, etc.) I'll get on this soon. Does this sound reasonable to you Fred?Jason's PR allowing pass through of keyword arguments looks close to me. I suspect that the `on_unused_input=ignore` issue will soon be possible. Question, should this be default? Are there other defaults that should be used? I'm curious why you both are making functions with unused inputs.
Jason, was Fred's note on directly passing constants useful? If not why not? It sounds like you should subs your constant symbols for numeric values prior to creating the theano graph.
Hey Fred,I am starting to play with making a theano.Op, and in principle, linking a custom theano.Op to an implemented sympy function will solve all of my problems (at least in the sense of getting my system running, not necessarily speed). My initial post spoke of both mapping Piecewise and interpolated functions into theano; after speaking with my advisor, due to wanting a smoothing criterion over our piecewise datasets, I think we will end up treating them as interpolated functions over smoothed data sets. My idea for how to use an interpolation routine will be a wrapped theano.Op, so I am still rather curious as to how implementing Piecewise would work in such way that theano can manipulate it in graph form (built out of native theano operations).Theano will merge duplicated subexpression automatically by default. What is your function? If it is too "simple", for example just working on scalar, Theano have by default much overhead. As said for the other benchmark, there is tricks to remove a big part or even all of it, but it ask for some work on the developer.The basic system/functions can be described as follows; we have some number of species (n_0 ...n_k) and we are creating a function for the vector of time derivatives. The different types of expressions come about from the reaction rates (K_0 ..... K_l) The whole system is described symbolically, for symbolic determination of the jacobian (spline interpolated functions are wrapped with a sympy implemented function so that taking a symbolic derivative also yields the derivative of the numerical implementation). Dummy example below with three types of base expressionsd(n_i)/dt = K_0*n_2*n_0 + K_1*n_3 + K_2*n_4*n_2*n_4/(sum(n_0....n_k))Each of the K terms can be described differently:--constant--analytical parameterized of some n_j (Kf ~ a0*(n_j**a1)*exp(a2/n_j) , a0..a2 are constants)--spline interpolation function of some n_j (K_interp(n_j))--analytical weighted with piecewise/interpolated (new_Kf ~ Kf*exp(-K_interp(n_j))I also have two energy equations, but these incorporate all of the same types of terms, just not in a nice summation as in the species equations. Any given base expression (eg. K_0*n_2*n_0) will appear in multiple time derivative equations, so if the interpolation is slow, it would be nice to get theano to recognize that it is a 'common expression.' I can make test cases with a few species and a few reactions (Ks), but the true systems I would like to be simulating will have 40-100 species and 300-2000 reaction rates. Are these functions too 'simple' to be sped by using theano? There is a lot of optimization that can be done due to the high number of shared expressions, so I was hoping to take advantage of the theano graph optimizations.
My initial test cases were using the numpy.float64 dtype, so I will switch it over to ndarrays if possible.I hope to have questions or a working spline interpolation theano.Op in the next few hours.Cheers,GuyQuick Codegen question: In principle, can one create a python callback within a compiled c code to a python function that is not compiled (interpolation function)?
Ok, Just went through this thread again.It should be simple to translate SymPy.Piecewise to a recursive Theano.switch (after translating SymPy.LT to theano.lt, etc.) I'll get on this soon. Does this sound reasonable to you Fred?
Jason's PR allowing pass through of keyword arguments looks close to me. I suspect that the `on_unused_input=ignore` issue will soon be possible. Question, should this be default? Are there other defaults that should be used? I'm curious why you both are making functions with unused inputs.Jason, was Fred's note on directly passing constants useful? If not why not? It sounds like you should subs your constant symbols for numeric values prior to creating the theano graph.
> SymPy C Codegen and Theano@Fred, how hard would it be to leverage SymPy's C codegen in Theano? This might be a lot cleaner than wrapping raw SymPy operations and might substantially extend Theano's support of scalar expressions. Do you have a performant Bessel function op? I'll bet SymPy could be made to do this quite well.@Aaron / @Ondrej, if you're reading this thread could you point us to the best place to start looking at C codegen in SymPy? Alternatively can you point to an active community member who would be able to do so?
> ODE IntegrationWhat are people's thoughts on scipy.integrate.odeint? Earlier this year I was looking at nodepy, a purely mathematical treatment/catalog of time stepping schemes. It should be easy to generate such schemes in Theano using simple abstractions like Butcher matrices. The code I was playing around with was https://github.com/mrocklin/tsgen. The goal was that we could easily experiment with and rapidly benchmark different time stepping schemes for specific problems.
It should be simple to translate SymPy.Piecewise to a recursive Theano.switch (after translating SymPy.LT to theano.lt, etc.) I'll get on this soon. Does this sound reasonable to you Fred?It sound reasonable and is the first thing I suggest to try.
> SymPy C Codegen and Theano@Fred, how hard would it be to leverage SymPy's C codegen in Theano? This might be a lot cleaner than wrapping raw SymPy operations and might substantially extend Theano's support of scalar expressions. Do you have a performant Bessel function op? I'll bet SymPy could be made to do this quite well.@Aaron / @Ondrej, if you're reading this thread could you point us to the best place to start looking at C codegen in SymPy? Alternatively can you point to an active community member who would be able to do so?@Matt, you already did a new Theano op with C code. I think it is the only "easy" way to wrap other people c code in Theano. If the person already know this C code AND a little of Python AND NumPy C-API, it isn't very hard to a new Theano op with C code. Otherwise, doing the first such op ask to learn a few think and could ask a few days. You already did this, so you have a good idea of the work it need.Now the questions is how is done the SymPy code gen? Is just just string template that is filled with dtype and other stuff? If we can just call one SymPy function with the information of what we want and it return a string with the C code it could be relatively easy. The only questions is about how to handle the variable name to pass the information around. At worst, we wrap the sympy c code in a c function, then make a small wrapper c code that take the Theano c variable name and call this function. So not very hard as Theano provide what is needed.
Presently I am trying to use the mapping between Theano and SymPy (sympy.printing.theanocode theano_function) to make my callable functions and take advantage of the optimization routines. I have two major problems and a few questions:1st major problem: Though piecewise functions exist in SymPy (sympy.functions.elementary.piecewise) there is no counterpart in Theano. Looking at the source of the inspiration for theanocode (https://github.com/nouiz/theano_sympy/ graph_translation.py) I see that some of the SymPy equivalents were defined as lambda functions. Is there an equivalent way to add Theano conditional expressions wrapped into a function to add to the mapping dictionary in theanocode.py?
2nd major problem: Similar to the problem above in that I am not sure that the Theano counterpart is; some of the terms that I use are interpolated functions (with one ODE variable as input) that we have wrapped symbolically while providing a numerical implementation (so that symbolic derivatives can be made, resulting in their own interpolations). Is it possible to recreate the interpolation function as a Theano operation for use within the system of ODEs?
Remain questions:I presently have to flatten my input to theano_function to a list of expressions and then wrap to return to a form (Jacobian is a matrix not a vector); is it possible to have a matrix of different expressions as an input to theano_function with a vector output?
I know that a huge amount of Theano speed up is due to parallelization of matrix operations (which I do not have), should I be focusing on SymPy Autowrap/Ufuncify or my own code generation instead of trying to get Theano to play nicely?
Stupid questions:Does sympy.printing.theanocode.theano_function automatically optimize the compiled graph?
Minor comment:Perhaps unnecessary for most uses of the theano_function, but I needed to modify function inputs so as to be able to use the keyword argument 'on_unused_input=ignore' as opposed to 'raise' so that I did not need to have all symbols in all equations. This may be avoided by having the unused symbols somehow (I don't know how) included in each expression.
I'm not sure I fully understand this piece. Is it possible to define your interpolation scheme with lower-level operators such as with Piecewise and polynomials? What is your interpolation scheme?
--
Hi,
I was in vacation. Have you been able to make this work?
Fred