Interest in GSoC 2025: Optimizing Jacobian/Hessian Codegen for ODE/Optimization

[李沛奇]

**Dear SymPy Community**,  

I'm a first-year dual-degree student in **Mathematics & AI** at Shanghai Jiao Tong University. I’m excited to contribute to SymPy’s codegen capabilities, specifically targeting the **"Efficient Jacobian/Hessian Evaluation for Optimization and ODE Integration"** project for GSoC 2025.  

**Why This Project?**  
During my coursework on numerical optimization and ODE modeling, I encountered bottlenecks in generating efficient derivative functions for large-scale problems. SymPy’s symbolic differentiation paired with codegen could bridge this gap, and I aim to integrate recent advancements (e.g., #26773’s Jacobian optimizations) into a unified API.  

**Technical Vision**:  
1. **CSE Optimization**: Eliminate redundant calculations via common subexpression elimination.  
2. **Sparsity Detection**: Automatically identify zero elements in Jacobian matrices for ODE problems, enabling sparse storage formats.  
3. **High-Performance Codegen**: Leverage `lambdify` and LLVM to generate C/Fortran code with near-handwritten performance.  

**Request for Guidance**:  
1. Are there ongoing discussions or roadblocks around derivative codegen I should prioritize?  
2. Would a prototype integrating Riccardo’s Jacobian method with `lambdify` be valuable as a starting point?  
3. Is there interest in sparse Jacobian support for ODE problems, and are there preferred sparse formats (e.g., COO/CSC)?  

I’m eager to refine my proposal with community feedback. Thank you for your time!  

**GitHub**: voyager-jhk

Best regards,  
Peiqi Li

[Jason Moore]

Hi Peiqi,

Your technical vision sounds good, but also a very large scope for a GSoC project.

> 1. Are there ongoing discussions or roadblocks around derivative codegen I should prioritize?

Yes, many, I recommend searching the Github repo for related terms to see past work. If cse=True on any codegen call (lambdify or other) and gradients, Jacobians, Hessians, etc. are involved then you can speed up differentiation by first converting expressions to a cse'd form and generating the derivatives from that form, writing all cse'd code directly to memory/disk without converting it back into long form expressions. So if someone wants lambdify(args, (expr, expr.diff(x))) there is a lot of room to improve code generation performance and numerical evaluation performance, without actually executing .diff(), rather you want something like lambdify(args, (expr, Derivative(expr, x))).

> 2. Would a prototype integrating Riccardo’s Jacobian method with `lambdify` be valuable as a starting point? 

Yes, that is the logical follow up to Riccardo's work last year. First would be to complete his work from last year, as it is unfinished.

> 3. Is there interest in sparse Jacobian support for ODE problems, and are there preferred sparse formats (e.g., COO/CSC)? 

Sure, but you first need to solve the dense scenario. This is out of scope until we have the basics working. In the idea, I showed the symjit package. That already does more than what we do in SymPy, so simply making something like that work with lambdify based tools for native SymPy use would be a big step forward.

Jason

moorepants.info
+01 530-601-9791

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sympy/CAPnDX9DixKS69%2B5mknBZuMy_HvL6fN2oM62obkV0EeSj97_utw%40mail.gmail.com.

[李沛奇]

Hi Jason,

Thank you for your detailed feedback! I appreciate your guidance in narrowing down and prioritizing the Intensive Jacobian workflow. I will be looking at Riccardo's work next:

I will look into PR #26773 and identify the remaining gaps (e.g. integration with lambdify).

Prototype CSE optimized Jacobian generators by combining cse() with Derivative objects to avoid redundant expr.diff(x) calls.

In addition, for Codegen performance:

Benchmark current lambdify-generated Jacobian code against hand-optimized C code.

Explore symjit's approach to JIT compilation and LLVM integration as a reference.

After some progress I will submit a draft PR demonstrating a Jacobian codegen optimized for CSE for simple ODE systems.

With best regards
Peiqi Li

Translated with DeepL.com (free version)