Re: Some questions

[Buyun Liang]

Hi Yuhang,

1). NCVX PyGRANSO is a PyTorch-enabled port of GRANSO incorporating auto-differentiation, GPU acceleration, tensor input, and support for new QP solvers.

2). Correct. The main difference is that these popular solvers cannot directly handle constraints. Besides that, these solvers are based on first-order methods, while our NCVX PyGRANSO is based on BFGS-SQP, a second order method that can provide solutions with higher precision and can converge faster.

3). Yes, only the optimization parts need to be modified. Get familiar with all provided examples, especially this one: https://ncvx.org/examples/D3_orthogonal_rnn.html

4). Yes, we're working on integrating PyGRANSO into the popular deep learning frameworks. Our current interface is pretty standard as the majority of constrained optimization software. I guess the difficulty should be from formulating the optimization problem (e.g., you need to specify which groups of neural network weights you wish to use in the constraint function). Could you point out some specific points that make PyGRANSO hard to use?

Best,

Buyun

On Tue, May 9, 2023 at 3:17 AM 李宇航 <2212...@bjtu.edu.cn> wrote:

hi Buyun,

  I have tried to send an email to nc...@umn.edu, but it always fails, so I have to turn to you again.

1). What is the relationship between PyGRANSO and pytroch?

2). Is it understood that PyGRANSO is an optimizer that solves neural networks with constraints, like some common optimizers of pytroch such as SGD, RMSprop, Adam?

3). If I need to use PyGRANSO in my existing pytroch project, do I need to modify the network optimization related code as in these examples? https://ncvx.org/examples/index.html.

4). If I understand question 1 correctly, would the PyGRANSO optimizer be considered for integration into pytroch, or would it continue to be developed as a separate project, because after reading some examples, I feel that it is a bit difficult to apply PyGRANSO to some existing pytorch projects.

Best,

Yuhang