How to set the gradient of a NN as a expression

[Yi Isaac Yang]

Dear all,

I need to used the squared norm of the gradients of the neural network as the loss function. So how should I write the expression?

For example, if I have build a one layer neural network f(x) with two inputs and one output. I want to set the squared norm of f'(x) or df/dx as the loss function. How should I write the expression?

    ParameterCollection pc;
    SimpleSGDTrainer trainer(pc);
    ComputationGraph cg;
    Expression W = parameter(cg, p_W);
    Expression b = parameter(cg, p_b);
   
    vector<dynet::real> x_values(2);
    Expression x = input(cg,{2},&x_values);
   
    Expression y_pred = tanh(W*x + b)*2;

This is the code of the neural network, here I used y_pred to represent f(x). I found the expression have the function gradient(). However, when I print the value of y_pred.gradient(), I found that the y_pred.gradient() has only one elements. But is should have two, because the function f(x) have two inputs. I also found some other functions that called "gradient()", but none of them is the gradient() I needed:

    x_values[0]=rand()/double(RAND_MAX);
    x_values[1]=rand()/double(RAND_MAX);
   
    cg.forward(y_pred);
    cg.backward(y_pred);
   
    cout<<"W = "<<W.value()<<endl;
    cout<<"W' = "<<W.gradient()<<endl;
    cout<<"b = "<<b.value()<<endl;
    cout<<"b' = "<<b.gradient()<<endl;
    cout<<"X = "<<x_values[0]<<" "<<x_values[1]<<endl;
    cout<<"X' = "<<x.gradient()<<endl;
    cout<<"Y = "<<y_pred.value()<<endl;
    cout<<"Y' = "<<y_pred.gradient()<<endl;
    cout<<"G' = "<<as_vector(cg.get_gradient(0))<<endl<<endl;

The output is:

W =    2 -3.4
W' = 0.000383162 0.000179855
b = 4.2
b' = 0.000456043
X = 0.840188 0.394383
X' = 0
0
Y = 0.999772
Y' = 1
G' = 0.000383162 0.000179855

So could anyone tell me how to write it?

Thank you very much and best regards,

Isaac