Johan, I think this may have come up before on this forum, but YALMIP can invoke cplexqcp with a non-psd matrix Q. Then cplexqcp fails. Meanwhile, YALMIP could have used another solver. I tried solving your formulation without specifying solver, and it chose cplexqcp. Meanwhile, fmincon and knitro were available, and in fact both could solve to a local optimum (but I needed to specify a non-default starting point to get what turned out to be a globally optimal solution, as opposed to the default x=y=[0;0]).
a=1;P=eye(2);
optimize([constraints1,constraints2],cost,sdpsettings('debug',1))
Presolve time = 0.00 sec. (0.00 ticks)
Error using cplexqcp (line 652)
CPLEX Error 5002: Q in %s is not positive semi-definite.
+++++++++++++++
BMIBNB using CPLEX LP, lower and upper solver solved the problem.
++++++++++++++
BARON failed (with or without 'usex0',1), but I can't tell you whose fault is it is (YALMIP side or BARON's matbar side). I used BARON 14.4.0 with matbar v 1.69 (which is the most recent) under MATLAB R2014A win64 and
YALMIP R20150204.optimize([constraints1,constraints2],cost,sdpsettings('solver','baron','debug',1))
MATLAB/BARON Interface Version: v1.69 [12-October-2014]
===========================================================================
BARON version 14.4.0. Built: WIN-64 Tue Dec 2 15:07:16 EST 2014
If you use this software, please cite:
Tawarmalani, M. and N. V. Sahinidis, A polyhedral
branch-and-cut approach to global optimization,
Mathematical Programming, 103(2), 225-249, 2005.
BARON is a product of The Optimization Firm, LLC.
http://www.minlp.com/ Parts of the BARON software were created at the
University of Illinois at Urbana-Champaign.
===========================================================================
ERROR: Two consecutive operators found in line 34
BARON: Syntax error. Execution will stop
ans =
yalmiptime: 0.0587
solvertime: 0.3223
info: 'Other identified error (BARON)'
problem: 11