Hi Mo,
With respect to the back boundary, stating "back = abs_2d" in params.txt will give you a 2D absorbing back boundary in any XBeach version since revision 1276. I find that this helps reduce confusion between for instance the "front = " numbers and "back = " numbers.
You set up for the 2DH case seems sensible. I used a similar combination of boundary conditions at Santa Rosa and in following models. One issue that may be confusing XBeach is that you do not have uniform bathymetry and/or forcing across the lateral boundaries. Have you tried increasing you grid size in longshore direction by a couple of grid cells in both directions and imposing a longshore uniform bathymetry there? If the instabilities are not created at the boundaries, they may be time step related. In a 2DH simulation with morphology I would not recommend setting the CFL parameter higher than 0.7.
Robert