What should you do depends on how you want the app to respond. The default response in AI2 is to allow the Android to decide the orientation of the screen. Notice that in the Designer screen, if you look at Screen1 Properties (in the column on the far right) the default response is set to Unspecified,

you can change that to Portrait, Landscape,Sensor, User depending on how you want the app to perform. ScreenOrientation
- The requested screen orientation. Commonly used values are unspecified (-1), landscape (0), portrait (1), sensor (4), and user (2). See the Android developer docuemntation for ActivityInfo.Screen_Orientation for the complete list of possible settings.
Yes, changing orientation can significantly alter how the screen objects are displayed...that is a function of how you selected the Width/Height settings of your components .. Automatic, FillParent or used specific pixels. Developers have to factor in whether the screen should rotate or not in their development plans and use the Automatic/FillParent appropriately after testing their app.
Regards,
Steve