http://www.instructables.com/id/Control-an-Arduino-With-Android-and-USB/
This tutorial was written for Thunkable but should work fine with AppInventor too
And some troubleshooting tips can be found here :
http://domhnallohanlon.com/thunkable_extensions/tools/arduino-extension.html
Moreover, I will look into that.