On your Android device, go to System Settings, Developer Options, turn them on, and be sure that "USB Debugging" is allowed.
On most devices running Android 3.2 or older, you can find this option under Settings > Applications > Development.
On Android 4.0 and newer, it's in Settings > Developer options.
Note: On Android 4.2 and newer, Developer options is hidden by default. To make it available, go to Settings > About phone and tap Build number seven times. Return to the previous screen to find Developer options, including "USB Debugging".
Connect your Android device to the computer using the USB cable - be sure that the device connects as a "mass storage device" (not "media device") and that it is not mounted as a drive on your computer. This may mean that you have to go to the Finder (on a Mac) or My Computer (on Windows) and disconnect any drive(s) that were mounted when you connected your Android device.
On Android 4.2.2 and newer, your device will pop up a screen with the message Allow USB Debugging? the first time you connect it to new computer. Press "OK". This authenticates the computer to the device, allowing the computer to communicate with it. You'll need to do this for each computer you want to connect to the device, but only once per computer.
Go to this Connection Test Page (opens in a new tab in your browser) and see if you get a confirmation that your computer can detect the device. If the test fails, go to General Connection Help and look at the USB help for your computer (Windows or Mac). You won't be able to use App Inventor with the USB cable until you resolve the connection issues. Return to this page when the test suceeds.
This page contains help for when you have connected your Android device to a Windows PC with the USB cable, and the Connection Test says that the device is not detected.
In order to connect your device to a Windows PC over the USB cable, you need to install a driver on your PC. The driver is a software module that lets the PC interface to the device hardware. Unfortunately, different devices require different drivers, and Google and Microsoft have generally left it to the device manufacturers to develop and distribute the drivers. The App Inventor setup software comes with drivers for a very small number of Android devices. You might be lucky enough that these work for your device -- but probably not. In that case, you'll have to get the driver from the manufacturer, and install it on your PC yourself.
Here is Google's link to OEM (Original Equipment Manufacturer) Drivers. It shows where to get drivers for the devices that Google knows about, and explains how to install them on PCs.
Google also distributes drivers for the Nexus devices that it manufactures. If you have a Nexus device. Look here for the Google USB Driver. To get the driver from there, you can use the link that says Click here to download the latest Google USB Driver ZIP file. You do not have to download the entire Android SDK.
When you are done installing the driver, go back and review the steps in Connecting to a phone or tablet with a USB cable. Hopefully, the connection test will succeed this time. If it does not, the most likely place for an error is in installing the drivers, especially if you have never done this before. So try again and/or get help.
If you are still having trouble, see Google's page on Using Hardware Devices. Other than that, try searching on the web for issues about using your particular device with USB.