The graphics on the Canvas are relatively slow. What you are doing is about as good a way of polling all the pixels in a grid of 40,368 pixels. Over 40 thousand operations!
Here are some links discussing the feasibility of scanning all the pixels in an image using AI2. Most of the lings discuss parts of your topic (some discuss other things). The links about pixels (there are several) explain why Ghica tells you that AI2 can not process the data fast enough.
https://groups.google.com/forum/#!searchin/mitappinventortest/pixel$20color
I wrote a small program to poll the pixels on a 100 x 100 pixel image; and discarded it. The app was slow, crashed frequently and would not work at all with larger images at all. That involved only 10000 pixels; the Ai2 environment is just too slow versus the type of response you would get using Java.
In the same post, ABG made a suggestion that might work for you. He said "Google for "Web based image recognition" and look
for a service where you can POST an image file and get back some Web text response. That's as close as you could come with App Inventor.
ABG"
Regards,
Steve