Webmaster tools is still a decent indicator of your site speed. As Matt Cutts said it shows "information very close to the information that we’re actually using in our ranking" (
http://www.mattcutts.com/blog/site-speed/).
Your example of a group of users who experience extremely slow load times is relevant, but not as important as you deem, I think. I doubt that Google is blindly averaging all of the load times together to give your site a speed ranking. Outliers (like a 10 minute load time when every other request is sub 2 seconds) are probably not going to yank up your overall load time in Webmaster Tools. I don't have a whole lot of evidence to support this, but in all performance analytics the average can be deceiving exactly because of issues like this - big outliers.
With that said, if the majority of your users are 6000 miles away from where you host your site and you aren't using a CDN you are going to see poor performance in Webmaster Tools, and your search engine ranking will suffer. This makes sense, your core user base IS experiencing slow load times, even if the site is fast for you, who is siting a couple miles from your datacenter/colo/basement server room/etc.
So I think the answer is to trust that real user data is going to provide a decent perspective on how fast your site is. Webmaster Tools also indicates how accurate the data is based on how many data points Google has. Presumably if the data is of low accuracy then Google won't weight it as heavily in your ranking.
If you are responsible for hitting certain performance metrics then I suggest you use a more specific measurement. As Sean Power suggested in his talk at Velocity last year, you need really well fleshed out SLAs if you are responsible for keeping your site fast. Something like "the homepage of our site will load in under 2 seconds at the 80th percentile measured with 100 synthetic tests from Chicago during peak business hours". This is his talk:
http://velocityconf.com/velocity2010/public/schedule/detail/14327. The numbers you use don't have to be synthetic, you can establish SLAs with any kind of data, but I think you need to establish exactly what numbers you are responsible for, or else you could get into a slippery slope situation where your boss says things like "I was browsing the site over dial-up while I was on vacation in China and it took 30 seconds to load our homepage!!! What's going on?!".
There are a number of tools you can use to get performance data, some free and some paid. Boomerang is a good free way to get real user data that you can then compare with Webmaster Tools to see if things tie out, so you can have a better idea of what you should be responsible for:
http://developer.yahoo.com/blogs/ydn/posts/2010/06/performance_testing_with_boomerang/I hope this helps.
-Jonathan