Hi,
Here is my 2 cents...
First thing I would do is change to a different channel. You should see some option like "Wireless Channel" on the router. Maybe you and a neighbor are interfering with each other. Once you change that on the router, the iMac might be smart enough to look on different channels so you may not need to change anything there.
If that doesn't improve anything, maybe you have an older wireless router. That might mean that even when you change the channel you are still on a frequency that is getting interference from something like a wireless phone or a microwave. A limited test would be to make sure your not using a wireless phone (to your home line, not a mobile phone) when on the iMac. It's a limited test because even if you are not doing this, perhaps a neighbor is. So, limited value here both worth mentioning.
If none of this is narrowing down the problem, maybe a new router would be worth it if you are not already on something like 802.11g/n (scroll about halfway down here)
http://en.wikipedia.org/wiki/Comparison_of_wireless_data_standards You could look for a router that supports both 2.4Ghz and 5 Ghz. If you do that, make sure the iMac can handle the 5Ghz frequency too. This should get you around any interference issues.
I have a Linksys wired/wireless router and don't have any complaints. It's wireless to a Tivo 802.11g adapter and wired to a Mac - both sharing Comcast. The wired connection is about 20Mbps. I don't know the wireless speed but it's enough to stream a video which has got to be more than 320Kbps.
John