If there is a decision to start off muted (hopefully that would be a non-default setting), it would make sense to both clearly indicate that the user is currently muted, and to make it very easy for them to turn that off.
I'm not convinced that starting off muted is a great solution to audio feedback. In the tests I've done with tokbox conferences, continuous feedback has not been a serious problem, although there have been some (non-continuous) echoing issues. The echoing issues are not clearly addressed by muting, since the user will ultimately have to either turn their speakers down or use headphones, whether or not they started out muted.
I don't mean to argue against the option altogether, just in favor of the default behavior being to start non-muted.
A compromise idea would be start out muted for the first, say, 5 seconds, and gradually unmute from there, along with some visual indication that this is happening (e.g. "Unmuting in 5...4...3.."). If the speakers are too loud, this gives the user a little time to turn down their volume, or put on headphones.