There are other accent marks, such as the breve which is something like (but isn't) a comma on it's side, or the circumflex which is angled. Perhaps you mean to use one of those instead?
Or there is the cedilla, which looks like a comma hanging off the bottom of a c (ç Ç).
But, just as Gene says, other common accents: acute (é É), grave (è È), circumflex (î Î), umlaut (ü Ü), tilde (ñ Ñ), etc., are definitely NOT comma shaped, and the proper position is centered just over their respective letters -- and not off to the right as they are so often misused.
Neil
That makes it a different question altogether!
You want to use "typographer's quotes" or "curly quotes" instead of the "primes" (which look much like the top of exclamation marks) that are the default. A lot of programs have an option to use "typographer's quotes" or "smart quotes", but how you do it depends on the application.
Otherwise, if you're on PC type alt+0146 on the numeric keypad. On MAC it's shift+option+]
Good luck,
Gene