The best ways to do this are quite platform-specific.
On Windows, this is done by creating some named object, like say a named mutex
via the CreateMutex WIN32 function. CreateMutex sets the error code to
ERROR_ALREADY_EXISTS. Such objects go away when the last process loses
the reference, so you don't have to worry that it will stick around and
prevent your program from coming up.
Programs that have a window (either a visible one, or just a hidden one for
receiving messages) can just look for that window by enumerating all the
windows and recognizing theirs. If the program is already running, then you
post a special message to its window and quit. The program responds to the
message in some way, like by coming to the foreground, or opening a new
document window, simulating a second instance that is really managed by the
single instance.