Hi Amit,
what trouble are you having with the GAP library exactly? Their reference manual is here:
https://www.gap-system.org/Manuals/doc/ref/chap0.html and the main website has a section on how to install it.
Checking if a given homomorphism is in fact an isomorphism is already possible (though determining if a general map is a homomorphism would probably be tricky in some cases). The task now is to determine if there exists an isomorphism between two given groups and, if so, try to find one. This is difficult in general. The handbook talks a bit about random isomorphism testing in chapter 11 I think.
As for kernels of homomorphism with infinite domains, do you have any ideas on computing normal subgroups of infinite groups? If we had an efficient algorithm for finding normal subgroups, we could determine if a given normal subgroup is part of the kernel but checking if its generators are in the kernel, but this still doesn't tell us if this normal subgroup is the whole of the kernel. Also, some subgroups may have infinitely many generators.