I deleted the add-on from the profile folder and because that did not
solve the problem, I rebooted to clear the memory which did not solve
the problem.
I did not realize the correct procedure would have been to reboot (which
was the only way for me to stop the "Firefox update" window, then delete
the add on from the profile folder _before_ testing Firefox again. I did
not realize that as long as the annoying update folder was showing (only
rebooting would close it) Firefox was re-writing the addon to the
profile folder and my deletions had no effect.
I could not close Firefox with all addons closed because it never got to
that stage! I was stuck at the "Firefox Update" window. Sort of a catch 22.