Note that goimports theses days adds the package name explicitly to the import if the package name is not identical to the last component of the import path. So on the readability front, the cost seem pretty low to me.
It does have a cost on the writing side, as goimports might not find the package and there's always an extra step to figure out the package name if you start with the import path (e.g. a GitHub repo).
So yes, I would generally try to avoid it, but it's also not the end of the world :)