Following up on our private conversation, and to benefit the community if anyone has a similar question, we have concluded that the behavior mentioned is not an issue with the guest environment itself. To verify, I tested SFTP with multiple simultaneous user authentications and successful file transfers, but I couldn't reproduce the same behavior. However, after investigating your use case, this behavior was rather due to a custom configuration that you implemented in your Linux instance.
You have been following this third party tutorial to implement SFTP, and disable shell access (SSH) to your developers on a Linux server. After checking the tutorial, you had to SSH to your instance and manually create a user group, create users, direct them to root as a home directory rather than the default home directory, modify the SSH daemon file (sshd_config), and change ownership of some files to root. At the same time and despite such modifications, you wanted to manage your SSH keys through the Google Compute Engine metadata.
According to this document, The guest environment does not update the keys for manually created users. Hence, it is not advisable to manually create them and then add the same users with their keys through metadata (or the opposite if possible) as this causes conflicts. The guest environment by default in a normal setup and without any manual modifications automatically creates the users, their home directories, and their corresponding (~/.ssh/authorized_keys) file which contains the public keys. Removing or restricting access to each user’s home directory and it’s content will cause authentication issues.
For this use case, it is advisable, when manually creating users, to manually add the keys to the corresponding directory that the manually configured SSH daemon checks, rather than adding them through metadata. On the other hand, If you would like to use metadata to manage your user keys, then this simple alternative could work without conflicts. If you choose the latter make sure to restore the original settings of the guest environment (only if you changed them, SSH daemon ..), delete the current users, and recreate them with their keys through metadata, and then use the commands in the provided link for each of your users. Please note that since these instructions are from a third party source, we cannot guarantee their accuracy. Google does not take any guarantee of its working behavior and any implications that it may cause.