Without having checked the code, i am pretty sure that the builtin Auth Modul of Web2py expects a plaintext email in the username field when checking the credentials.
I suppose now it fails as the encrypted email value does simply not match the plaintext one.
You would need to modify the Auth module to encrypt the plain text email after submitting and then also checking against an encrypted value. During user registration you would also need to insure user name encryption.
This is in fact pretty much how passwords are stored, with a hash, so inr way encryption and only the hashes are compared.