Decrypting Encrypted Customer Data from Django Queries Not Working, But Working with Regular DB Queries

[G Z]

so I have a database of information with encrypted customer names. When I use a plain python script to decrypt the names it works correctly and decrypts all of the customer names.

However, when I used Django it will decrypt only one name and the remaining names will still be encrypted.

I encrypt by salting my password key with a 256 bit sha has and then I use AES encryption on the data then i base 64 encode it and stick it into the db.


Regular Python Script:

def pad(string):
   return string + ((16-len(string) % 16) * '{' )


password = hashlib.sha256("hidmykey").digest() 
IV = 16 * '\x00'
mode = AES.MODE_CBC
decryptor = AES.new(password, mode, IV=IV)


config_values = metering_config('VMStats',['logfile','daemon', 'pguser','pgpassword','pghost','pgport'])     #
postgres = psycopg2.connect(database='metering', user=config_values['pguser'], password=config_values['pgpassword'],host=config_values['pghost'], port=config_values['pgport'])
pg_cursor = postgres.cursor()
pg_cursor.execute("set timezone TO 'GMT'")
storage_cursor_pg = postgres.cursor()

customer_list = []
pg_cursor.execute(''' select customer_name, customer_id from customers''')

for customer_name, customer_id in pg_cursor:
   data = base64.b64decode(customer_name)
   plain = decryptor.decrypt(data)
   lenofdec = plain.count('{')
   customer_list.append((plain[:len(plain)-lenofdec], customer_id))

this works perfectly fine.


DJANGO



#these are functions django views calls to decrypt the data.

def pad(string):
   return string + ((16-len(string) % 16) * '{' )

def show_name(customer_name):
   password = hashlib.sha256("hidden").digest() 
   IV = 16 * '\x00'
   mode = AES.MODE_CBC
   decryptor = AES.new(password, mode, IV=IV)
  
   data = base64.b64decode(customer_name)
   enc_customer_name = decryptor.decrypt(data)
   lenofdec = enc_customer_name.count('{')
   customer_name = enc_customer_name[:len(enc_customer_name)-lenofdec]
   return customer_name

The actual django page:


@login_required
def portal(request):
   #define var's
   customer_list = []

   user = request.user
   user_name = request.user.username
  
   user.is_superuser = check_user(user, user_name) #my own custom function to do some Active Directory stuff.
         
   if user.is_active and user.is_superuser:
      customers = Customers.objects.all().order_by('customer_name')
   else:
      customers = Customers.objects.get(customer_id=customer_id_filter)
     
   for customer in customers:
      customer_name = show_name(str(customer.customer_name)) #ive tried with and without making it a string.
      customer_list.append((customer_name, int(customer.customer_id)))
             
   current_month = date.today().month
                                                                                                                                                                                                                              
   context = Context({'customer_list': customer_list,
                         'user.is_superuser':user.is_superuser,
                         'current_month':current_month,
                          })
   return render(request, 'portal.html', context)


Here is the output:

as you see below ('TESTCUSTOMER', 81) is decrypted in real life that is a customer name that gets decrypted and its always the same name. However in my python script outside of django goes through them all and decrypts them.

[('\xa9\xdc\xbf\x86#\x7f4\xc7\xcc\xf1q\x8f3\xe2X\xf3', 26), ('\x8a\xbc\x16\x90\r-B\x7f\xabU\x1d]\xb9M\xc5\xe3', 142), ('g\x04\xa3aMk\xe49\xb7\xba\xf2Q4\xc6\xc6\xac', 25), ("\x92\xbbqA\x0fvE\x94S}\x01|\\BX'", 41), ('\x07\x9a\xe9\xb1G\\\x8e\xa0\xbe\x83A\x86/\xe5%\xb7', 101), ('a\xf4\x88\xc0H\xd3\x87\xe1\x05\x18\xe1X\xb8\xe6\xf3\xf3', 23), ('l\xe7\x9c\xd0\x0eOz\x8d\x85\x94\xeb\xd9\x1c\xc2\x08\xc8', 38), ('\xd0&\x88,T\xe2(\xac7\xbd\xaa\x90s\x17\xec\xae', 31), ('\x15"\xf3"\xaf[*\xa8\xe0\xb0;L\xb5\x98\x1a\xef', 30), ('6\x15$\xddH\xdbw\xef\x03\xec\x8bk,\xbc2\x8a', 32), ('\x1f\xf4\x08S\x1c\xa5\xf0\x11L\x1b\xbe\xe5Cq\xea3', 28), ('\xda\x99\xd6\xa8:3bc\x96\x90u\xca\xd3\xc1|L', 22), ('\xe3*\x9d\x95\x83\x8c\xe4O\x99\x03\x12\xd4\x87\x1f\xc1\xa7', 35), (':\x8f\xbc#\x81g\xe5,fk\xca\x0b9\xca\x99&', 24), ('0 \t\xfb\xafAT\xae\x8d\x0cW\x86\xa2\xf9\x11\x8f', 21), ('\xce\xe3\x17^\xc5\xabI\x9aT\x80\xcaIm\xc4\xcaS', 36), ("tA\xe9\xe5S\x90z\xbcw'\x86\x17\x98\xab=[", 37), ("\\m\xf9e\xe9\xff\xe5X\xfa\xc9@\xc8'Jh\x81", 34), ('\x15)h\x12\x06\xc6\xfd\x86\xcd\x81\xfa\x10\xb6\xad\xbc\x9f', 121), ('TESTCUSTOMER', 81), ('\x15\x8b\x131\xd8\x1c\x04\xf7\x01\xb2\x8b\x84t\x11l\x07', 61), ('XU\xed+\xad*}\\L[M_\r$`\xc9', 141), ('l\xc7\xa3\xfc\xc6\xc3\xc5\xe6\xf3\x1ff\nq\x8d\x18"', 29), ('k\xb8/}Y\xbb\xf5\xe7`R8N(\x81pWy', 33), ('\x90\n\x07\xe8\xd2R6\x89L\xd8\xe5\x92\x95Q %', 1), ('\x92 \xad8|!\x82?\xa92\xc3\xe4\x8b7\xb4\x97', 27), ('1\xbd\x1c\xd1\xd0\xd1\xe7\x89\xa1\xcbKo\x16bW\x9e', 39)]

[Cal Leeming [iops.io]]

For the record, this approach towards data security is completely wrong.

Consider using FDE on your databases, rather than encrypting individual fields.

Cal

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/b25146c0-3819-4dc5-b1c5-875ec2d619f5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[G Z]

How is encrypting at the application level wrong? The research I have done, says that application level encrypting is better because its encrypted over a link, and this is why most web apps use the salt method. I would love to hear more about what your talking about I'm still learning.

Also whats wrong with my code tho?

[Cal Leeming [iops.io]]

Encrypting data such as passwords is one acceptable use case. However encrypting other types of data, such as name/phone numbers etc, is poor because you lose many search/indexing capabilities. For example, you wouldn't be able to perform wildcard searches, you could only do "case sensitive + exact match". 

It would also appear you are storing this data as base64 hex representation in a string field, which is wasting a lot of page space, although this could be resolved by using a BINARY/VARBINARY and forcing the raw bytes instead of hex repr.

As for data being encrypted over a link, this really doesn't provide any protections against that at all. If you're worried about link security, then you should look at using IPSec instead. 

It's difficult to say why your script isn't working, as the code is quite messy with no commenting/documentation. Perhaps try starting again from scratch, comment/document as you go along, and then compare each step with your working Python code to see where it's breaking.

Cal

[G Z]

I know exactly what my code is doing, and it works in just python something django is doing is screwing it up.

[G Z]

I'll post some more on this later, I'll post comments and be more descriptive.

[Collin Anderson]

Do you need to re-initialize your decryptor each time in your django for loop, like you do in pure python?

Are you sure you're using the same database and key?

Is the base64 decoded, encrypted data the same between pure python and django before you decrypt it?

Same python version? Do you have __future__ imports at the top of any of the files?