Here are my test results:
cPickle ziplvl: 0
dump: 1.671010s
load: 0.764567s
size: 3297275
cPickle ziplvl: 1
dump: 2.033570s
load: 0.874783s
size: 935327
json ziplvl: 0
dump: 0.595903s
load: 0.698307s
size: 2321719
json ziplvl: 1
dump: 0.667103s
load: 0.795470s
size: 458030
marshal ziplvl: 0
dump: 0.118067s
load: 0.314645s
size: 2311342
marshal ziplvl: 1
dump: 0.315362s
load: 0.335677s
size: 470956
marshal ziplvl: 2
dump: 0.318787s
load: 0.380117s
size: 457196
marshal ziplvl: 3
dump: 0.350247s
load: 0.364908s
size: 446085
marshal ziplvl: 4
dump: 0.414658s
load: 0.318973s
size: 437764
marshal ziplvl: 5
dump: 0.448890s
load: 0.350013s
size: 418712
marshal ziplvl: 6
dump: 0.516882s
load: 0.367595s
size: 409947
marshal ziplvl: 7
dump: 0.617210s
load: 0.315827s
size: 398354
marshal ziplvl: 8
dump: 1.117032s
load: 0.346452s
size: 392332
marshal ziplvl: 9
dump: 1.366547s
load: 0.368925s
size: 391921
The results do not include datastore operations,
#!/usr/bin/env python # # Copyright 2012 MiuMeet AG # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # from google.appengine.api import datastore_types from google.appengine.ext import db import zlib import marshal MARSHAL_VERSION = 2 COMPRESSION_LEVEL = 1 class JsonMarshalZipProperty(db.BlobProperty): """Stores a JSON serializable object using zlib and marshal in a db.Blob""" def default_value(self): return None def get_value_for_datastore(self, model_instance): value = self.__get__(model_instance, model_instance.__class__) if value is None: return None return db.Blob(zlib.compress(marshal.dumps(value, MARSHAL_VERSION), COMPRESSION_LEVEL)) def make_value_from_datastore(self, value): if value is not None: return marshal.loads(zlib.decompress(value)) return value data_type = datastore_types.Blob def validate(self, value): return value
Nice. I like it. Going to try playing with your method vs some of the things I tried. But I suspect you are faster.
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/lohoWLsuVD0J.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
I took your class and made the tiny modifications for it to use pickle instead of marshal and it is working well for me. Thanks.
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/E7uCryqsk2QJ.
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
class JsonMarshalZipProperty(ndb.BlobProperty):
def _to_base_type(self, value):
return zlib.compress(marshal.dumps(value, MARSHAL_VERSION))
def _from_base_type(self, value):
return marshal.loads(zlib.decompress(value))
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/qKSg7YkFW5YJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.