when i import large data to db using django orm, the django orm use too many memory

37 views
Skip to first unread message

jingniao

unread,
Apr 1, 2018, 8:58:21 AM4/1/18
to Django users
django.db.backends.base.base.BaseDatabaseWrapper
    self.queries_limit is 9000
which will effect self.queries_log, this is a query log cache
when I import a large data to db use Models.objects.bulk_create function
the long sql will be cache, so will be out out memory

Jani Tiainen

unread,
Apr 1, 2018, 9:35:43 AM4/1/18
to django...@googlegroups.com
Hi,

You need to set DEBUG = False to disable SQL logging.

Better yet, don't use Django for large data import (it's not very suitable for that) but use your database tools for that.


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/b9f79199-181f-432f-af0e-6c36455ad282%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Jani Tiainen

- Well planned is half done, and a half done has been sufficient before...

jingniao

unread,
Apr 1, 2018, 10:34:12 AM4/1/18
to Django users
thanks
在 2018年4月1日星期日 UTC+8下午9:35:43,Jani Tiainen写道:
Hi,

You need to set DEBUG = False to disable SQL logging.

Better yet, don't use Django for large data import (it's not very suitable for that) but use your database tools for that.

On Sun, Apr 1, 2018 at 1:07 PM, jingniao <rulo...@gmail.com> wrote:
django.db.backends.base.base.BaseDatabaseWrapper
    self.queries_limit is 9000
which will effect self.queries_log, this is a query log cache
when I import a large data to db use Models.objects.bulk_create function
the long sql will be cache, so will be out out memory

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages