Sync local Database with remote REST API

197 views
Skip to first unread message

Leonard

unread,
Mar 17, 2014, 7:40:01 PM3/17/14
to django...@googlegroups.com

Hello everybody, I'm having an hard time to figuring this out.
I'm designing a django project based on data retrieved from a remote public REST API that will:

  • Retrieve data from a REST API
  • Saving the data in my models (my local db)
  • Analyze and process the local data asynchronously and create statistics and trends
When a user request the data:
  • Check if the local data (models) are in sync with the remote, otherwise fetch the remote data and save them in the local db
  • Present the data
What is the best approach to do this? 
My questions:
  1. Should I design my models replicating all the remote API fields?
    • Can I use JSonField and store data in json format to speed things up?
      • In this case would it be complex to calculate statistics on the data?
        In other words I can't use the django ORM and foreign keys if I store data in JSON format, right?
  2. Where should I call the remote API?
    • Considering that the remote data should "replicate" the database models, first I tough to add model methods that call the REST API inside the models
    • Or I could put them in the manager, considering that when a user needs the data he/she doesn't need to know if they are local or remote
    • How do I tell django to grab the remote data and put them in db before presenting them?
      • Maybe this should be an asynchronous task? 
Sorry for the complexity, I hope I was clear!
Thanks in advance for your help

Eron Lloyd

unread,
Aug 13, 2017, 6:43:38 PM8/13/17
to Django users
Has anyone ever come up with a best practices approach for this? Particularly if the remote service is the primary one.
Reply all
Reply to author
Forward
0 new messages