Entering initial data into django database

[Lawrence Muriuki]

Hi All,

I have built a web scraper to scrape a website and get all match fixtures for a particular day. Wanted to know how to insert that initial data into django database and where that code should live..I have currently put the code in init.py but it looks and sounds wrong according to a quick google search..

Any help will be appreciated..
Thank You

[ludovic coues]

There is a page in the documentation dealing with initial data [1].
The good news is that you don't have to write more code. You insert
your data in a file and load it with the manage.py command. Out of the
box, JSON, XML and SQL are supported.


[1] https://docs.djangoproject.com/en/1.9/howto/initial-data/

> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/3de1d2d2-b774-4c39-b5b8-2b9602cff829%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--

Cordialement, Coues Ludovic
+336 148 743 42

[Lawrence Muriuki]

Thanks ludovic, here is my code for the scraper..




 

from .models import Betting

import re
import requests
import urllib
from bs4 import BeautifulSoup


urls= "https://sms.betyetu.co.ke/tomorrow.html"
htmlfile = urllib.urlopen(urls)
htmltext = htmlfile.read()
soup = BeautifulSoup(htmltext, "html.parser")


for i in soup.findAll("table"):
    print i.contents[1].contents[1].findAll("th")[0].text
    print i.contents[1].contents[3].findAll("th")[0].text


   
   


       
    try:
        momo = 1
        a = len( i.contents[3].contents)
        while momo <= a:
            print i.contents[3].contents[momo].findAll("td")[2].text
           
            foo = Betting(matches=i.contents[3].contents[momo].findAll("td")[2].text)
           
            momo += 2

           
            foo.save()
           

       
   
    except:
        momo = 1

   



I have a model called "betting" which has only one field "matches"..For it to work, I have placed it in init.py which is a wrong way to do it..Wanted to understand where this code should be placed..ie..which file and how to interact with it

[ludovic coues]

First, you should create a fixture file with some sample data. Two
entry, just like in the documentation is good. This way, you know you
have a working fixture file.

Then, you write a python function which take a list of bet and output
a fixture file just like the exemple you have done.

Finally, you adjust you scraper to build the list, use the function to
transform the list into a fixture file and import the fixture file.

> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-users/d3677ab5-06eb-47aa-bdf8-d789a4c454b7%40googlegroups.com.

[Lawrence Muriuki]

Ok..thanks..it worked...Created a script to create the fixture file and update it with every record...what a community this is!