hi,
I am extracting data from other site and I want to do that my user agent should be change randomly whenever i run my python script for extracting data
my python file--
from bs4 import BeautifulSoup
import urllib2
import simplejson as json
import sys
from random import choice
def ext():
url = "
https://www.example.com/"
headers = [
{'User-Agent' : 'Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)'},
{'User-Agent' : 'Opera/9.80 (Windows NT 6.0) Presto/2.12.388 Version/12.11'},
{'User-Agent' : 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'},
{'User-Agent' : 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; ko; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2'}
]
headerRandom = choice(headers)
req = urllib2.Request(url, None, headerRandom)
page = urllib2.urlopen(req)
data = BeautifulSoup(page.read())
-------
.
.
.
It's working fine but but when I am tracking packet and method using wireshark then it's showing default firefox user agent string, That means it's not taking my own user agent.
I want- It should be change every time user agent randomly.
Please help me, how is it possible?
Thanks