Mapping None. Why?
Paddy
Iam wondering why the peculiar behavior of map when the function in
given as None:
Help on built-in function map in module __builtin__:
map(...)
map(function, sequence[, sequence, ...]) -> list
Return a list of the results of applying the function to the items
of
the argument sequence(s). If more than one sequence is given, the
function is called with an argument list consisting of the
corresponding
item of each sequence, substituting None for missing values when
not all
sequences have the same length. If the function is None, return a
list of
the items of the sequence (or a list of tuples if more than one
sequence).
It seems as the action whith none is the same as using a function of
lambda *x: x
As in the following example:
>>> l1 = 'asdf'
>>> l2 = 'qwertyuip'
>>> l3 = range(3)
>>> l1,l2,l3
('asdf', 'qwertyuip', [0, 1, 2])
>>> map(lambda *x: x, l1,l2,l3) == map(None, l1,l2,l3)
True
>>>
On looking up map on Wikipedia there is no mention of this special
behaviour,
So my question is why?
Thanks, Paddy.
Diez B. Roggisch
Because it is undefined what should happen in case of no function given
at all - and because there is no identity function in python
pre-defined, it could be considered sensible to make None the quivalent
of that function.
And it only follows that *if* you imply a function even though there is
None given, that the passed tuple is returned.
I don't see anything on wikipedia that defines any other behavior.
Diez
Diez
Ian Kelly
>
> Iam wondering why the peculiar behavior of map when the function in
> given as None:
Because that's the way it's always been! Seriously, I don't know. I
can tell you that it's going away in Python 3.0, though.
Ian
Ian Kelly
> Because it is undefined what should happen in case of no function given at
> all - and because there is no identity function in python pre-defined, it
> could be considered sensible to make None the quivalent of that function.
It makes more sense to raise an error when a non-function is passed
where a function is expected. If we're going to have a special
behaviour for None, why not have special behaviours for True, False,
and 42 as well? The proper solution to the lack of a built-in packing
(not identity) function is to define a packing function, not to
special-case an arbitrary value to *mean* the packing function in
certain situations.
Ian
Robert Kern
>> given as None:
>
> can tell you that it's going away in Python 3.0, though.
There was a time before zip(). Basically, it's a really useful feature, and the
original implementor thought that map() was a reasonable place to put it; it's a
somewhat natural outgrowth of the map(func, list1, list2, ... listn) semantics.
Since then, after more thought was put into it, it was realized that a separate
builtin function is more appropriate for this common use, and thus zip() was
born and map(None, ...) was deprecated.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
Diez B. Roggisch
You are right with the packing-function, it's not identity of course.
However I don't see that this as an area that is really important
(especially since map is being replaced by list-comps most of the time).
And the OP's question was about map not being conforming to the
definition on wikipedia - which I don't think it's not. It is not
defined what map is to do with None (or NULL or nil or... ) as argument.
Diez
Terry Reedy
"Paddy" <padd...@googlemail.com> wrote in message
news:a8874705-9f90-4acb...@27g2000hsf.googlegroups.com...
|
| Iam wondering why the peculiar behavior of map when the function in
| given as None:
The 'peculiar behavior' is the same as zip (except for padding short
iterators versus truncating long iterators. Map was added years before
zip. After that, map(None,...) was kept for back compatibility.
In 3.0, the doc for map is
"Return an iterator that applies function to every item of iterable,
yielding the results. If additional iterable arguments are passed, function
must take that many arguments and is applied to the items from all
iterables in parallel. With multiple iterables, the iterator stops when the
shortest iterable is exhausted."
Using a map defined with None raises
TypeError: 'NoneType' object is not callable
tjr
Robert Kern
> On looking up map on Wikipedia there is no mention of this special
> behaviour,
> So my question is why?
My question is why you are looking up the semantics of Python functions on
Wikipedia instead of the Python documentation. I don't see any particular
discussion of map() there at all. Am I missing something?
Paddy
>
> And the OP's question was about map not being conforming to the
> definition on wikipedia - which I don't think it's not. It is not
> defined what map is to do with None (or NULL or nil or... ) as argument.
>
> Diez
Oh no!
Sorry to give that impression. I don't think that map should be like
what Wikipedia says, I was just looking for another example of an
implementation that might mention the behaviour.
I just want to know the thoughts behind this behaviour in the Python
map.
- Paddy.
Paddy
> Paddy wrote:
> > On looking up map on Wikipedia there is no mention of this special
> > behaviour,
> > So my question is why?
>
> My question is why you are looking up the semantics of Python functions on
> Wikipedia instead of the Python documentation. I don't see any particular
> discussion of map() there at all. Am I missing something?
>
> --
> Robert Kern
As I said in an answer to Diez B. Roggish, I had been reminded of the
behaviour, thought it odd, and looked for other implementations that
might have the behaviour via wikipedia. My intension was most
definitely NOT to say that Pythons map should do what Wikipedia says
slavishly.
Sometimes when I pick at these seeming inconsistencies I learn a lot
from the c.l.p replies. Someone elses thread on -0.0 versus +0.0
taught me some more on floating point for example.
- Paddy.
Paddy
> "Paddy" <paddy3...@googlemail.com> wrote in message
I really should get into the habit of reading the 3.0 docs before
asking questions :-)
My original question came about after answering this query:
http://gmcnaughton.livejournal.com/27955.html?thread=70451#t70451
- Paddy.
David C. Ullrich
<padd...@googlemail.com> wrote:
>
>Iam wondering why the peculiar behavior of map when the function in
>given as None:
If you start with a value x and then apply no function
at all to it, what results is x.
David C. Ullrich
Paddy
> On Thu, 12 Jun 2008 12:05:02 -0700 (PDT), Paddy
>
>
> >Iam wondering why the peculiar behavior of map when the function in
> >given as None:
>
> If you start with a value x and then apply no function
> at all to it, what results is x.
>
True, but None is not a function. It's a sentinel value to turn on the
functionality.
- Paddy.
David C. Ullrich
<d5897630-558e-428a...@j22g2000hsf.googlegroups.com>,
Paddy <padd...@googlemail.com> wrote:
Uh, thanks. I think I knew that - I was just suggesting why
the way map works makes sense.
> - Paddy.
--
David C. Ullrich
bruce
got a short test app that i'm playing with. the goal is to get data off the
page in question.
basically, i should be able to get a list of "tr" nodes, and then to
iterate/parse them. i'm missing something, as i think i can get a single
node, but i can't figure out how to display the contents of the node.. nor
how to get the list of the "tr" nodes....
my test code is:
--------------------------------
#!/usr/bin/python
#test python script
import re
import libxml2dom
import urllib
import urllib2
import sys, string
from mechanize import Browser
import mechanize
#import tidy
import os.path
import cookielib
from libxml2dom import Node
from libxml2dom import NodeList
########################
#
# Parse pricegrabber.com
########################
# datafile
tfile = open("price.dat", 'wr+')
efile = open("price_err.dat", 'wr+')
urlopen = urllib2.urlopen
##cj = urllib2.cookielib.LWPCookieJar()
Request = urllib2.Request
br = Browser()
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
values1 = {'name' : 'Michael Foord',
'location' : 'Northampton',
'language' : 'Python' }
headers = { 'User-Agent' : user_agent }
url ="http://www.pricegrabber.com/rating_summary.php/page=1"
#=======================================
if __name__ == "__main__":
# main app
txdata = None
#----------------------------
# get the kentucky test pages
#br.set_cookiejar(cj)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)
br.addheaders = [('User-Agent', 'Firefox')]
br.open(url)
#cj.save(COOKIEFILE) # resave cookies
res = br.response() # this is a copy of response
s = res.read()
# s contains HTML not XML text
d = libxml2dom.parseString(s, html=1)
print "d = d",d
#get the input/text dialogs
#tn1 = "//div[@id='main_content']/form[1]/input[position()=1]/@name"
t1 =
"/html/body/div[@id='pgSiteContainer']/div[@id='pgPageContent']/table[2]/tbo
dy"
tr =
"/html/body/div[@id='pgSiteContainer']/div[@id='pgPageContent']/table[2]/tbo
dy/tr[4]"
tr_=d.xpath(tr)
print "len =",tr_[1].nodeValue
print "fin"
-----------------------------------------------
my issue appears to be related to the last "tbody", or tbody/tr[4]...
if i leave off the tbody, i can display data, as the tr_ is an array with
data...
with the "tbody" it appears that the tr_ array is not defined, or it has no
data... however, i can use the DOM tool with firefox to observe the fact
that the "tbody" is there...
so.. what am i missing...
thoughts/comments are most welcome...
also, i'm willing to send a small amount via paypal!!
-bruce
Dan Stromberg
BeautifulSoup is a pretty nice python module for screen scraping (not
necessarily well formed) web pages.
Paul Boddie
>
> url ="http://www.pricegrabber.com/rating_summary.php/page=1"
[...]
> tr =
> "/html/body/div[@id='pgSiteContainer']/div[@id='pgPageContent']/table[2]/tbo
> dy/tr[4]"
>
> tr_=d.xpath(tr)
[...]
> my issue appears to be related to the last "tbody", or tbody/tr[4]...
>
> if i leave off the tbody, i can display data, as the tr_ is an array with
> data...
Yes, I can confirm this.
> with the "tbody" it appears that the tr_ array is not defined, or it has no
> data... however, i can use the DOM tool with firefox to observe the fact
> that the "tbody" is there...
Yes, but the DOM tool in Firefox probably inserts virtual nodes for
its own purposes. Remember that it has to do a lot of other stuff like
implement CSS rendering and DOM event models.
You can confirm that there really is no tbody by printing the result
of this...
d.xpath("/html/body/div[@id='pgSiteContainer']/
div[@id='pgPageContent']/table[2]")[0].toString()
This should fetch the second table in a single element list and then
obviously give you the only element of that list. You'll see that the
raw HTML doesn't have any tbody tags at all.
Paul
Terry Reedy
"David C. Ullrich" <dull...@sprynet.com> wrote in message
news:dullrich-571EC3...@text.giganews.com...
filter(None, iterable) works the same way: None-> identity function,
The immediate reason is the Python has no builtin id().
But apparently there is also historical precedent in the functional
community for this convention.
Ian Kelly
> filter(None, iterable) works the same way: None-> identity function,
> The immediate reason is the Python has no builtin id().
> But apparently there is also historical precedent in the functional
> community for this convention.
Another way of viewing it is that filter(None, iterable) applies no
function at all before testing the truth values, which does make some
sense. With map, however, this is not strictly true.
bruce
Thanks for the reply. Came to the same conclusion a few minutes before I saw
your email.
Another question:
tr=d.xpath(foo)
gets me an array of nodes.
is there a way for me to then iterate through the node tr[x] to see if a
child node exists???
"d" is a document object, while "tr" would be a node object?, or would i
convert the "tr[x]" to a string, and then feed that into the
libxml2dom.parseString()...
thanks
[...]
[...]
Paul Boddie
>
> Thanks for the reply. Came to the same conclusion a few minutes before I saw
> your email.
>
> Another question:
>
> tr=d.xpath(foo)
>
> gets me an array of nodes.
>
> is there a way for me to then iterate through the node tr[x] to see if a
> child node exists???
You can always use the DOM or perform another XPath query:
for node in tr[x].childNodes:
<do something with node>
for node in tr[x].xpath(some_other_query_inside_tr):
<do something with node>
> "d" is a document object, while "tr" would be a node object?, or would i
> convert the "tr[x]" to a string, and then feed that into the
> libxml2dom.parseString()...
There's no need to parse anything again: just use the methods on the
object that tr[x] produces, including the xpath method, of course.
Remember that the document object is just a special node object, so
most of the methods are available on both. If in doubt, run your
program using Python's -i option and then inspect the objects at the
interactive prompt.
Paul
Larry Bates
FYI: Mechanize includes all of BeautifulSoup's methods and adds additional
functionality (like forms handling).
Larry