Urlnet on Mac OS 10.6

[Bruno Amaral]

I have tried to install urlnet on my mac,now running snow leopard.

For some reason, it doesn't find the required files, altough the chkurlnetcfg.py file shows the following:

Python version: 2.6.1
System OS: Darwin (64bit)
Processor: i386
Node: Thing.local
simplejson module not available, some examples will not work
twitter module not available, some examples will not work
Ft.Xml module not available, some examples will not work
HOME env var: </Users/brunoamaral>
Search paths that will be checked:
    .
    /Users/brunoamaral
    /usr/bin
    /bin
    /usr/sbin
    /sbin
    /usr/local/bin
    /usr/X11/bin
trying to open : /usr/bin/urlnet.cfg
001: # UrlNet config file
002:
003: # syntax of an entry is <name><equals-sign><value>
004: # Any line not in that format is ignored.
005: # Any line that starts with '#' will be ignored.
006: # <name> is not case-sensitive, but <value> is returned with case left intact
007: # whitespace is tripped from both <name> and <value>
008:
009: # make sure you set these to values appropriate to your system!
010:
011: workingDir=/Users/brunoamaral/Desktop/
012: _email=ma...@brunoamaral.com
013:
014: # out-of-the-box, the entries in this file are set to the defaults that
015: # will be used if the config file is not found.
016:
017: default_socket_timeout=15.0
018: sleeptime=0
019: userAgent=Mozilla/4.0 (compatible; MSIE 5.5; Windows NT; UrlTree v0.90.06)
020:
021: # obtain your own Technorati API key at http://technorati.com/developers/
022: technoratiKey=439315d647ad542c9a15223c0f2dceed
023:
*** name-value pairs ***
  _email: ma...@brunoamaral.com
  sleeptime: 0
  workingdir: /Users/brunoamaral/Desktop/
    (directory /Users/brunoamaral/Desktop/ is writeable)
  default_socket_timeout: 15.0
  technoratikey: 439315d647ad542c9a15223c0f2dceed
  useragent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT; UrlTree v0.90.06)

I understand that the system is running 2.6, but the same thing happens when I run python 2.5.4.

Not sure if the procedure to install urlnet is different on the 10.6 version of OS X, but any ideas you guys may have will be greatly appreciated.

--
Bruno Amaral

www.brunoamaral.eu
ma...@brunoamaral.com
(+351) 91 287 58 56

--
You received this message because you are subscribed to the Google Groups "UrlNet Python Library" group.
To post to this group, send email to urlnet-pyt...@googlegroups.com.
To unsubscribe from this group, send email to urlnet-python-li...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/urlnet-python-library?hl=en.

[Dale Hunscher]

Hi Bruno,

I'm using Snow Leopard too, and though I still use Python 2.5, I don't
think 2.6 should be a problem.

Where is your Python install's site-packages directory? Mine is at
/Library/Python/2.5/site-packages. That's where my Ft module got
installed. Yours would have 2.6 instead of 2.5 but otherwise should be
the same.

If you open Terminal and start python from the command line, import
sys, then type in sys.path, you can see the module search path as a
list of folders:

Dale-Hunschers-Cielo-MacBook-Pro:~ dalehunscher$ python
Python 2.5.4 (r254:67916, Feb 11 2010, 00:50:55)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/Library/Python/2.5/site-packages/setuptools-0.6c9-py2.5.egg',
'/Library/Python/2.5/site-packages/simplejson-2.0.9-py2.5-macosx-10.5-i386.egg',
'/Library/Python/2.5/site-packages/python_twitter-0.6-py2.5.egg',
'/Library/Python/2.5/site-packages/docutils-0.6-py2.5.egg',
'/Library/Python/2.5/site-packages/virtualenv-1.4.3-py2.5.egg',
'/Library/Python/2.5/site-packages/lxml-2.2.6-py2.5-macosx-10.6-i386.egg',
'/Library/Python/2.5/site-packages/PyGreSQL-4.0-py2.5-macosx-10.6-i386.egg',
'/Library/Python/2.5/site-packages/reportlab-2.4-py2.5-macosx-10.6-i386.egg',
'/Library/Python/2.5/site-packages/PIL-1.1.7-py2.5-macosx-10.6-i386.egg',
'/Library/Python/2.5/site-packages/PyXML-0.8.4-py2.5-macosx-10.6-i386.egg',
'/Library/Python/2.5/site-packages/ZSI-2.1_a1-py2.5.egg',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-darwin',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/lib-scriptpackages',
'/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload',
'/Library/Python/2.5/site-packages',
'/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/PyObjC',
'/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/wx-2.8-mac-unicode']
>>>

Let me know if this isn't enough info to get things working. I really
like working on Snow Leopard, but it did take a bit of doing to get
everything working right Python-wise.

Dale Hunscher
---
Dale A. Hunscher, MSI
CTO, Cielo Medsolutions LLC
3520 Green Ct.
Suite 150
Ann Arbor, MI 48105
Cell: (734) 678-5178
Office: (734) 827-1000 x5679
Fax: (734) 661-2668
http://www.cielomedsolutions.com

[Bruno Amaral]

Dale, that worked together with me adding the urlnet library to both 2.5 and 2.6.

From my research on the problem, it seems that Apple ships their OS with outdated python libraries (not really their fault, only a matter of out-of-sync development cycles).

By the way ...

Sorry if this is too off topic, but is it possible to give URLnet a list of URLs and have it return the following:

http://www.urlonthelist1.com/; Page html title

http://www.urlonthelist2.com/; Page html title

?

I have been doing this with beatiful soup, who crashes at the slightest malformed html tag.

warm regards, 

bruno

-- 

Bruno Amaral

www.brunoamaral.eu
ma...@brunoamaral.com
(+351) 91 287 58 56

[Dale Hunscher]

Let me see what I can do with this. It shouldn't be very hard - we'll
see if it works better than Beautiful Soup. I think it should.

Dale

[Bruno Amaral]

I actually managed to solve this the other day and forgot to post:

Filename:gettitles.sh

#!/bin/sh

while read line

do

  url=$line

  title=$(curl -s $line | awk '/<title>/' | cut -d '>' -f 2 | cut -d '<' -f 1)

  echo $url";"'"'$title'"'

done < $1

exit 0

usage:

./gettitles.sh listfile

the list file must contain one url per line and it will output them one by one followed by the title :)

--
Bruno Amaral

www.brunoamaral.eu
ma...@brunoamaral.com
(+351) 91 287 58 56

[Tom Smith]

I have been doing this with beatiful soup, who crashes at the slightest malformed html tag.

Malformed html can be a huge problem... I think BeautifulSoup can help though ...

soup = BeautifulSoup(html)

html = soup.prettify() 

I also use any2ascii to bash things into shape...

Using this at the top of your python file ...

import sys, string, socket, urllib, codecs, sys

streamWriter = codecs.lookup('utf-8')[-1]

sys.stdout = streamWriter(sys.stdout)

...helps with the Terminal failing on print statements

Tom

[Bruno Amaral]

Had not thought of that, thank you Tom! :)

--
Bruno Amaral

www.brunoamaral.eu
ma...@brunoamaral.com
(+351) 91 287 58 56