better way than: myPage += 'more html' , ...

0 views
Skip to first unread message

Gobo Borz

unread,
Jun 25, 2003, 12:20:27 PM6/25/03
to
Hi everyone,

I have a python cgi program that uses print statements to write html.
The program has grown, and for reasons I won't bore you with, I need to
build the page in a string and "print" it at once.

I remember reading somewhere that building the string with successive
"+=" statements is inefficient, but I don't know any alternatives, and
my search attempts came up empty. Is there a better, more efficient way
to do it than this:

#---------------------------------
htmlPage = "<html><header></header><body>"
htmlPage += "more html"
...
htmlPage += "even more html"
...
htmlPage += "</body></html>"
print htmlPage
#-------------------------------------

Thanks, Gobo

Alan Kennedy

unread,
Jun 25, 2003, 12:36:29 PM6/25/03
to
Gobo Borz wrote:

> I remember reading somewhere that building the string with successive
> "+=" statements is inefficient, but I don't know any alternatives, and
> my search attempts came up empty. Is there a better, more efficient way
> to do it than this:
>
> #---------------------------------
> htmlPage = "<html><header></header><body>"
> htmlPage += "more html"
> ...
> htmlPage += "even more html"
> ...
> htmlPage += "</body></html>"
> print htmlPage
> #-------------------------------------

Best to do something like

#---------------------------------

htmlbits = []
htmlbits.append("<html><header></header><body>")
htmlbits.append("more html")
# ...
htmlbits.append("even more html")
# ...
htmlbits.append("</body></html>")

# And now for the non-intuitive bit
print "".join(htmlbits)

#---------------------------------

HTH,

--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/mailto/alan

Anthony Baxter

unread,
Jun 25, 2003, 12:30:39 PM6/25/03
to

>>> Gobo Borz wrote

> I remember reading somewhere that building the string with successive
> "+=" statements is inefficient, but I don't know any alternatives, and
> my search attempts came up empty. Is there a better, more efficient way
> to do it than this:
>
> #---------------------------------
> htmlPage = "<html><header></header><body>"
> htmlPage += "more html"
> ...
> htmlPage += "even more html"
> ...
> htmlPage += "</body></html>"
> print htmlPage
> #-------------------------------------

Instead, build up a list of strings, then join them all at
the end - this way, the system only has to allocate the
space once. Much quicker.

#---------------------------------
htmlPage = []
htmlPage.append("<html><header></header><body>")
htmlPage.append("more html")
...
htmlPage.append("even more html")
...
htmlPage.append("</body></html>")
page = ''.join(htmlPage)
print page
#-------------------------------------

Anthony

Andy Jewell

unread,
Jun 25, 2003, 1:30:14 PM6/25/03
to


An easy way, which requires little changes tou your code would be to use a
list and join it up into a string in one go at the end:

-----------8<--------------
htmlPage=["<html><header></header><body>\n"]
htmlPage.append("more html\n")
....
htmlPage.append("more html\n")
....
htmlPage.append("<\body><\html>\n")
htmlDoc="".join(htmlPage)
-----------8<--------------

This is more efficient, as the string joining is done all at the same time,
rather than in little chunks.

hope that helps
-andyj


mirko...@zeibig.net

unread,
Jun 25, 2003, 2:48:31 PM6/25/03
to
In article <3EF9CBC...@yahoo.com>, Gobo Borz wrote:
> I remember reading somewhere that building the string with successive
> "+=" statements is inefficient, but I don't know any alternatives, and
> my search attempts came up empty. Is there a better, more efficient way
> to do it than this:
>
> #---------------------------------
> htmlPage = "<html><header></header><body>"
> htmlPage += "more html"
> ...
> htmlPage += "even more html"
> ...
> htmlPage += "</body></html>"
> print htmlPage
> #-------------------------------------
Hello Gobo,

you may use the list solution provided in the former chapters. Some other
solution would be to use string substitution:
--------- snip -----------
htmlPage = """
<html>
<head>
<title>%(title)s</title>
</head>
<body>
<h1>%(title)s</h1>
%(body)s
</body>
</html>
"""

body = []
body.append("<p>my first snippet</p>")
body.append("<ul>")
for in range(10):
body.append("""<li>%s. entry</li>""" % i)
body.append("</ul>")

contents = { "title": "My Foopage",
"body": "\n".join(body) }

print htmlPage % contents
-------- snap -------------

This type of substitution goes for a very simple templating system.

Best Regards
Mirko

Gobo Borz

unread,
Jun 25, 2003, 2:52:02 PM6/25/03
to
Thanks Alan, Anthony, and Andy! B's thru Z's need not respond, I have
my answer.

--Gobo

Gobo Borz

unread,
Jun 25, 2003, 3:02:08 PM6/25/03
to
Thanks, that opens up a world of possibilities I haven't began to
explore. In general, I'm not fond of templating systems because of the
overhead, but it seems as your suggestion might be fast, since it uses
ordinary string substitution.

--Gobo

Dave Kuhlman

unread,
Jun 25, 2003, 5:07:48 PM6/25/03
to
Andy Jewell wrote:

[snip]

> An easy way, which requires little changes tou your code would be
> to use a list and join it up into a string in one go at the end:
>
> -----------8<--------------
> htmlPage=["<html><header></header><body>\n"]
> htmlPage.append("more html\n")
> ....
> htmlPage.append("more html\n")
> ....
> htmlPage.append("<\body><\html>\n")
> htmlDoc="".join(htmlPage)
> -----------8<--------------

If you find that you are explicitly adding a new-line at the end
of every string that you append, as a convenience, you might want
to consider using the following minor modification:

-----------8<--------------
import string
...
htmlPage=["<html><header></header><body>"]
htmlPage.append("more html")
....
htmlPage.append("more html")
....
htmlPage.append("<\body><\html>")
# Use string.join() instead of ''.join() to add new-lines.
htmlDoc= string.join(htmlPage, '\n')
-----------8<--------------

Funny that string.join() supports the optional second parameter,
while 'abc'.join() does not.

- Dave

--
Dave Kuhlman
http://www.rexx.com/~dkuhlman
dkuh...@rexx.com

Irmen de Jong

unread,
Jun 25, 2003, 5:20:02 PM6/25/03
to
Dave Kuhlman wrote:
> Funny that string.join() supports the optional second parameter,
> while 'abc'.join() does not.

Use: "\n".join(yourlist) ...

--Irmen

Max M

unread,
Jun 25, 2003, 5:44:28 PM6/25/03
to Gobo Borz
Gobo Borz wrote:
> Thanks Alan, Anthony, and Andy! B's thru Z's need not respond, I have
> my answer.

Actually there are yet a few twists to put on this.

# renaming the append method makes it a bit easier on the eye.

htmlPage=[]
a = htmlPage.append # ah dynamic languages

a("<html><header></header><body>")
a("more html")
....
a("more html")
....
a("<\body><\html>")
htmlDoc="\n".join(htmlPage)


# using cStringIO should be the fastest approach

import cStringIO
io = cStringIO()
a = io.write

a("<html><header></header><body>\n")
a("more html\n")
....
a("more html\n")
....
a("<\body><\html>\n")
htmlDoc = io.getvalue()
io.close()


regards Max M

Gerrit Holl

unread,
Jun 25, 2003, 5:23:51 PM6/25/03
to
Gobo Borz wrote:
> I have a python cgi program that uses print statements to write html.
> The program has grown, and for reasons I won't bore you with, I need to
> build the page in a string and "print" it at once.

Another way, which has not yet been mentioned but which I like much,
is the cStringIO module. You can write to a string as if it is a
file:

0 >>> import cStringIO
1 >>> mystr = cStringIO.StringIO()
2 >>> mystr.write("<html")
3 >>> mystr.write("<body>")
4 >>> mystr.write("<h1>Header</h1>")
5 >>> mystr.write("<p>Hello, world!</p>")
6 >>> mystr.write("</body></html>")
10 >>> mystr.getvalue()
'<html<body><h1>Header</h1><p>Hello, world!</p></body></html>'

The cStringIO module is documented at:

http://www.python.org/dev/doc/devel/lib/module-StringIO.html

cStringIO is a faster C implementation with the same API.

yours,
Gerrit.

--
196. If a man put out the eye of another man, his eye shall be put out.
-- 1780 BC, Hammurabi, Code of Law
--
Asperger Syndroom - een persoonlijke benadering:
http://people.nl.linux.org/~gerrit/
Het zijn tijden om je zelf met politiek te bemoeien:
http://www.sp.nl/

sism...@hebmex.com

unread,
Jun 25, 2003, 5:14:24 PM6/25/03
to
> From: Dave Kuhlman [mailto:dkuh...@rexx.com]
> Sent: Miércoles, 25 de Junio de 2003 04:08 p.m.

>
> -----------8<--------------
> import string
> ...
> htmlPage=["<html><header></header><body>"]
> htmlPage.append("more html")
> ....
> htmlPage.append("more html")
> ....
> htmlPage.append("<\body><\html>")
> # Use string.join() instead of ''.join() to add new-lines.
> htmlDoc= string.join(htmlPage, '\n')
> -----------8<--------------
>
> Funny that string.join() supports the optional second parameter,
> while 'abc'.join() does not.
>
> - Dave
>

That's because string.join() is a module function, and in the
other case, ".join()" is a method of the string object "abc",
so in that case, "abc" is the "self" parameter.

And, in the case of string.join()'s optional second parameter,
it's default value is "", an empty string (if I'm not mistaken).

It's the same thing as "".join()

Salutations!

-gustavo

---

Advertencia:La informacion contenida en este mensaje es confidencial y
restringida, por lo tanto esta destinada unicamente para el uso de la
persona arriba indicada, se le notifica que esta prohibida la difusion de
este mensaje. Si ha recibido este mensaje por error, o si hay problemas en
la transmision, favor de comunicarse con el remitente. Gracias.

Max M

unread,
Jun 25, 2003, 6:13:01 PM6/25/03
to Gobo Borz
#Gobo Borz wrote:
#> Thanks, that opens up a world of possibilities I haven't began to
#> explore. In general, I'm not fond of templating systems because of the
#> overhead, but it seems as your suggestion might be fast, since it uses
#> ordinary string substitution.
#

Often an adapter is a good choice:


class HtmlViewAdapter:

"""
Base class that Adapts any object to html output.
"""

def __init__(self, obj):
self._obj = obj

def __getattr__(self, attr):
"Returns values for the attributes"
return getattr(self._obj, attr)

def __getitem__(self, item):
"get attrs as items for use in string formatting templates"
value = getattr(self, item)
if callable(value):
return value()
else:
return value

import time

class SomeObject:

"The object we want displayed in html"


def __init__(self):
self.id = 42
self.now = time.localtime()
self.title = 'the title'

class SomeObjectHtmlView(HtmlViewAdapter):

"""
an adapter for the object, here you can fine adjust, format dates
etc.
"""

def title(self):
return self._obj.title.upper()

def now(self):
return time.strftime('%y:%m:%d %H-%M-%S',self._obj.now)


view = SomeObjectHtmlView(SomeObject())

print '<a href="%(id)s">%(title)s</a> <i>%(now)s</i>' % view
>>> <a href="42">THE TITLE</a> <i>03:06:26 00-12-16</i>

Delaney, Timothy C (Timothy)

unread,
Jun 25, 2003, 8:30:21 PM6/25/03
to
> From: Gerrit Holl [mailto:ger...@nl.linux.org]

>
> Another way, which has not yet been mentioned but which I like much,
> is the cStringIO module. You can write to a string as if it is a
> file:
>
> 0 >>> import cStringIO
> 1 >>> mystr = cStringIO.StringIO()
> 2 >>> mystr.write("<html")
> 3 >>> mystr.write("<body>")
> 4 >>> mystr.write("<h1>Header</h1>")
> 5 >>> mystr.write("<p>Hello, world!</p>")
> 6 >>> mystr.write("</body></html>")
> 10 >>> mystr.getvalue()
> '<html<body><h1>Header</h1><p>Hello, world!</p></body></html>'
>
> The cStringIO module is documented at:
>
> http://www.python.org/dev/doc/devel/lib/module-StringIO.html
>
> cStringIO is a faster C implementation with the same API.

Indeed, the following might make things nice and easy for you ...

import cStringIO as StringIO

# Subclass of str so that it can be used nearly anywhere that requires a real
# string.

class MutableString (str):

def __init__ (self, s=""):
super(MutableString, self).__init__()
self.data = StringIO.StringIO()
self.data.write(s)

def __add__(self, s):
m = MutableString(self.data.getvalue())
m.data.write(s)
return m

def __iadd__(self, s):
self.data.write(s)
return self

def __str__(self):
return self.data.getvalue()

def __repr__(self):
return repr(str(self))

s = MutableString("Hello,")
s += " world!"
print s
print repr(s)

Thomas Güttler

unread,
Jun 26, 2003, 2:56:51 AM6/26/03
to
Gobo Borz wrote:

> Hi everyone,
>
> I have a python cgi program that uses print statements to write html.
> The program has grown, and for reasons I won't bore you with, I need to
> build the page in a string and "print" it at once.

Hi!

I do it like this:

def foo(mydict):
ret=[]

rows=[]
for key, value in mydict.items():
rows.append('<tr><td>%s</td><td>%s</td></tr>' % (key, value))
rows=''.join(rows)

ret.append('<table>%s</table>' % rows)
return ''.join(ret)

I try to keep the start-tag and the end-tag in one string.
This keeps the code from becomming ugly.

thomas

Gerrit Holl

unread,
Jun 26, 2003, 5:25:10 AM6/26/03
to
Delaney, Timothy C (Timothy) wrote:
> import cStringIO as StringIO
>
> # Subclass of str so that it can be used nearly anywhere that requires a real
> # string.
>
> class MutableString (str):
>
> def __init__ (self, s=""):
> super(MutableString, self).__init__()
> self.data = StringIO.StringIO()
> self.data.write(s)
>
> def __add__(self, s):
> m = MutableString(self.data.getvalue())
> m.data.write(s)
> return m
>
> def __iadd__(self, s):
> self.data.write(s)
> return self
>
> def __str__(self):
> return self.data.getvalue()
>
> def __repr__(self):
> return repr(str(self))

You could even add __setitem__ and __getitem__ and implement it with
.seek.

yours,
Gerrit.

--
62. If he do not plant the field that was given over to him as a
garden, if it be arable land (for corn or sesame) the gardener shall pay
the owner the produce of the field for the years that he let it lie
fallow, according to the product of neighboring fields, put the field in
arable condition and return it to its owner.

François Pinard

unread,
Jun 26, 2003, 10:30:34 AM6/26/03
to
[Max M]

> I remember reading a post where I saw meassurements that showed it to be
> twice as fast. I was too lazy too google it. But here goes. [...]

[Adrien Di Mascio]

> I've a made a quite basic test on these methods : [...]

Thanks to both! Have a good day!

--
François Pinard http://www.iro.umontreal.ca/~pinard

Fredrik Lundh

unread,
Jun 26, 2003, 11:02:53 AM6/26/03
to
Adrien Di Mascio wrote:

> I've a made a quite basic test on these methods :
>

> /snip/
>
> def write_thousands_join(self, char):
> """Writes a 100000 times 'char' in a list and joins it
> """
> str_list = []
> for index in range(100000):
> str_list.append(char)
>
> return ''.join(str_list)

no time to repeat your tests, but

def write_thousands_join(self, char):
"""Writes a 100000 times 'char' in a list and joins it
"""
str_list = []
append = str_list.append # bind method to local variable
for index in range(100000):
append(char)

return ''.join(str_list)

"should" be slightly faster.

also note that the results "may" differ somewhat if you append strings
of different lengths (mostly due to different overallocation strategies;
or maybe they're not different anymore; cannot remember...)

I always use join, but that's probably because that method is more likely
to run code that I once wrote. Never trust code written by a man who
uses defines to create his own C syntax ;-)

</F>


Chuck Spears

unread,
Jun 26, 2003, 1:51:43 PM6/26/03
to
I have a question with regards to the sprintf style of replacing
strings. I have a table in my HTML with a 50% width specifier. How
can i get the formatter to ignore it. For example in the example
below how can i code the 50% to be ignored?

keep the "%s 50% %s" % ("a","b")

Max M

unread,
Jun 26, 2003, 3:57:35 PM6/26/03
to Chuck Spears

Double the %

"%s 50%% %s" % ("a","b")


regards Max M

Reply all
Reply to author
Forward
0 new messages