Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
pylons-discuss
Home
+ new post
About this group
Join this group
byte vs string issues with pyramid_viewgroup on python3
Options
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
   6 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Andreas Reuleaux  
View profile  
 More options Jul 18 2012, 3:51 pm
From: Andreas Reuleaux <reule...@web.de>
Date: Wed, 18 Jul 2012 21:51:41 +0200
Local: Wed, Jul 18 2012 3:51 pm
Subject: byte vs string issues with pyramid_viewgroup on python3
Print | Individual message | Show original | Report this message | Find messages by this author
I have ported a little pyramid app of mine to python3, almost
successfully, i. e. after some tweaking (using waitress etc,
won't go in the details here) it is already running.

But when it comes to rendering some pages, I get error msgs
concering byte vs string, see below. Those pages are assembled with
pyramid_viewgroup, which did install fine also for python3.

my app running on python2 had no particular problems (but see below: "what makes
things worse") when it came to rendering something like

  <div tal:replace="structure provider('content')" />

omitting my configuration details here (but can provide them of course,
if needed), as I think they are fairly in line with the pyramid_viewgroup
docs.

Now, when running this same app on python3 I get:

  ...
  File "/tmp/tmpdt09n6/page_936675d2dcac12f54a19c37066990109e30ddb34.py", line 341, in render
    __cache_58307216 = getitem('provider')('content')
  File "/home/reuleaux/eggs/pyramid_viewgroup-0.5-py3.2.egg/pyramid_viewgroup/__in it__.py", line 37, in __call__
    return render_view(self.context, self.request, name, secure)
  File "/home/reuleaux/work/website/py3/pyramid/pyramid/view.py", line 139, in render_view
    return ''.join(iterable)
TypeError: sequence item 0: expected str instance, bytes found

 - Expression: "provider('content')"
 - Filename:   ... /reuleaux/work/website/py3/src/website/templates/page.pt
 - Location:   (85:29)
 - Source:     ... tal:replace="structure provider('content')" />
                                          ^^^^^^^^^^^^^^^^^^^
 - Arguments:  repeat: {...} (0)
               renderer_name: templates/page.pt
               req: <Request - at 0x3774f50>
               Markup: <function Markup at 0x371a2f8>
               request: <Request - at 0x3774f50>
               provider: <Provider - at 0x37870d0>
               renderer_info: <RendererHelper - at 0x3787110>
               context: <NotFound - at 0x3787150>
               escape: False
               view: <NoneType - at 0x84bf40>

Any idea? I guess/hope that pyramid_viewgroup is just not all that
popular, and noone has actually tried running it on python3.

What makes things worse, is that my simple case

  <div tal:replace="structure provider('content')" />

worked even on python 2 only for pages that had no umlauts in them,
but I am german, living in France and we have lots of umlauts here
in Europe, so my workaround in python 2 was to
use

  <div tal:replace="structure Markup(provider('content'))" />

instead, with Markup() defined as

  def Markup(s):
    return s.decode('utf-8')

and provided for rendering with

  return render_to_response(
        'templates/page.pt',
        { 'provider': Provider(cntxt, req),
          'Markup': Markup,      
          },
        request=req,
        )

Now in python3 not even the simple pages without umlauts render any
more, no matter if I use this Markup() or not. But of course, any
page with or without umlauts (templates written in utf-8) should render.

Any idea?

-Andreas


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Andreas Reuleaux  
View profile  
 More options Jul 20 2012, 8:41 pm
From: Andreas Reuleaux <reule...@web.de>
Date: Sat, 21 Jul 2012 02:41:24 +0200
Local: Fri, Jul 20 2012 8:41 pm
Subject: Re: byte vs string issues with pyramid_viewgroup on python3
Print | Individual message | Show original | Report this message | Find messages by this author
OK, not having gotten any feedback, I guess my problem description was
to vague.

I have therefore prepared a little sample program, that should be easy
to extract and run from this e-mail: when running it with python2 I have
no problems, when running it with python3 I get the aforementioned byte
vs. string problems (also shown below once more).

While I roughly know, what has changed between python2 and python3
concerning strings/bytes, I am kind of stuck in this situation: I don't
know what to do in this case and I would hope that someone will try out
this sample program and point me in the right direction.

To make the installation experience easier, I have avoided to use to
many different python/configuration files: the whole program lives in
one file sample.py, which make use of a couple of templates.

To try out the program, you should therefore extract from this e-mail

* the file sample.py

* as well as the templates page.pt, home.pt, other.pt, and notfound.pt
  (note that only page.pt is a complete html page, the others are
  just html snippets)

steps to install/run the program on python2

create the environment
$ virtualenv foo
$ cd foo
$ ./bin/easy_install pyramid
$ ./bin/easy_install pyramid_viewgroup

copy the python script and the page templates to your foo env
$ cp .../simple.py .
$ cp .../page.pt .
$ cp .../home.pt .
$ cp .../other.pt .
$ cp .../notfound.pt .

adjust the path to the templates in simple.py, like
path=os.path.dirname('/path/to/my/foo/')

start the server
$ ./bin/python simple.py -port 8080

and have a look at some pages
http://myserver:8080/home
http://myserver:8080/other
http://myserver:8080/whatever - not found!
http://myserver:8080/         - default view

(see the explanations below, what's going on under the hood.)

--
now do the same with python3

create the environment
$ virtualenv-3.2 bar3
$ cd bar3
$ ./bin/easy_install-3.2 pyramid
$ ./bin/easy_install-3.2 pyramid_viewgroup

copy the simple.py script and the page templates as above,
and adjust the path to the templates

start the server with
$ ./bin/python3 simple.py -port 8080

and have a look at some pages as above.

While http://myserver:8080/, the default view renders fine,
as soon as I look at some of the other pages: home, other, whatever (not found)
I get

File "/home/reuleaux/tmp/bar3/lib/python3.2/site-packages/pyramid-1.3.2-py3.2.eg g/pyramid/view.py", line 139, in render_view
    return ''.join(iterable)
TypeError: sequence item 0: expected str instance, bytes found

 - Expression: "provider('content')"
 - Filename:   None
 - Location:   (24:33)
 - Source:     ... l:replace="structure provider('content')">page content</div>
                                        ^^^^^^^^^^^^^^^^^^^
 - Arguments:  repeat: {...} (0)
               renderer_name: /home/reuleaux/tmp/bar3/page.pt
               req: <NoneType - at 0x84bf40>
               request: <NoneType - at 0x84bf40>
               provider: <Provider - at 0x28059d0>
               renderer_info: <RendererHelper - at 0x2805a10>
               context: <Home - at 0x2805a50>
               escape: False
               view: <NoneType - at 0x84bf40>
192.168.1.7 - - [21/Jul/2012 01:58:49] "GET /home HTTP/1.1" 500 59  

As mentioned before: I am lost in this case, any help is appreciated.

Some explanations of what's going on in the python2 case /
how it is supposed to work:

While viewlets/content providers are not everyone's preferred style,
I have used them here to assemble some pages from building blocks
e. g. my home page looks just like this:

  @implementer(IPage)
  class Home(object):
      content=HomeContent()
      title='Home'

I. e. it is a page (implements IPage), and its contents is just
the home content.

Another page, Other, is very similar, but has of course some other
content:

  @implementer(IPage)
  class Other(object):
      content=OtherContent()
      title='Other'

I have used traversal to find those objects, e. g.

  http://myserver:port/home

will render Home, .../other will render Other and so on

The views operate just on the interfaces IPage, IContent,
i. e. view_page can render any page, and will make use of
view_content to fill in / show the content provided for
this particular page.

Anyway, thanks for reading, and thanks in advance for trying out.

-Andreas

simple.py
--------------------------------------------------
from wsgiref.simple_server import make_server
from pyramid.config import Configurator
from pyramid.response import Response

from pyramid_viewgroup import Provider

from pyramid.renderers import render_to_response

import sys
import os

import argparse

opts=None

# path = os.path.dirname('/home/reuleaux/tmp/foo/')
path = os.path.dirname('/home/reuleaux/work/website/py3/')

from zope.interface import Interface, implementer

class IContent(Interface):
    pass

class IPage(Interface):
    pass

@implementer(IContent)
class HomeContent(object):
    msg="hi from home"
    tmplt=os.path.join(path, 'home.pt')

@implementer(IPage)
class Home(object):
    content=HomeContent()
    title='Home'

@implementer(IContent)
class OtherContent(object):
    msg="hi from other"
    tmplt=os.path.join(path, 'other.pt')

@implementer(IPage)
class Other(object):
    content=OtherContent()
    title='Other'

@implementer(IContent)
class NotFoundContent(object):
    tmplt=os.path.join(path, 'notfound.pt')
    msg="not found!"

@implementer(IPage)
class NotFound(object):
    content=NotFoundContent()
    title='not found'

app={
    'home': Home,
    'other': Other,
    }

@implementer(IPage)
class App(object):
    def __getitem__(self, name):
        return app.get(name, NotFound)()

class Root(object):
    def __init__(self, request):
        self.request = request

    def __getitem__(self, name=None):
        if name in ['static',
                    ]:
            raise KeyError
        else:
            return App()[name]

def default_view(request):
    return Response('Hello world!')

def content_view(request):
    content=getattr(request.context, 'content', None)

    if hasattr(content, 'tmplt'):
        return render_to_response(content.tmplt,
                                  { },
                              )
    else:
        return Response('<span class="mark">content</span>')

def page_view(request):
    context=request.context
    provider=Provider(context, request)
    return render_to_response(
        os.path.join(path, 'page.pt'),
        {
            'provider': provider,
            'context': context,
        },
    )

def parser():

    parser=argparse.ArgumentParser()

    parser.add_argument("-port",
                        '--port',
                        dest="port",
                        action="store",
                        default=6543,
                        help="port")

    return parser

def run(args):

    p=parser()
    global opts
    opts = p.parse_args(args)
    # print(opts)

    config = Configurator(root_factory=Root)

    config.add_view('.default_view',)

    config.add_view('.page_view',
                    context='.IPage',)

    config.add_view('.content_view',
                    name='content',
                    context='.IPage',)

    app = config.make_wsgi_app()
    server = make_server('0.0.0.0', int(opts.port), app)
    server.serve_forever()

if __name__ == '__main__':
   run(sys.argv[1:])
--------------------------------------------------

page.pt
--------------------------------------------------
<?xml version="1.0" encoding="utf-8"?>

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:tal="http://xml.zope.org/namespaces/tal"
      >
  <head>
    <title>${context.title}</title>

    <style type="text/css"
           media="screen">
      .mark  {
      font-weight: bold;
      font-style: italic;
      }
    </style>

  </head>
  <body>
     <p>any page will have this structure</p>

     <p>and here comes the content:</p>

     <div tal:replace="structure provider('content')">page content</div>

     <p>rest of the page...</p>

  </body>
</html>
--------------------------------------------------

home.pt
--------------------------------------------------
<div
    xmlns:tal="http://xml.zope.org/namespaces/tal"
    tal:omit-tag=""
    >

  <p>
    this is my <span class="mark">home page</span>...
  </p>  

</div>
--------------------------------------------------

other.pt
--------------------------------------------------
<div
    xmlns:tal="http://xml.zope.org/namespaces/tal"
    tal:omit-tag=""
    >

  <p>  
    this is my <span class="mark">other page</span>
  </p>

</div>
--------------------------------------------------

notfound.pt
--------------------------------------------------
<div
    xmlns:tal="http://xml.zope.org/namespaces/tal"
    tal:omit-tag=""
    >

  Sorry, page not found

</div>
--------------------------------------------------


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Andreas Reuleaux  
View profile  
 More options Jul 21 2012, 3:44 pm
From: Andreas Reuleaux <reule...@web.de>
Date: Sat, 21 Jul 2012 21:44:06 +0200
Local: Sat, Jul 21 2012 3:44 pm
Subject: Re: byte vs string issues with pyramid_viewgroup on python3
Print | Individual message | Show original | Report this message | Find messages by this author
OK, I have a solution: if I patch the function

  render_view(context, request, name='', secure=True)

in my .../pyramid-1.3.2-py3.2.egg/pyramid/view.py file
by replacing

  return ''.join(iterable)

at the end with the following if-then-else code

  if isinstance(iterable[0], str):
      return ''.join(iterable)
  else:
      return b''.join(iterable)

then everything works as expected, but I'd certainly not
want to do this forever / would very much appreciate, if
someone could have a closer look at the problem.

-Andreas

...

read more »


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Simon Yarde  
View profile  
 More options Jul 21 2012, 5:27 pm
From: Simon Yarde <simonya...@me.com>
Date: Sat, 21 Jul 2012 22:27:03 +0100
Local: Sat, Jul 21 2012 5:27 pm
Subject: Re: byte vs string issues with pyramid_viewgroup on python3
Print | Individual message | Show original | Report this message | Find messages by this author
You'd probably want to walk the iterable and cast all elements to string? Then do your join operation with the empty unicode string. Your code only checks the first item and assumes all other elements are also strings - perhaps your code is working because there is only one element in the iterable for your circumstances.

I'm not familiar enough with this part of the Pyramid library to say more than this.. but something must be casting to bytes somewhere otherwise all strings would be defaulting to Unicode under Python 3.

On 21 Jul 2012, at 20:44, Andreas Reuleaux <reule...@web.de> wrote:

...

read more »


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Andreas Reuleaux  
View profile  
 More options Jul 21 2012, 8:23 pm
From: Andreas Reuleaux <reule...@web.de>
Date: Sun, 22 Jul 2012 02:23:50 +0200
Local: Sat, Jul 21 2012 8:23 pm
Subject: Re: byte vs string issues with pyramid_viewgroup on python3
Print | Individual message | Show original | Report this message | Find messages by this author
Well, yes you are right, seems to work in my case, because all entries
in the iterable are bytes (but haven't even bothered to see how many
entries there are in my iterable).

But the real thing is: I don't want to mess at all with
patching the pyramid source, I just want to use its api properly,
and if it doesn't work,

* then this could either be a flaw in my
  program / usage of the pyramid api
  - and I'd hope that someone would point me
  in the right direction of how to do it better - maybe there is some
  magic to use when working with python3?

* or there could be a flaw in the pyramid / pyramid_viewlet
  source - I guess, that can only be answered by someone
  familiar with the source

Anyway, thanks for answering.

-Andreas

...

read more »


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Andreas Reuleaux  
View profile  
 More options Jul 22 2012, 1:46 pm
From: Andreas Reuleaux <reule...@web.de>
Date: Sun, 22 Jul 2012 19:46:37 +0200
Local: Sun, Jul 22 2012 1:46 pm
Subject: Re: byte vs string issues with pyramid_viewgroup on python3
Print | Individual message | Show original | Report this message | Find messages by this author
OK, I found a workaround/solution:

Instead of patching render_view() in
.../pyramid-1.3.2-py3.2.egg/pyramid/view.py I am overriding Providers'
__call__() method with my own my_render_view() function now like so:

  from pyramid.view import render_view_to_iterable

  # cf. pyramid-1.3.2-py3.2.egg/pyramid/view.py
  def my_render_view(context, request, name, secure):
      iterable = render_view_to_iterable(context, request, name, secure)
      if iterable is None:
          return None

      # rx
      # return ''.join(iterable)
      if isinstance(iterable[0], str):
          return ''.join(iterable)
      else:
          return b''.join(iterable)

  from pyramid_viewgroup import Provider

  class MyProvider(Provider):

      def __call__(self, name='', secure=True):
          return my_render_view(self.context, self.request, name, secure)

then I use this MyProvider (instead of just Provider formerly)
in page_view():

  def page_view(request):
      context=request.context
      # provider=Provider(context, request)
      provider=MyProvider(context, request)
      return render_to_response(
          os.path.join(path, 'page.pt'),
          {
              'provider': provider,
              'context': context,
          },
      )

While there still might be a better / easier solution
- and I would like to here about it - this workaround
at least allows my to use pyramid_viegroup in the way
I used to in python2.x

My guess is still, that noone else has bothered to
try pyramid_viewgroup on python3.

-Andreas

...

read more »


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »

Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
©2013 Google