Serving Buildbot webpages via Pyramid ?
57 views
Skip to first unread message
Laurent DAVERIO
Jul 19, 2012, 3:35:32 PM7/19/12
to pylons-...@googlegroups.com
Hello list,
I'm currently working on a Pyramid-based website which, among other things, federates under a common user interface (Bootstrap-based), information coming from Sphinx, Doxygen and Buildbot.
- Sphinx is a document generator written in Python. The documentation is generated as a collection of static files. At first, I considered creating an ad hoc Sphinx (Jinja2) template to facilitate the integration of HTML fragments into a Pyramid .mako template. But I found it simpler to use the "pickle" export offered by Sphinx. The doc pages are pickled as dictionaries, which can be used directly from my Mako template. Problem solved
- Doxygen can generate the API documentation of a C Program as a collection of static (HTML, ...) files. So, the situation is similar to Sphinx, but HTML generation is not so customizable. For the moment, I'm using a libxml2 parser via the lxml module to extract, on the fly, the content of the <body> block, and inject it into a Mako template. Not very efficient, but satisfactory.
- Now, about Buildbot: it's a continuous integration framework written in Python. It uses Twisted.Web to display status information as web pages. It also offers a basic JSON API. I see three ways I could integrate Buildbot to my app :
1/ Use Pyramid as a web proxy, to get and process the Buildbot web pages on the fly.
2/ Use the Buildbot JSON API inside Pyramid views, and present the results via a Mako template.
3/ Take advantage from the fact that Buildbot is a python module, and thus can expose its functionalities as a Python API. But the API is not so well documented (use the Source, Luke)...
Choices 2/ and 3/ seem saner to me. 3/ is probably the cleaner, but 2/ seems easier. What would you choose?
Many thanks for your suggestions!
Laurent DAVERIO.
I'm currently working on a Pyramid-based website which, among other things, federates under a common user interface (Bootstrap-based), information coming from Sphinx, Doxygen and Buildbot.
- Sphinx is a document generator written in Python. The documentation is generated as a collection of static files. At first, I considered creating an ad hoc Sphinx (Jinja2) template to facilitate the integration of HTML fragments into a Pyramid .mako template. But I found it simpler to use the "pickle" export offered by Sphinx. The doc pages are pickled as dictionaries, which can be used directly from my Mako template. Problem solved
- Doxygen can generate the API documentation of a C Program as a collection of static (HTML, ...) files. So, the situation is similar to Sphinx, but HTML generation is not so customizable. For the moment, I'm using a libxml2 parser via the lxml module to extract, on the fly, the content of the <body> block, and inject it into a Mako template. Not very efficient, but satisfactory.
- Now, about Buildbot: it's a continuous integration framework written in Python. It uses Twisted.Web to display status information as web pages. It also offers a basic JSON API. I see three ways I could integrate Buildbot to my app :
1/ Use Pyramid as a web proxy, to get and process the Buildbot web pages on the fly.
2/ Use the Buildbot JSON API inside Pyramid views, and present the results via a Mako template.
3/ Take advantage from the fact that Buildbot is a python module, and thus can expose its functionalities as a Python API. But the API is not so well documented (use the Source, Luke)...
Choices 2/ and 3/ seem saner to me. 3/ is probably the cleaner, but 2/ seems easier. What would you choose?
Many thanks for your suggestions!
Laurent DAVERIO.
Laurent DAVERIO
Jul 19, 2012, 4:43:22 PM7/19/12
to pylons-...@googlegroups.com
> 3/ Take advantage from the fact that Buildbot is a python module, and
> thus can expose its functionalities as a Python API. But the API is not
> so well documented (use the Source, Luke)...
(Sorry, answering to myself) In fact, the Buidbot API *is* documented (http://buildbot.net/buildbot/docs/current/reference/), but not it is referenced from the Buildbot manual section, which is what you would expect. Instead, the link is found on the home page of the Buildbot website...> thus can expose its functionalities as a Python API. But the API is not
> so well documented (use the Source, Luke)...
Laurent.
Jonathan Vanasco
Jul 19, 2012, 5:10:44 PM7/19/12
to pylons-discuss
4/ set up a solr-lucene instance , have a cronjob ( or a triggered
task ) pull all the information from the various components and
standardize into a common document. use pyramid to front requests via
pysolr , and then display.
task ) pull all the information from the various components and
standardize into a common document. use pyramid to front requests via
pysolr , and then display.
Laurent DAVERIO
Jul 19, 2012, 6:21:19 PM7/19/12
to pylons-...@googlegroups.com
Le 19/07/12 23:10, Jonathan Vanasco a écrit :He he, I hadn't thought of that one! But I'm afraid I'm not great friends with Solr :-( Last time I tried to replace Xapian with Solr (because I needed to avoid indexing conflicts), I ended up keeping Xapian and implementing a Celery-based indexing tasklist...
Laurent.
Laurent.
Jonathan Vanasco
Jul 20, 2012, 10:51:47 AM7/20/12
to pylons-discuss
it's pretty easy, but how about saving it into a sqlite or bdb
filestore ?
you *could* do a lot of proxying and re-writing , but it seems like
what you want to accomplish is standardizing the interface to all this
documentation. periodically pulling everything out as json/whaetever
and then storing would be the easiest ( imho ) way to handle this.
you'd be able use a snapshot of the database to build all your mako/
etc templates against, and the last-valid version of each services
docs would always be available ( instead of seeing docs go down if one
of the services goes offline )
filestore ?
you *could* do a lot of proxying and re-writing , but it seems like
what you want to accomplish is standardizing the interface to all this
documentation. periodically pulling everything out as json/whaetever
and then storing would be the easiest ( imho ) way to handle this.
you'd be able use a snapshot of the database to build all your mako/
etc templates against, and the last-valid version of each services
docs would always be available ( instead of seeing docs go down if one
of the services goes offline )
Reply all
Reply to author
Forward
0 new messages