Questioning the current vispy design paradigm

[volker....@inqbus.de]

Hi!

I worked a bit with pyqtgraph a very mature apllication code to be used as an example or skeleton of building very fast and stunning graphical scientific applications based on QT and Python.
Recently I builded a LIDAR data analysator apllication with pyqtgraph and it works well.

But to build it I had to have some minor changes:
* An additional menu entry in ViewBox to set the limits
* Displaying contour data with a fixed colour for above/below a the range of the colour bar.
* Rotating the HistoramLUTItem about 90 degrees.

To do this I had to override 6 Classes

HistogramLUTItem

AxisItem
GradientEditorItem
ViewBox
ImageItem
ViewBoxMenu

with more than 1000 lines of code copy, Notably ViewBox, ImageItem and ViewBoxMenu are sort of BaseClasses of the pyQTGraph "FrameWork"

I am sure that most of the sientists and other users out there live good with pyqtgraph. They will download it and fiddle as long as it works and are happy.
Looking at vispy I see the same (poor) design.

But there are better paradigms to bring code to the masses.

A real world example:
Last year we builded a python scientific application for a huge automotive company. Every prototype machine they have in the wild produces a several Mbyte data file per day and out comes a report of 20 pages with 4 diagrams on each page.
The whole process of selecting data, calculaing data, refining data, rendering of the data is done by a single python script per machine the engineers at our customer are programing, utilizing our Classes of Operators, Tables and Charts.
The reporting of one machine is done in 2 minutes,

We programmed this software twice. First we used the usual object oriented approach. And we failed. Not in terms of the performance.
We failed at the following:
* We builded some classes and based on them. The customer wants to replace these classes with some of its own.and since some other classes in use dependend on our base classes the customer draws more and more code into its own code base.
* The complexity of the problem at all was high. Every new usecase (Operator/Chart) introduces more "if/then/else" constructs in the base classes.
* With every addition to the code the base classes become fatter.

After three months of coding we throw away all of the code and reprogrammed in component design utilizing the ZCA (Zope Component Architecture).
Some weeks gone we finished the code and the customer was very happy.
The code overlap between our customers code and ours is under 5%.

A year after the customer called back. He currently uses several GByte of data processing a report of 1000+ pages and the RAM of its strongest server runs out.
Due to the component design the fix was easy we replaced the Memory/Caching component with a new component to respect the RAM boundaries and we were done.
The report redering is down to 20 Minutes, utilizing 5 GB of RAM. Without component design we would have had to fix all code derived from our base classes since
the aspect of accessing/caching data wouldhave been spread through the wohle code of the customer.

Sorry for the lenghty and authorative tone in the excurse but sometimes a real world example is helpful.

We like to support your projects with our experiance in component design. It's no witch-work and it helps so good!
You are aiming at the right target, and the missile can be improved - easily.

Best Regards

Volker










 

[Luke Campagnola]

This is interesting (I have definitely found subclassing to be an inconvenient mechanism for user customization). 

Can you give an example of something you would change in vispy, and how the API would look afterward?

--
You received this message because you are subscribed to the Google Groups "vispy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vispy-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vispy-dev/a506dd99-39d2-4a07-9d50-245ee256b2a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jérémy viau trudel]

Very interesting remarks Volker.

I think it worth saying that scientists are not programmers. Basic concepts like object oriented paradigm and even classes may be unknown in some community that make intensive use of superComputers. So you won't be surprise if I say to you that most of scientists do not know anything about ZCA or other framework for component based approach. But I do not want to criticize anyone, each community have its own history and its own goals..

Do you think that ZCA-like paradigm is radically different from OOP? It seems to me that ZCA only add (very useful and very efficient) mechanisms for class interoperability and reuse. Am I wrong? Would you suggest to rewrite vispy using ZCA?

It would be interesting to work on a real case to identify more specifically vispy's design problem. Lucas should work on a graph manager API next week (https://github.com/sh4wn/vispy/issues/5), maybe you could contribute to the design of this API and help us to make some changes toward component-based design paradigm?

[Cyrille Rossant]

I tend to agree that OO can be a dangerous paradigm when it's followed
blindly. I think the main problem is not the classes themselves but
inheritance. See for example
http://radar.oreilly.com/2015/06/the-art-of-design-patterns.html for
similar ideas ("Favor object composition over class inheritance").

> --
> You received this message because you are subscribed to the Google Groups
> "vispy-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vispy-dev+...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/vispy-dev/1551eb8a-086c-4c77-bcc1-07005af3818b%40googlegroups.com.

[volker....@inqbus.de]

Hi Jeremy!

Am Donnerstag, 18. Juni 2015 21:00:00 UTC+2 schrieb jérémy viau trudel:

I think it worth saying that scientists are not programmers. Basic concepts like object oriented paradigm and even classes may be unknown in some community that make intensive use of superComputers. So you won't be surprise if I say to you that most of scientists do not know anything about ZCA or other framework for component based approach. But I do not want to criticize anyone, each community have its own history and its own goals..

I have a PhD in Meteorology and can only agree. Most scientist believe that fortran is a cool new invention :-).

Do you think that ZCA-like paradigm is radically different from OOP? It seems to me that ZCA only add (very useful and very efficient) mechanisms for class interoperability and reuse. Am I wrong? Would you suggest to rewrite vispy using ZCA?

The ZCA is not radically different form the OOP - it is mainly an usefull extension to the OOP.
1) The ZCA introduces interfaces to the classes. This has some very positive advantages. First, if a class has e.g. an Interface that states that this Interface exposes the member function derive() then you can rely that this class exposes this function. Other way around if you like to build a replacement for this class you only have to implement the functions and properties that are required by the interface.
The implication of this is, that in the framework you are building with ZCA components the component compatibility does no longer is based on inheritance from a certaion base class but to implement and expose the interface. This is a very interesting feature if you thing of replacing a core python class by a wrapper around a C-library and you no longer have to do such ugliness as mixing classes to fullfill the required  inheritence of  a certain class. Second: The interface definition alone is a very good documentation of the functionality of the component.

2) The ZCA introduces the adapter pattern. This pattern is the best way I have found in 30 years of programming to keep complexity at bay. A simple example. We have some shape classes. Say Circle, Rectangle, and Star. All these classes are usually derived from a common base class SHAPE. Now we like the shape classes to have some functionality. For instance to print themselves an screen. So we put a method print() to the base and implement there or modify it in the derived classes. Now we like to have the functionality to send our shapes via FTP as a file to a data storage. Also we put this functionality in the base class. [This is what I see in lots of projects. Look at your base classes, see how huge they are. The Plone CMS has had a base class called "simple item" with serveral hundreds of functions. Now they have switched to ZCA and the code is much better.]
Back to our shape example. But why should a circle class bears the hundreds of lines of code to interact with an FTP-Server? The circle class is just a description of the a shape. A better approach is to put the functionality of the aspects (print on screen, push to ftp server) into a distinct class. So we build to classes Print and FTP which have the functionallity to print shapes and to send shapes to an FTP server. This one can do without the ZCA. It's a simple design pattern thing. But the ZCA brings a standartized way to make the connection of shapes and our Print/FTP classes easier Adapters.
And adapter is a component that can be plugged into one interface and exposes an other. So lets give our shape classes an Interface of say IShape. Our FTP-Class gets an Interface called IFTP where IFTP requires that the class implement a function called send_to_ftp(). After registering the classes in the ZCA componet registry we can write code like this:

a_circle = Circle()
IFTP(a_circle).send_to_ftp()

and our circle is on the FTP server. What happens here? THe INstance that IFTP() returns is a instance of the class FTP we have registered as an adapter from IShape to IFTP
Lets asume we need a different class to send Rectangle shapes to the FTP-Server. So we build a RectFTP class and register it as adapter from IRectangle to IFTP. Also we give the rectangle class the Interface IRectangle which is (since interfaces are classes) simply inherited from IShape.

a_rect = Rect()
IFTP(a_rect).send_to_ftp()

Now even if the cod estructure is identical IFTP(a_rect) returns a instance of RectFTP to deal with retangles and FTP.
For printing the shapes there will be other adapters that deal with printing. If an third functionality e.g. send per email is needed a new adapter for email operation is created.
The adapter pattern is very usefull to seperate different "aspects" that a class should perform from the class itself. some people think that this pattern also can be achieved by mixing classes, they are wrong. Mixing classes a considered harmful. At design time everything looks nicely, but at debugging time it is pure hell. At runtime you have instances wich expose all the functions of all ancestor classes. This is no better than to put every thing in the base classes.

3) Factories are the key to uncouple classes.
Everytime you write code like this:

class Foo(object):

[volker....@inqbus.de]

Sorry hit the wrong key and the message was sended before complete.

Am Freitag, 19. Juni 2015 00:33:16 UTC+2 schrieb volker....@inqbus.de:

3) Factories are the key to uncouple classes.
Everytime you write code like this:

 class Foo(object):
    def __init__(self)__:
        self.bar = Bar()

    def xxxx():
        another_bar = Bar()

you are glueing the two classes so thight that later nobody can build a drop-in replacement NewBAR for the class Bar, even if NewBAR is derived from Bar.
Due to such constructions in PyQtGraph I had so much code copies, recently.

Factories are the solution to that. A simple a BarFactory may no more than a function (usually it is a class).

def BarFactory():
  return Bar()

 class Foo(object):
    bar_factory = BarFactory

    def __init__(self)__:
        self.bar = self.bar_factory()

    def xxxx():
        another_bar = self.bar_factory()

Now we have two points where user code can change. The factory function can be overwritten by monkey patching - no good idea.
Better: The class can be derived and the class variabel bar_factory can be set to an other factory.

class NewFoo( Foo):
   bar_factory = NewBarfactory

This pattern produces a loose coupling between classes in a way weakref produces a loose coupling of instances.
This pattern is usually combined with the adapter pattern. In fact most adapters are not classes but factories that produce the instances. The advantage of adapter factories is that the factoriy can decide from the object to be adapted:
a) which class to use to produce the instance
b) how to custom initialize the produced instance
 
4) Model view controller paradigm. This is no ZCA topic but a more general one.
I noticed while reading the PyQTGraph code lots of classes that have several or all of these three aspects. It may be better to split such classes into distinct classes that are loosely coupled.

5) Handling of mass data. My experiance with mass data is that at a certain point you ran out of ram and have to swap data out of the interpreter.
It is sensible to implement such an usecase from the beginning. Since the garbage collector of the interpreter is picky its a very good idea never to store any reference to a numpy data array directly.
The solution for this problem is to wrap numpy arrays into containers.

class Container(object):

    def __init__( self, data ):
         self._data = data

@property
def data(self):
    # Check if data was deleted due to the caching policy
    if self._data == None:
        logger.info( 'Reload table: %s' % self._params.url )
        group_path, object_name, file_name = split_hdf_url(self._params.url)
        nav = navigators.get_navigator(file_name)
        self._data = retrieve_table(nav,  group_path + '/' + object_name)
    return self._data

The idea behind is to store references to the container instances and swap the numpy array from self._data onto disc and set self._data to None to force the garbage collector to free the memory.
All access to the numpy data is via the self.data property. This property checks for the existance of self._data and retrieves the numby data back from disc. We utilize HDF5 files as caches with very good performance. The only drawback may be lack of discipline of the coders. If one does:

self.my_numpy array = container.data

in a long living instace then the whole swapping mechanism is stuck. So the key is to hold direct reference to numpy data only in function local variables.
Also there has to be a central storage facility (I call it the data manager DM) that knows all the containers. The DM is a simple FIFO buffer that stores references to the containers and a bit of metadata.
If a new container is to be stored the DM checks if there is enough RAM to hold the data. If so the container is simply stored. If the RAM is low the DM looks up the oldes element (First one in the Buffer)
and swaps the data to disk and forces the interpreter to scrub it from RAM. If a swapped out Container data is accessed and its data if read in again from disc. It is moved to the end of the FIFO buffer to ensure that the container has a long time to be be swapped out again. More sophisticated caching/swappign schemes may be implemented but the effect is so tremendous that I never had to do.

6) Using the ZCA

The usage of the ZCA is simple, just use 
'zope.interface'
'zope.component'
'zope.proxy'

These packages have no external dependencies.

>It would be interesting to work on a real case to identify more specifically vispy's design problem.

The best to say of the ZCA is that it can be introduced partially. Most frameworks require that you use their base or mixing classes. The ZCA has no such requirements. So one can start with introducing the ZCA a certain point in code and let it spread.

>Lucas should work on a graph manager API next week (https://github.com/sh4wn/vispy/issues/5), maybe >you could contribute to the design of this API and help us to make some changes toward >component-based design paradigm?

I is a pleasure!

Cheers

Volker

[volker....@inqbus.de]

Hi Luke!

Am Donnerstag, 18. Juni 2015 04:23:59 UTC+2 schrieb Luke Campagnola:

This is interesting (I have definitely found subclassing to be an inconvenient mechanism for user customization). 

Can you give an example of something you would change in vispy, and how the API would look afterward?

I am sure that we will have at least two groups that will use the API. At first users that will only use the components but not exchange ones.
The second group will dig deep and try to replace some core components e.g. having fancy axis that can split themselves instead of the standart axis.
It will be sensible to address these two groups with two distinct APIs (like e.g. matplotlib and other frameworks do).
The first group (lets call them users) should be able to use the componets without knowing anything about ZCA. [One of the biggest errors of the Zope community was to introduce the ZCA and propose it as the new way to program the  Zope Apllication Server. Many Zope users were not capable to do so and in the end Zope died.]
The second group (lets call them developers) lik to change existing components The best and cleanest way to achieve this task is to use the ZCA. The developers will change the componet registry replacing existing adapters with their new ones. The will program new components that consits out of our framework components and these components in the end may be pulled into the framework. And it is clear that  these components later on may be modified by a new generation of developers. So it may be a good thing to define some coding standards to encourage developers to produce high quality components. Such codign standards begin with strict usage of PEP8 and end by recomedations of literature to design patterns books.

I have not read much of the vispy code.

Here an fictive example how a core component like a colour bar that features an axis may be designed to get a new axis type without changing the component code.

class Axis(a base):
     implements(IAxis)
     ...

class ColorBar(other base):
    implements(IColorBar)

    def __init__(self):
        ... lot of code ...
         self.axis = IAxis(self)
        ..  lot of code ...

# registerin Axis class as adapter for IColorBar

from zope.component import getGlobalSiteManager
gsm = getGlobalSiteManager()
gsm.registerAdapter(Axis, (IColorBar,), IAxis)

Now the developer has two possibilites to change the component Axis to his new one NewAxis

1) Registering the NewAxis instead of Axis in the registry. But this has the drawback that also existing ColorBar components will be affected, this may or may not be desired.
2) Subclassing IColorBar and change the Interface of it.

class INewAxis(IAxis):
      pass

class NewAxis(a third base):
     implements(INewAxis)
     ...


class NewColorBar(other base):
    implements(INewColorBar)

# registring the new component NewAxis for the NewColorBar component

from zope.component import getGlobalSiteManager
gsm = getGlobalSiteManager()
gsm.registerAdapter(NewAxis, (INewColorBar,), IAxis)

This example is not very realistic but I choose it for its simplicity. More realistic would be the usecase if axis depend on data.
Let's think of a X/Y axis on a image component for a contour plot. The row and column values of the image must be mapped into the X/Y sampling space of the axis data
to render the tick labels. I builded such a DataAxis recently for the LIDAR analyzer. This case makes it much easier to introduce new axis since we glue the axis adapter no longer to the colorBar but to the data for the Axis.

class Axis(a base):
     implements(IAxis)
     ...
    def __init__(self, axis_data):
        self.axis_data = axis_data    

class AxisData( Container) :
    implements(IAxisData)

class ColorBar(other base):
    implements(IColorBar)

    def __init__(self, axis_data):
        ... lot of code ...
         self.axis = IAxis(axis_data)
        ..  lot of code ...

# registring the component Axis for the IAxisData component

from zope.component import getGlobalSiteManager
gsm = getGlobalSiteManager()
gsm.registerAdapter(Axis, (IAxisData,), IAxis)

Now we have a much better design pattern since a new axis mostly is introduced due to the nature of the data it represents.

If we have a different type of Data

class NewAxisData( Container) :
    implements(INewAxisData)

class NewAxis(a third base):
     implements(INewAxis)

# registring the component Axis for the IAxisData component

from zope.component import getGlobalSiteManager
gsm = getGlobalSiteManager()
gsm.registerAdapter(NewAxis, (INewAxisData,), IAxis)

and voila! We have replaced the axis of the ColourBar without even subclassing it.

Cheers

Volker

[Luke Campagnola]

The parts of pyqtgraph that users most frequently want to customize are (1) AxisItem--text and tick placement, (2) ViewBox--user input handling

In vispy we have already dealt with (2) by allowing different cameras to be attached to a ViewBox. I would be all in favor of doing the same for the axes--use a swappable component that determines the tick locations and text.

<volker....@inqbus.de> wrote:

3) Factories are the key to uncouple classes. 

Agreed (although I usually just let the class be its own factory). This has been on my list for pyqtgraph as well for a long time. Want to open an issue for this?

The usage of the ZCA is simple, just use 
'zope.interface'
'zope.component'
'zope.proxy'
These packages have no external dependencies.

I suspect zope's framework is overkill for us. I'm pretty sure we can solve these issues without adding new dependencies.

[ snip ]

# registring the component Axis for the IAxisData component

from zope.component import getGlobalSiteManager
gsm = getGlobalSiteManager()
gsm.registerAdapter(NewAxis, (INewAxisData,), IAxis)

and voila! We have replaced the axis of the ColourBar without even subclassing it.

This seems like a rather complicated way of writing 

    # Swap the axis of one colorbar

    colorbar.axis = my_axis

..or perhaps

 

    # Create new colorbar with my axis

    colorbar = ColorBar(axis=my_axis)

..or even

    # change the default axis class for ColorBar

    ColorBar.axis_class = MyAxisClass

It's not clear to me why you suggest using the ZCA for this?

 

5) Handling of mass data. My experiance with mass data is that at a certain point you ran out of ram and have to swap data out of the interpreter.
It is sensible to implement such an usecase from the beginning. Since the garbage collector of the interpreter is picky its a very good idea never to store any reference to a numpy data array directly.
The solution for this problem is to wrap numpy arrays into containers. 

Not sure that this applies in our case. Most of the time, we can upload data to the GPU and discard it on the CPU (this is not always done at present, but should be wherever possible (want to open an issue for this too?). It's definitely worth considering, though, for any visuals that require repeated access to the data on the CPU.

 

[Dr. Volker Jaenisch]

Hi Luke!

Am 19.06.2015 um 14:56 schrieb Luke Campagnola:

The parts of pyqtgraph that users most frequently want to customize are (1) AxisItem--text and tick placement, (2) ViewBox--user input handling

In vispy we have already dealt with (2) by allowing different cameras to be attached to a ViewBox. I would be all in favor of doing the same for the axes--use a swappable component that determines the tick locations and text.

That is from my point of view the right way to do it. The adapter patter is very good for such a thing.

Interludium:
I have not got the whole picture of vispy into my head. So please correct me if I am talking garbage. If I got it right, you are planning to render the axis and other graph and UI components in OpenGL and not only the drawing canvases?
If so a lot of classes have to be programmed to mimic a toolset, eventhandling, user interaction, etc. that has to be maintained beside the Visualisation and OpenGL code. I am asking myself if this is the right approach? May some sort oh hybride approach mixing QT and OpenGL canvases be a better option? Don't get me wrong I do not like to criticise the whole project - but this point looks critical to me. I am sure that I do not understand completly now.

<volker....@inqbus.de> wrote:

3) Factories are the key to uncouple classes. 

Agreed (although I usually just let the class be its own factory). This has been on my list for pyqtgraph as well for a long time. Want to open an issue for this?

The usage of the ZCA is simple, just use 
'zope.interface'
'zope.component'
'zope.proxy'
These packages have no external dependencies.

I suspect zope's framework is overkill for us. I'm pretty sure we can solve these issues without adding new dependencies.

The ZCA is just one way to achieve components. Sure one can programm the same style without a component framework. But why should one invent a new way if a good battle hardened and absolutly lightweight framework is already there. Just to make it clear: ZCA has nothing to do with the huge Zope application server code! It's only a minimal extension to the Python interpreter and a simply component registry which is in fact a couple of dicts . Also this code is stable for nearly a decade without any changes.

[ snip ]
# registring the component Axis for the IAxisData component

from zope.component import getGlobalSiteManager
gsm = getGlobalSiteManager()
gsm.registerAdapter(NewAxis, (INewAxisData,), IAxis)

and voila! We have replaced the axis of the ColourBar without even subclassing it.

This seems like a rather complicated way of writing

You get me wrong. I have noticed that kinds of workaround in your PyQTgraph code on some locations.
And it is not the same. For all the three workarounds you present here you have to inherited a custom class from the ColorBar class.
And why should one derive a new class if only a subcomponent of this class has to be changed its class?

    # Swap the axis of one colorbar

    colorbar.axis = my_axis

This is monkey patching. And you will agree that monkey patching is an option of last resort, but not a method. And it may have unintended side effects if like in your pyqtgraph code in the init constructor is lots of code running on the newly created subcomponents.

..or perhaps

 

    # Create new colorbar with my axis

    colorbar = ColorBar(axis=my_axis)

This is the way you do it in PyQTGraph. Yes, this method works and is not too bad in this particular case. It is IMHO the only way to solve this problem utilizing pure OOP. Will you introduce for every new subcomponent the ColorBar should expose to the user/developer a new parameter in the constructor? This is an API change for every newly exposed sub component . And you have to do this API change on every class that uses an axis.

But remember I choose this example because of its simplicity - therefore are other simple solutions availabe.
The current problem can be abstractly described by having a parent component with a subcomponent - and I like to bring in a replacement for the class of the subcomponent. But let us think on the code evolution in the future. There will be new and many subcomponets for the parent component and other parent components will use the subcomponents, also. Having a number of s subcomponents and a number p of parent components using them you wil end with O(s * k) APi changes. This is the complexity I am warnig of. In the beginning if s and k are small numbers (in our example they are both equal 1) every thing looks harmless. But the quadratic API explosion will kill you in the long run. And this calculation only holds for one parent componetn layer and one  subcomponent layer. No imagine that you will have sub sub sub component layers and then your change requirements may be going like O(n**4). Surely there may be other clever ways to circumvent this complexity problem for some special cases - but they are not a method but clever programming - and clever programming, in opposite to a paradigm, needs lots of maintenance.

My solution does not change the API, nor the constructor and no derived custom class is needed.
Also the method presented is apllicable for as much subcomponents one like to make exchangeable for a unlimited number of components using the new subcomponents without introducing code changes. Keeping the number of API changes at an order of O(s) for every new sub component entering the code.
But you are right - mother nature does not like to be betrayed - for every gain is a take. Yes the new paradigm looks a bit akward, firstly. But after starting programming with assembler the first time I used a compiler, or the first time I looked at OOP code I had the same questions: Why all this boilerplate? It has worked before why do I need this now? There must be a solution without this new stuff? I use ZCA not because it is fancy or some shiny new buzzword. I use it because I have had stuck every time before writing large codes without ZCA. I used other paradigms like mixing classes (Django programming) and I stuck. I refactured the mixing class code three times but it still was bad code. But there were other projects that run flawlessly and after some years i come to the conclusion that the difference between the bad going and the good running projects is the difference in the programming paradigms.

[The calculus framework I wrote for this automotive company at first was used by hand written code by their engeneers. Currently they use python code generators that write the code that is using our framork. Producing user code of 10000 of lines per report per vehicle. We had never expect that this may be even possible to work at all. The lesson to be learned is that you never can predict the future requierements for your code.

Also a lesson learned. While moving our calculus code from OOP to FCA (introducing adapters and writing new operator classes) I made a testrun and the calculus machine worked perfectly but in the end no PDF was printed but the code finished without error. I looked for the problem and found none in the code. Every thing was there the printing adapter components were there and all tests in the printing module were working. In the end I found the problem. At no place in the main code I had imported the printing python module. The printing python module contained the adapter registration code fo the printing adapters. Not importing the printing module caused the printing adapters not to been registered and the main code therefore had simply skipped the printing. This behavior was never intended nor explicitly programmed - it just followed from the ZCA paradigms. Before programming with ZCA I had often problems with circular import problems since there are every time to many cross dependencies between the modules. Now I had scratched the other side of this scale - not enough dependencies to had all the modules needed included automagically. Or to say it with other words: Due to the ZCA paradigm our code has become truely modular.]

..or even

    # change the default axis class for ColorBar

    ColorBar.axis_class = MyAxisClass

This is also monkey patching. This holds for a quickfix but not for code I would like to include into my framework.
What happens to the instances of ColorBar that already have been created, before you monkey patch the Class? Imagine a usecase where the user can interactivley add new axis to the ColorBar instances. On the old instances now new axis classes will be used. But to be fair such side effects can also occur with the adapter pattern. :-)

It's not clear to me why you suggest using the ZCA for this?

Maybe it is more clear now.

In the very large Plone CMS code every customization is done by adapters. The common way of writing user customisable code is:

Check if there is an adapter for the purpose registered.
* if not - do nothin or fall back to a hardcoded default code
* if yes - use the adapter

This is merely a single IF statement to extends existing code with some functionality utilizing an adapter.

 

5) Handling of mass data. My experiance with mass data is that at a certain point you ran out of ram and have to swap data out of the interpreter.
It is sensible to implement such an usecase from the beginning. Since the garbage collector of the interpreter is picky its a very good idea never to store any reference to a numpy data array directly.
The solution for this problem is to wrap numpy arrays into containers. 

Not sure that this applies in our case. Most of the time, we can upload data to the GPU and discard it on the CPU (this is not always done at present, but should be wherever possible (want to open an issue for this too?). It's definitely worth considering, though, for any visuals that require repeated access to the data on the CPU.

I have thought critically on this point too as I read lots of vispy code and papers concerning vispy last night. As long as the CPU-RAM is large enough and the GPU can hold all the data you are right - we need no Swapping/data management. But if you go in the direction of realtime or longtime data visualiziation (The nature of most the data I am processing) you will run out of CPU or GPU RAM at some point. With long runnign applications you addiionally have to deal with the python interpreter memory leaks due to memory fragementation that reduces the available CPU RAM with the time.

For the LIDAR-Code I am writing crurrently I have not included a datamanager from the start. But I used DataContainers that allow for implementing a datamanagement (Swapping/Caching) later on without too much trouble.
But without such an encapsulation of the numpy arrays from the start it will be very hard work to introduce a datamagement later on - especially if the users have written their code also with plain numpy arrays. You will never bring your users to refacture their whole code.
So it may be wise to consider this point carefully and maybe use an approach similiar to my LIDAR code. In fact a datamangement for CPU- and GPU-RAM will be needed for realtime processing datasets larger than the RAM. The bandwidth bottleneck between CPU and GPU will make this a challenge in any event. So my suggestion is not to write code that later on will inhibit the intoduction of a datamanagement. The implementation of such DataContainer classes is trivial. But it is a lot of time needed to find all places in code where they have to be introduced and used. So it's a good Idea to do it upfront and not later on.


Cheers

Volker

-- 
=========================================================
   inqbus Scientific Computing    Dr.  Volker Jaenisch
   Richard-Strauss-Straße 1       +49(08861) 690 474 0
   86956 Schongau-West            http://www.inqbus.de
=========================================================	    
	    

[Luke Campagnola]

On Fri, Jun 19, 2015 at 11:38 AM, Dr. Volker Jaenisch <volker....@inqbus.de> wrote:

Hi Luke!

Am 19.06.2015 um 14:56 schrieb Luke Campagnola:

The parts of pyqtgraph that users most frequently want to customize are (1) AxisItem--text and tick placement, (2) ViewBox--user input handling

In vispy we have already dealt with (2) by allowing different cameras to be attached to a ViewBox. I would be all in favor of doing the same for the axes--use a swappable component that determines the tick locations and text.

That is from my point of view the right way to do it. The adapter patter is very good for such a thing.

Interludium:
I have not got the whole picture of vispy into my head. So please correct me if I am talking garbage. If I got it right, you are planning to render the axis and other graph and UI components in OpenGL and not only the drawing canvases?
If so a lot of classes have to be programmed to mimic a toolset, eventhandling, user interaction, etc. that has to be maintained beside the Visualisation and OpenGL code. I am asking myself if this is the right approach? May some sort oh hybride approach mixing QT and OpenGL canvases be a better option? Don't get me wrong I do not like to criticise the whole project - but this point looks critical to me. I am sure that I do not understand completly now.

That's correct--everything is rendered in OpenGL, and we already have a complete UI event system that proxies events from various backends (Qt, GLFW, SDL, IPython, etc.). There are currently no plans to implement other GUI tools such as provided by Qt  (and for now, I hope we do not venture too far down that path). 

[ snip ]
# registring the component Axis for the IAxisData component

from zope.component import getGlobalSiteManager
gsm = getGlobalSiteManager()
gsm.registerAdapter(NewAxis, (INewAxisData,), IAxis)

and voila! We have replaced the axis of the ColourBar without even subclassing it.

This seems like a rather complicated way of writing

You get me wrong. I have noticed that kinds of workaround in your PyQTgraph code on some locations.
And it is not the same. For all the three workarounds you present here you have to inherited a custom class from the ColorBar class.
And why should one derive a new class if only a subcomponent of this class has to be changed its class?

I'm confused--none of my 3 examples involved making a subclass of ColorBar.

 

    # Swap the axis of one colorbar

    colorbar.axis = my_axis

This is monkey patching.

No; in this case `colorbar.axis` would be a property.

 

But remember I choose this example because of its simplicity - therefore are other simple solutions availabe.

Well let's pick one that's complex enough to warrant the addition of a new component framework. So far I do not see the benefits..

I imagine a PlotWidget that includes an image in a ViewBox with axes, plus a colorbar with its own axis. The image, colorbar, and axes can all be swapped out by property assignment. Likewise the ViewBox's camera, or the individual tick-generators for the axes, or the colormap used on the image and the colorbar. There is no combinatorial API explosion here, just a collection of objects that can be swapped in and out.

 

My solution does not change the API, nor the constructor and no derived custom class is needed.

In that case the ColorBar would have already created its default AxisItem, only to delete it later when the new axis arrives. For any swappable component, I think it makes sense for that component to be specifiable at initialization time (either by making a subclass or with __init__ args). 

 

..or even

    # change the default axis class for ColorBar

    ColorBar.axis_class = MyAxisClass

This is also monkey patching.

If the API docs say "this attribute may be reassigned to change the default axis class", then I don't see a problem with it. Of course the class would be written with this API in mind. (If you wanted to change _prior_ instances of the class, well, then that's a different problem)

[Dr. Volker Jaenisch]

Hi Luke!

* I am not your enemy!
* I am using your code. Without pypqgrapth I had never get the code for the LIDAR ready at time. Thank you very much!
* I have no intentions to debase the work you have done (for vispy).
* Sorry for using pyqtgraph code as a "bad" example in the recent discussions. After some probing: The current vispy code is very clean! So I was wrong at my first assumption to warn of bad design, reffering to you! Sorry for that.
* The idea bundling the kowlegde of so many skilled people to build a perfectly new and massiv performant plattform for visualisation stunned me. After a lot of strugles with your PyQTGraph code I projected these negative feelings onto vispy. This was unfair to the vispy project and not at least to you.

Sorry for that.

We germans say: The foe of the good is the better.
So let us improove the code together.

Please notice my comments below ..

Am 19.06.2015 um 18:35 schrieb Luke Campagnola:

On Fri, Jun 19, 2015 at 11:38 AM, Dr. Volker Jaenisch <volker....@inqbus.de> wrote:

That's correct--everything is rendered in OpenGL, and we already have a complete UI event system that proxies events from various backends (Qt, GLFW, SDL, IPython, etc.). There are currently no plans to implement other GUI tools such as provided by Qt  (and for now, I hope we do not venture too far down that path).

Very cool. Guys, you have done lots of important work here! *Impressed*

[ snip ]
# registring the component Axis for the IAxisData component

from zope.component import getGlobalSiteManager
gsm = getGlobalSiteManager()
gsm.registerAdapter(NewAxis, (INewAxisData,), IAxis)

and voila! We have replaced the axis of the ColourBar without even subclassing it.

This seems like a rather complicated way of writing

You get me wrong. I have noticed that kinds of workaround in your PyQTgraph code on some locations.
And it is not the same. For all the three workarounds you present here you have to inherited a custom class from the ColorBar class.
And why should one derive a new class if only a subcomponent of this class has to be changed its class?

I'm confused--none of my 3 examples involved making a subclass of ColorBar.

Sorry for the misunderstandig. Your code snippets had not much context so I may have stumbled over their ambiguity. Also is your coding style quite different in some places to mine and therefore we way live in two different ways of thinking.

 

    # Swap the axis of one colorbar

    colorbar.axis = my_axis

This is monkey patching.

No; in this case `colorbar.axis` would be a property.

Yes. And this is monkey patching.

"""

Monkey patching is used to:

• Replace methods/attributes/functions at runtime, e.g. to stub out a function during testing;

"""

https://en.wikipedia.org/wiki/Monkey_patch

 

But remember I choose this example because of its simplicity - therefore are other simple solutions availabe.

Well let's pick one that's complex enough to warrant the addition of a new component framework. So far I do not see the benefits..

I imagine a PlotWidget that includes an image in a ViewBox with axes, plus a colorbar with its own axis. The image, colorbar, and axes can all be swapped out by property assignment. Likewise the ViewBox's camera, or the individual tick-generators for the axes, or the colormap used on the image and the colorbar. There is no combinatorial API explosion here, just a collection of objects that can be swapped in and out.

Thats a very good playing ground. This example features four leves of interwoven components and every level has a different structural signature.
I will come up with a component based solution for this scenario in a couple of days.

 

My solution does not change the API, nor the constructor and no derived custom class is needed.

In that case the ColorBar would have already created its default AxisItem, only to delete it later when the new axis arrives. For any swappable component, I think it makes sense for that component to be specifiable at initialization time (either by making a subclass or with __init__ args). 

 

..or even

    # change the default axis class for ColorBar

    ColorBar.axis_class = MyAxisClass

This is also monkey patching.

If the API docs say "this attribute may be reassigned to change the default axis class", then I don't see a problem with it. Of course the class would be written with this API in mind. (If you wanted to change _prior_ instances of the class, well, then that's a different problem)

Yep. I agree changing prior instances is a problem I personally do not like to address if possible. :-)


Here starts the competiton.

What I like to address in my example code is the following:

There are some base components: ViewBox, Axis, TickGenerator, Image.
These are used to build up a new component ImagePlot.

The user should do something like:

image = Image(data)
plot = ImagePlot(image)

and is delivered a pretty 2d plot of the image. The plot features a bottom X and a left Y axis as well as the image content.

I have abstracted this into dummy code: At the moment the application code does simply:

class PlotImage( BaseComponent):
    """
    Makes a 2D plot for an image
    """
    implements(IPlotImage)

    def __init__(self, parent, image):
        super(PlotImage, self).__init__(parent)
        self.image = image
        self.x_axis = IAxis(self)(orientation='bottom',  start=0, end=8)
        self.x_axis = IAxis(self)(orientation='left',  start=-5, end=2)

        self.vb = IViewBox(self)

        self.vb.addItem( self.x_axis )
        self.vb.addItem( self.y_axis )
        self.vb.addItem( self.image )

    def render(self):
        return self.vb.render()

image = Image(None, None)
plot =  PlotImage( None, image)
print( plot.render())

Output:
axis: bottom:0 1 2 3 4 5 6 7
axis: left:-5 -4 -3 -2 -1 0 1 2
Image: Rendered image

As you can see I followed a bit your pyqtgraph approach from the user perspective. But this is not relevant for the underlying concept.

The usecases is to change this code as a developer:
a) The user likes to modify some parameters on the components. e.g. the start value of the X axis shall now be -5.
b) The Axis class of all axis used by PlotImage should be swapped to a custom Axis Class "NewAxis".
c) The TickGenerator for the Y-Axis should to be swapped to a new TickGenerator that is aware of the image type that goes into PlotImage. If the Image is of Type Q (the former not kown property image.type equals "q") the tickLabes should be prefixed by a "Q".

The over all requirement is :
No changes on the apllication code.
No changes on the components code.
Minimal code copies.

I would appreciate if you add some requirements to be fullfilled by my code. Only due to thinking of weard usecases software can be made good.

[Luke Campagnola]

On Sat, Jun 20, 2015 at 9:02 PM, Dr. Volker Jaenisch <volker....@inqbus.de> wrote:

Hi Luke!

* I am not your enemy!
* I am using your code. Without pypqgrapth I had never get the code for the LIDAR ready at time. Thank you very much!
* I have no intentions to debase the work you have done (for vispy).
* Sorry for using pyqtgraph code as a "bad" example in the recent discussions. After some probing: The current vispy code is very clean! So I was wrong at my first assumption to warn of bad design, reffering to you! Sorry for that.

I'm not upset--sorry if I came off that way. I am quite convinced that you are correct about the problems with using inheritance as a means of customization. Please DO use pyqtgraph (and vispy where appropriate) as examples of bad code; this is how we learn to make them better. 

The only thing I am not convinced about is whether the extra complexity of a system like ZCA is balanced by its benefits. I am not trying to fight you on this, I'm just asking you to convince me.

[volker....@inqbus.de]

Hi Luke!

Am Sonntag, 21. Juni 2015 04:40:56 UTC+2 schrieb Luke Campagnola:

I'm not upset--sorry if I came off that way. I am quite convinced that you are correct about the problems with using inheritance as a means of customization. Please DO use pyqtgraph (and vispy where appropriate) as examples of bad code; this is how we learn to make them better. 

The only thing I am not convinced about is whether the extra complexity of a system like ZCA is balanced by its benefits. I am not trying to fight you on this, I'm just asking you to convince me.

Sorry for the delay. I got myself a cold (ugly weather in germany. This summer is more like winter) and were not able to do much work. But I discussed some ideas for an architecture for the graphics components with my wife and my coworker. The problem is more complex than I thought at the beginning of our debate. I have some promissing  ideas and startet some prototipish code that I will discuss with my coworker, tomorrow. After we two agree that these ideas have true potential we will present the prototype here.

Here the concepts I have in mind:

There are some base components: ViewBox, Axis, TickGenerator, Image.
These are used to build up a new component ImagePlot.

The user should do something like:

image = Image(data)
plot = ImagePlot(image)

and is delivered a pretty 2d plot of the image. The plot features a bottom X and a left Y axis as well as the image content.

I have abstracted this into dummy code: At the moment the application code does simply:

class PlotImage( BaseComponent):
    """
    Makes a 2D plot for an image
    """
    implements(IPlotImage)

    def __init__(self, parent, image):
        super(PlotImage, self).__init__(parent)
        self.image = image
        x_axis = IAxis(self)(orientation='bottom',  start=0, end=8)
        y_axis = IAxis(self)(orientation='left',  start=-5, end=2)

        self.vb = IViewBox(self, axis_items={'bottom': x_axis, 'left': y_axis })
        self.vb.addItem( self.image )

    def render(self):
        return self.vb.render()

image = Image(None, None)
plot =  PlotImage( None, image)
print( plot.render())

Output:
axis: bottom:0 1 2 3 4 5 6 7
axis: left:-5 -4 -3 -2 -1 0 1 2
Image: Rendered image

[ hg clone https://hg.inqbus.de/volker/graph_components
to obtain the python packages.]

As you can see I followed a bit your pyqtgraph approach from the user perspective. But this is not relevant for the underlying concept.

Also this code is far off from a coding style I would propose for building reusable components. One aim of this code is that it is 
based on typicall OOP code constructs and therefore it is not easily extended. 
I will show how this code can be used (without changing a line of it) to fullfill additional usecases the programmer has never intended by founding this code on component architecture. In fact the code is already base on ZCA as you can see by the constructors prefixed with an "I" but from a user perspective it is the same to write self.x_axis = Axis() or self.x_axis = IAxis(self) - it's just a command to generate an Axis instance. 

*The usecases to change this code :*
a) The user likes to modify some parameters on the components. e.g. the start value of the X axis shall now be -7.
b) The Axis class of all axis used by PlotImage should be swapped to a custom Axis Class "NewAxis".
c) The TickGenerator for the Y-Axis should be swapped to a new TickGenerator that is aware of the image type that goes into PlotImage. If the Image exposes the interface IQImage the tickLabes should be prefixed by a "Q".

*The over all requirement is :*
No changes on the apllication code.
No changes on the components code.
If the former is not possible the changes to the code should be as non invasive as possible.

All these usecases can be applied using the ZCA (code examples will follow tomorrow). But I have learned already that this additional code that manipulates the classes via ZCA is not straight foreward nor good nor easy to maintain. 

You will surely mention at this point how easy it is to e.g. put axis_item as a parameter to the ImagePlot-Class's constructor (and modify some lines of code in it) to solve the usecases. But Imagine that in the ViewBox class the author has written code that gives the axis the tickgenerators by explicitly doing self.axis['bottom'].tick_labels = ITicksLabels(self). In this case you have to swap the ViewBox class as well. I have done lots of such uglyness to achieve class changes in the pyqtgraph code. 

So in the end both alternatives 
a) Using ZCA to have more points (In fact every I<something>() statement) to set the crank on to tweak the code 
nor 
b) inheritance with lot of copied code 
are making me happy.

So even if the ZCA brings the possibility to change code that is not intended to be changed this is no sole concept a framework should be founded on. Therefore it takes more than just the use of ZCA. Additionally the framework has to follow some coding conventions and some base classes are needed to make the use of the ZCA more comfortable. Also the code changes to make a subcomponent swappable later on has to be minimal invasive (at best search and replace suffices).

Building these conventions and base structures is the work to be done tomorrow. If me and my coworker succeed we come up with structures to build components that are 
* as easy to programm as with normal OOP (with slightly more boilerplate)
* powerful methods to manipulate the classes that are used through the whole framework. E.G. swap all Axis classes in several components at once or as specific as :In Component A -> Subcomponent AA -> Subsubcomponent AAA swap the use of Axis class to NewAxis class.

But currently we struggle with the stretch between being easy and being powerful at the same time. Both ends do not fit together.
Please stay tuned what tomorrow brings.

Cheers

Volker
 

[Luke Campagnola]

On Sat, Jun 20, 2015 at 9:02 PM, Dr. Volker Jaenisch <volker....@inqbus.de> wrote:

    # Swap the axis of one colorbar

    colorbar.axis = my_axis

This is monkey patching.

No; in this case `colorbar.axis` would be a property.

Yes. And this is monkey patching.

"""

Monkey patching is used to:

• Replace methods/attributes/functions at runtime, e.g. to stub out a function during testing;

"""

https://en.wikipedia.org/wiki/Monkey_patch

Please read up on properties: https://docs.python.org/2/library/functions.html#property
Assigning a value to a property is equivalent to calling a `set_property()` method. 

Here starts the competiton.

What I like to address in my example code is the following:

There are some base components: ViewBox, Axis, TickGenerator, Image.
These are used to build up a new component ImagePlot.

The user should do something like:

image = Image(data)
plot = ImagePlot(image)

and is delivered a pretty 2d plot of the image. The plot features a bottom X and a left Y axis as well as the image content.

[snip]

The usecases is to change this code as a developer:
a) The user likes to modify some parameters on the components. e.g. the start value of the X axis shall now be -5.

plot.view.camera.rect = (-5, ...)

 

b) The Axis class of all axis used by PlotImage should be swapped to a custom Axis Class "NewAxis".

plot = ImagePlot(image, axis_class=MyAxis)

OR

plot = ImagePlot(image, axes=[my_x_axis, my_y_axis])

OR

plot.x_axis = my_x_axis

plot.y_axis = my_y_axis

 

c) The TickGenerator for the Y-Axis should to be swapped to a new TickGenerator that is aware of the image type that goes into PlotImage. If the Image is of Type Q (the former not kown property image.type equals "q") the tickLabes should be prefixed by a "Q".

plot.y_axis.ticker = my_ticker

 

[Luke Campagnola]

On Mon, Jun 22, 2015 at 8:43 PM, <volker....@inqbus.de> wrote:

All these usecases can be applied using the ZCA (code examples will follow tomorrow). But I have learned already that this additional code that manipulates the classes via ZCA is not straight foreward nor good nor easy to maintain. 

You will surely mention at this point how easy it is to e.g. put axis_item as a parameter to the ImagePlot-Class's constructor (and modify some lines of code in it) to solve the usecases. But Imagine that in the ViewBox class the author has written code that gives the axis the tickgenerators by explicitly doing self.axis['bottom'].tick_labels = ITicksLabels(self). In this case you have to swap the ViewBox class as well. I have done lots of such uglyness to achieve class changes in the pyqtgraph code. 

So in the end both alternatives 
a) Using ZCA to have more points (In fact every I<something>() statement) to set the crank on to tweak the code 
nor 
b) inheritance with lot of copied code 
are making me happy.

I think the point I am getting at is that it will take work on our part to ensure that all of the compound objects we are building can be torn apart and reassembled by the user. This is not so much a question of choosing the right architecture as it is a matter of just *doing* the work. Unless I am mistaken, ZCA does not magically enable components to be swapped in and out; you still need the code that handles the connection and disconnection of components. These use cases could be easily realized in pyqtgraph as well, but they take effort and nobody (to my knowledge) has put forth that effort.

[Dr. Volker Jaenisch]

Hi Luke!

Am 23.06.2015 um 05:44 schrieb Luke Campagnola:

On Sat, Jun 20, 2015 at 9:02 PM, Dr. Volker Jaenisch <volker....@inqbus.de> wrote:

    # Swap the axis of one colorbar

    colorbar.axis = my_axis

This is monkey patching.

No; in this case `colorbar.axis` would be a property.

Yes. And this is monkey patching.

"""

Monkey patching is used to:

• Replace methods/attributes/functions at runtime, e.g. to stub out a function during testing;

"""

https://en.wikipedia.org/wiki/Monkey_patch

Please read up on properties: https://docs.python.org/2/library/functions.html#property
Assigning a value to a property is equivalent to calling a `set_property()` method.

Yes. I the 15 years of programming with python I have come across this :-)
Properties are very useful. And they will be one building block for our proposed architecture. But properties are not the cure to the problems we are discussing - see below.

[snip]
The usecases is to change this code as a developer:
a) The user likes to modify some parameters on the components. e.g. the start value of the X axis shall now be -5.

plot.view.camera.rect = (-5, ...)

Yes, this is the easy one for starters. :-)

 

b) The Axis class of all axis used by PlotImage should be swapped to a custom Axis Class "NewAxis".

plot = ImagePlot(image, axis_class=MyAxis)

In our ZCA solution the swap of the new class has the same effect as this one you proposed here

   plot = ImagePlot(image, axis_class=MyAxis)
but without the need to introduce a parameter in the constructor for it. Also we will be capable to introduce this functionality afterwards to existing code without changing a line of code. This is the code we are working on currently.

OR

plot = ImagePlot(image, axes=[my_x_axis, my_y_axis])

OR

plot.x_axis = my_x_axis

plot.y_axis = my_y_axis

Yes. Yes. I understand the way you like it to do. But as I already stated. I am not primarily on how to write the code so that the axis can be swappen afterwards, My primary objective is to build an architecture that is ment to deal with existing code that is not written with this flexibility in mind. Despite this unflexibility the framework have to guarantee that it's possible to change the code with no invasion at all to work without flaws and to swap in NewClasses.

For your third proposed solution:

plot = PlotImage()  <- I assume

plot.x_axis = my_x_axis

plot.y_axis = my_y_axis

Your Idea solves not all problems and it introduces new ones.
* First you have to modify the ImagePlot signature by introdicing a new parameter. API changes are known to irritate the users..
* Second your properties e.g. self.x_axis will surely has some initialisation code to introduce the axis assigned to the component that hold the axis. That is good design. But,
* Third you have to fix the code in Plotimage that modifies the Axis. If the user constructs it's x_axis Axis classes, modifies it (think of attaching a local variable self.type="foo" to the axis instance) and gives it then to the self.x_axis property. The init code of the self.x_axis propery will firstly operate on the axis instance by the user and registers it for the component. Then the property  is overwritten afterwards by assigning the "external" my_x_axis  to the x_axis property at runtime. Surely the external axis will be introduced to the component correctly. But the prior initialisation done to the first axis that is overridden by the external axis is lost.
Now imagine that further down the code some functionality of plotImage depends on the additional axis local variable self.type and everything is broken.

For your third proposed solution:
PlotImage:
  def __init__()
     self.x_axis = Axis()
     self.x_axis.type = 'foo'

  def render()
     if self.x_axis.type == "bar": <- Bang!
        ...

Your code usage

plot_image = PlotImage()
plot_image.x_axis = NewAxis()

will break this really simple code example.

For your second solution:

plot = ImagePlot(image, axes=[my_x_axis, my_y_axis])

PlotImage(PlotBase):
  def __init__()
     self.x_axis = Axis()
     self.x_axis.type = 'foo'

The init has to be written anew since the axis parameter in the constructor has to be introduced somehow.
The most clean way may be to put this into a base class.

PlotBase():
  def __init__(axes=None)
     if axes:
         self.x_axis = axes[0]
         self.y_axis = axes[1]

Then PlotImage has to use the inherited functionality:

PlotImage(PlotBase):
  def __init__(axes)
     super(PlotImage).__init__(axes)    <- here
     self.x_axis = Axis()
     self.x_axis.type = 'foo'
     super(PlotImage).__init__(axes)    <- or better here

As you can see this depends strongly on the code already there. So you will have a chicken/egg problem, again.

Since with ZCA you swap the class of a component right at the time the instance is created there is no prior time for the programmer to change the instance. So none of the obstacles from above can occur.

 

c) The TickGenerator for the Y-Axis should to be swapped to a new TickGenerator that is aware of the image type that goes into PlotImage. If the Image is of Type Q (the former not kown property image.type equals "q") the tickLabes should be prefixed by a "Q".

plot.y_axis.ticker = my_ticker

Maybe you have not read my post from yesterday.

"""
But Imagine that in the ViewBox class the author has written code that gives the axis the tickgenerators by explicitly doing self.axis['bottom'].tick_labels = ITicksLabels(self). In this case you have to swap the ViewBox class as well. I have done lots of such uglyness to achieve class changes in the pyqtgraph code. """

Also plot.y_axis.ticker = my_ticker follows the same pattern as for the axis, above. You override the property ticker of the axis at runtime. Any code that works on the ticker prior to the time you overwrite the ticker with a new one is lost. This is no clean solution. Nobody can hinder a programmer to do whatever modification he likes to a component before you have to possibility to overwrite this component at runtime.

Since with ZCA you swap the class of a component right at the time the instance is created there is no prior time for the programmer to change the instance. So none of the obstacles from above can occur.

[Naveen Michaud-Agrawal]

I'm coming into this discussion a bit late, but this kind of design is now possible in python with abstract base classes (https://docs.python.org/2/library/abc.html), which was a lightweight implementation of Zope's component architecture. So it should be possible to use this type of architecture without any new dependencies (from py2.7 onward).

[Almar Klein]

We're aiming to be compatible with Python 2.6, though. Although I
suppose we'd need to let it go and only support 2.7+ at some point ...
(but probably not just jet)

> logger.info <http://logger.info>('Reload table: %s'%self._params.url )

> 6)Using the ZCA

>
>
> The usage of the ZCA is simple, just use
> 'zope.interface'
> 'zope.component'
> 'zope.proxy'
>
> These packages have no external dependencies.
>
> >It would be interesting to work on a real case to identify more
> specifically vispy's design problem.
> The best to say of the ZCA is that it can be introduced partially.
> Most frameworks require that you use their base or mixing classes.
> The ZCA has no such requirements. So one can start with introducing
> the ZCA a certain point in code and let it spread.
>
> >Lucas should work on a graph manager API next week
> (https://github.com/sh4wn/vispy/issues/5

> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fsh4wn%2Fvispy%2Fissues%2F5&sa=D&sntz=1&usg=AFQjCNEDc8enJne1Mwc0Z8awwjOwr4TuKA>),

> maybe >you could contribute to the design of this API and help us to
> make some changes toward >component-based design paradigm?
> I is a pleasure!
>
> Cheers
>
> Volker
>
>
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "vispy-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to vispy-dev+...@googlegroups.com

> <mailto:vispy-dev+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/vispy-dev/66a77b7f-4189-431e-9dcd-424e694f8d25%40googlegroups.com
> <https://groups.google.com/d/msgid/vispy-dev/66a77b7f-4189-431e-9dcd-424e694f8d25%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Dr. Volker Jaenisch]

Hi Naveen!

Am 10.08.2015 um 16:33 schrieb Naveen Michaud-Agrawal:
> I'm coming into this discussion a bit late, but this kind of design is
> now possible in python with abstract base classes
> (https://docs.python.org/2/library/abc.html), which was a lightweight
> implementation of Zope's component architecture. So it should be
> possible to use this type of architecture without any new dependencies
> (from py2.7 onward).

One of the main goals of the ZCA is the use of Interfaces instead of
classes to identify a component. There is no replacement to Interfaces
in ABC and therefore ABC is something like OOP+ but not a true component
architecture like CORBA, JAVA2EE or the ZCA.

Since ABC lacks the notion of interfaces the natural bloat of base
classes will occur like in normal OOP. From general experiance in
component design only the adapter pattern is a cure for intrinsic OOP
base class bloat problem. The adapter pattern does can be applied in the
normal OOP paradign but only to limited extend without the power of
interfaces. Although it is possible to develop a adapter machinrery with
pure OOP (or ABC) the question arises if it makes sense to invent the
wheel again.

To state it again the dependencies the ZCA introduces are absolute
minimal. The ZCA is available on every major plattform and is included
in PyPi at least since 2004. The ZCA drives Zope and Plone two very
mature projects with a huge codebase and lots of developers working on it.

Usually I have more problems to make my customers accept a MySQL
database instance in the project than to use ZCA. :-)

I think we should focus on the questions Luke came up with, recently.

Best regards,

[Bill Janssen]

Interesting discussion.  I did the first CORBA Python bindings, and was one of the stronger proponents of abstract base classes in Python 3.  But I'm not sure I understand your argument here.  The "abc" mechanism can be used to define interfaces, and classes can be defined to use those interfaces (via a mixin).  You can test to see whether a class provides a specific interface using isinstance().  What functionality do you think is missing?

Mixins are a criminally underused part of Python.

Bill