Protected Methods and Python
sri...@yahoo.com
Is this a serious limitation for Object Oriented Programing with Python?
Or may be it is possible to emulated Protected Methods in Python.
I look forward to some discussion by OOP experts.
Regards,
Srijit
Mads Orbesen Troest
> Or may be it is possible to emulated Protected Methods in Python.
It can be "emulated" by prepending the method/attribute name with 2
underscores. This will make Python mangle the name so noone accidentally
can access it. Evil minds can still do whatever they want, but then again -
it's their own responsibility if they break things then. ;)
I also feel it's weird that Python doesn't have a truly enforced
encapsulation scheme, but one can live with it.
HTH,
/\/\\ads Orbesen Troest
Mads Orbesen Troest
>> Or may be it is possible to emulated Protected Methods in Python.
>
> It can be "emulated" by prepending the method/attribute name with 2
> underscores.
Whoops, I was too quick and didn't see that you wrote "protected" and not
"private".
At any rate: the mangling scheme can be used for "protected" methods, since
you know the mangled name is _classname__attribute/method. You can then be
one of the "evil" persons I talked about in the previous post, and access
the "private" member that way.
Alex Martelli
sri...@yahoo.com wrote:
> I do not see a concept of Protected Methods in Python 2.2 like C++ and
Right: you don't see it because it isn't there.
> Ruby. Is this a serious limitation for Object Oriented Programing with
Judge for yourself: the introduction of "protected" is one of the few
aspects of C++ which Stroustrup *regrets*, according to his excellent book
on the design and evolution of the C++ programming language.
> Python? Or may be it is possible to emulated Protected Methods in Python.
To quote St. Paul out of context, everything is permissible, but not
everything is opportune. And that is very much Python's philosophy:
rather than focusing on trying to make some things impossible (and
generally failing -- a simple cast in most implementations of C++ lets
you blast away any "protected" that a silly library designer may have
tried to impose on you;-), Python empowers and trusts the programmer.
At the recent ACCU (Association of C and C++ Users) conference in Oxford,
Greg Colvin gave a wonderful keynote speech titled "in the spirit of C".
The FIRST items on the list are to trust programmers, and not stand in
their way. I found it interesting that, while Greg doesn't even KNOW
Python (at least that's what he said more than once at the conference),
he was making a substantial part of the case for Python so well:-).
If you don't trust programmers' skills and good intentions, you should
not give them powerful tools at all -- then, C, C++, Java, Ruby, Python,
would ALL be totally inappropriate, as each of them is easily powerful
enough to let an unskilled or evil programmer make a horrid mess of
things. If you DO trust programmers, then "not standing in their way",
letting them get the job done, is very important.
In practice, what happens in languages such as C++ or Java is a very
interesting, almost schizophrenic split -- the programmer is "trusted"
when he's WRITING a library or framework, but NOT trusted when he's
USING that library or framework. The library designer must be so
omniscient as to KNOW all uses to which his code will be put, and in
particular what pieces of code, that he found useful and put in his
library, will NOT be useful to other programmers in slightly different
contexts. The library user is presumed to be such an idiot that a
simple INDICATION of "this code is intended for internal library use
only" would not suffice -- nay, the using programmer must be "coerced"
to keep his grubby hands away from said code. In practice, there is
no such split in the population of programmers -- authors and users
of libraries are drawn from the same pool. The sociology implicit in
the language design just isn't a good match for reality.
In Python, the amount of trust vested in you, programmer, is just
about the same whether you're using an existing library or coding
up a new one (most of the time, you're doing both -- using some
existing libraries AND also producing code that you or somebody
else will want to reuse in the future in slightly different ways).
You indicate "this is meant for internal use only" by following the
convention of starting a name with a single underscore; you respect
that convention in libraries you use, unless you've got deucedly
good reasons not to (and can live with the consequences, such as
your code not working any more with a newer release of the library).
The latter possibility is basically the same thing as putting
#define private public
#define protected public
at the top of your .cpp file, before any #include, in most C++
implementations -- except that instead of this one lump reminder
that you're breaking conventions, you get reminded of the fact
throughout your code, whenever you refer to a name starting with
an underscore.
As for the specific indication "this name may be used by
derived classes but not by other code", there's never really any
particularly good reason for that (whence Stroustrup's regrets) --
public and private have reasons for existing (public is about
a library's interface to the world, private about its current
implementation), protected doesn't (no real need for a library
to distinguish SEVERAL "interfaces to the world" depending on
where inheritance is being used -- inheritance is just a tool
like many others and it's wrong to put it upon some altar...).
Alex
Jan Dries
of the C++ language, considers them to be a design mistake, and now
regrets they were ever added to C++. See "The Design and evolution of
C++" by Bjarne Stroustrup, page 302.
Java and I assume Ruby as well simply copied the concept from C++.
Regards,
Jan
Jp Calderone
> I do not see a concept of Protected Methods in Python 2.2 like C++ and Ruby.
> Is this a serious limitation for Object Oriented Programing with Python?
Not in my experience.
> Or may be it is possible to emulated Protected Methods in Python.
If you have methods you don't want people calling, simply do one of the
folloowing:
1) Document them as such
2) Prefix their names with "_"
3) Don't document them at all
> I look forward to some discussion by OOP experts.
It may just be me, but I don't see how this has anything to do with object
oriented design/programming.
Jp
--
Somewhere, something incredible is waiting to be known.
-- Carl Sagan
--
up 24 days, 15:01, 1 user, load average: 0.99, 0.90, 0.85
Roy Smith
> #define private public
> #define protected public
> at the top of your .cpp file, before any #include, in most C++
> implementations
I'm confused. You started out to show that C++'s "protected" keyword is
evil, but what you've really shown is that C++'s preprocessor is evil.
Can they both be true? :-)
Alex Martelli
> Alex Martelli <al...@aleax.it> wrote:
>> #define private public
>> #define protected public
>> at the top of your .cpp file, before any #include, in most C++
>> implementations
>
> I'm confused. You started out to show that C++'s "protected" keyword is
> evil,
I did no such thing -- I just mentioned Stroustrup's regrets about it (and
my agreement with him on this point). It's more of a redundant complexity
than any substantial "evil".
> but what you've really shown is that C++'s preprocessor is evil.
Nope. If your preprocessor didn't implement #define, you could do
just the same trick, even more messily, with a copy and paste from
the relevant .h files and any text editor of your choice. The
preprocessor just makes it handier, avoiding the evils of copy and
paste coding for this, at least;-).
> Can they both be true? :-)
Possibly, but that's not what we're discussing, anyway.
Alex
Jp Calderone
Double underscore prefix is for avoiding name collisions in inheritance,
not preventing people from touching things you consider private to the
implementation.
Language enforced private attributes are a Wrong idea because they
needlessly increase the difficulty of people trying to use code which
employs this feature.
There are a few possible scenarios:
1) I am a skillful coder and I am using a library written by a skillful
coder. The library is bug-free (for the purpose which I use it) and I have
no need to touch the implementation's internal variables. In this scenario,
runtime protected private attributes are not useful.
2) I am not a skillful coder and I am using a library written by a
skullful coder. The library is bug-free (for the purpose which I use it)
but I feel the need to touch the implementation's internal variables anyway.
Whether or not they are protected by the runtime environment, I will
probably find a way to screw up my application, because I am not skilled.
In this scenario, runtime protected private attributes are not useful.
3) I am a skillful coder and I am using a library written by someone who
is not skillful. The library contains bugs which disrupt my application.
In this scenario, runtime protected private attributes make it impossible
for me to work around bugs in the library, and so are worse than useless.
Since Python provides no real runtime protection of private attributes,
the best you can hope to do is make it excessively painful for people using
your code to work around your bugs. I can't imagine why anyone would want
to do this.
If an attribute is private to your implementation, document it this way
and prefix it with a single underscore. Nothing else is necessary.
high-and-mightily y'rs,
Jp
--
"There is no reason for any individual to have a computer in their
home."
-- Ken Olson, President of DEC, World Future Society
Convention, 1977
--
up 24 days, 19:01, 7 users, load average: 1.07, 1.20, 1.45
Dirk Gerrits
> Don't worry too much about protected methods. Bjarne Stroustrup, creator
> of the C++ language, considers them to be a design mistake, and now
> regrets they were ever added to C++. See "The Design and evolution of
> C++" by Bjarne Stroustrup, page 302.
> Java and I assume Ruby as well simply copied the concept from C++.
I don't have the book with me, but are you sure it wasn't protected DATA
members that Bjarne regretted?
Regards,
Dirk Gerrits
James M. Stern
wrote:
It was. From _The Design and Evolution of C++_, Section 13.9 (Protected
Members):
"Fortunately, you don't have to use protected data in C++; 'private'
is the default in classes and is usually the better choice. Note that
none of these objections are significant for protected member
*functions*. I still consider 'protected' a fine way of specifying
operations for use in derived classes.
That paragraph is near the end of the section, and "these objections"
refers to arguments just made against protected data.
Admittedly, the next paragraph says that 'protected' "overcame my better
judgment [...]" but I think he means (data) members there.
... As an aside, I view every Python method I write as intended either
for the world, for this class and derived classes only, or for this
class only. In short, as public, protected, or private.
Maybe it's my C++ heritage.
--
Opinions expressed above are not necessarily my employer's.
James M. Stern st...@itgssi.com (213) 270-7955
ITG Software Solutions, Inc.
Culver City, CA 90230
Venkatesh Prasad Ranganath
All said, how do you ensure "design by interface" in Python? If every feature of a class is open to the user then he/she
will use them as he/she sees fit. However, the writer of the class could have coded some methods for ancillary purposes and
later decides to do away with them in the next version of the class. How should the user deal with this situation? In my
opinion, protected and private as in C++ or Java empowers the library developer to expose just the interface that he/she
intends the user to see. Nothing more and nothing less. (It's an issue dealing with abstraction.) As a result, the user can
be more care-free while using the library code as he cannot screw up the internals if the library code doesn't do it on it's
own. On the other hand, if all members of a class are visible to the user then the user needs to be more diligent while using
the library. I prefer the first option.
On the issue of working around bugs in a library via runtime protected attributes, it would be better if the skillful coder
fixed the bug ;-) or use another library. This is just what I would do.
Well, that's my opinion.
> Since Python provides no real runtime protection of private attributes,
> the best you can hope to do is make it excessively painful for people using
> your code to work around your bugs. I can't imagine why anyone would want
> to do this.
>
> If an attribute is private to your implementation, document it this way
> and prefix it with a single underscore. Nothing else is necessary.
>
> high-and-mightily y'rs,
>
> Jp
>
--
Venkatesh Prasad Ranganath,
Dept. Computing and Information Science,
Kansas State University, US.
web: http://www.cis.ksu.edu/~rvprasad
Jp Calderone
> [snip]
>
> All said, how do you ensure "design by interface" in Python? If every
> feature of a class is open to the user then he/she will use them as he/she
> sees fit. However, the writer of the class could have coded some methods
> for ancillary purposes and later decides to do away with them in the next
> version of the class. How should the user deal with this situation? In my
> opinion, protected and private as in C++ or Java empowers the library
> developer to expose just the interface that he/she intends the user to see.
> Nothing more and nothing less. (It's an issue dealing with abstraction.)
> As a result, the user can be more care-free while using the library code as
> he cannot screw up the internals if the library code doesn't do it on it's
> own. On the other hand, if all members of a class are visible to the user
> then the user needs to be more diligent while using the library. I prefer
> the first option.
If it isn't documented, don't use it :) You have a point though. Some
libraries don't make it clear what is public and what is an implementation
detail. What's the worst case, though? You upgrade, your software breaks,
you downgrade, it works again, and maybe you fire off an unhappy email to
the dev list for the library.
>
> On the issue of working around bugs in a library via runtime protected
> attributes, it would be better if the skillful coder fixed the bug ;-) or
> use another library. This is just what I would do.
Good point. I omitted this possibility in hopes of keeping the point
clear, but it's what I try to do as well :)
Jp
--
up 27 days, 3:02, 3 users, load average: 1.45, 1.42, 1.43
Alex Martelli
...
> All said, how do you ensure "design by interface" in Python? If every
> feature of a class is open to the user then he/she
> will use them as he/she sees fit. However, the writer of the class could
> have coded some methods for ancillary purposes and
> later decides to do away with them in the next version of the class. How
> should the user deal with this situation?
Quite simple, really: a name that starts with a leading underscore
indicates that the name is there "for ancillary purposes" as you put
it -- it's not in the "published public interface" of the module,
class, etc; a name _without_ such a leading underscore is part of
the public interface, and won't be "done away with" in the next
version of the package, module, class, and so forth.
Thus, the user should deal with this situation by using only names
exposed in the public interface of the package, module, class, etc --
names that don't start with a leading underscore -- unless of course
said user *WANTS* to tie himself or herself to the specific version
of the reusable component.
> In my opinion, protected and
> private as in C++ or Java empowers the library developer to expose just
> the interface that he/she
> intends the user to see. Nothing more and nothing less. (It's an issue
> dealing with abstraction.)
Not quite true in C++, since casts allow the user to work around this
(in most implementations) -- just as Python allows working around
exactly the same attempts by the library designer to impose his or her
views of appropriate abstraction by using marked-as-internal names.
In Python and Java the user can also work around such limitations by
introspection, unless the user's code is treated as untrusted and
is run inside a sandbox that tries to limit the damage the user can
do (that was the design intent of standard library modules rexec and
Bastion in Python -- enforcing those restrictions that are normally
just respected by convention, by treating user code as untrusted and
potentially intended to be actively harmful -- they're currently broken,
but equivalent alternatives such as package Sandbox are being explored).
But issues of security are really very different from issues of
abstraction, IMHO, even though Java does its best to conflate them --
if you must think of user code as actively trying to do maximum damage
to the system, you've got a LOT of things to worry about. For normal
use, having the library designer consider the future library user as
a nasty adversary to be foiled at every turn is really an unproductive
mindset, even though it's the one that accessibility restrictions suggest.
> As a result, the user can be more care-free
> while using the library code as he cannot screw up the internals if the
> library code doesn't do it on it's
> own. On the other hand, if all members of a class are visible to the user
> then the user needs to be more diligent while using
> the library. I prefer the first option.
The user can rather easily "screw up" things if and when he or she
does it DELIBERATELY -- and in fact, even an accident is quite likely.
E.g., consider the hypersimplified code where the library has
something like:
class Base {
private: virtual int another() {
return 100;
}
public: virtual int amethod() {
return 23+another();
};
};
Method another is virtual because class Base is actually derived from
others and that method is overridden inside the library here and there,
etc, etc. But it's private, so the user can't tamper accidentally
with it, right? Yeah, right...:-). So, way over there in userland,
the user, not even "seeing" (you do keep saying "visible" while what's
supposed to happen has to do with "accessible" instead...) the private
parts of the class Base it gets from the library, happily codes:
class Derived: public Base {
public: virtual int another() { return 1000; }
};
int xx(Base* d)
{
return d->amethod();
}
int main()
{
Derived dd;
std::cout << xx(&dd) << '\n';
return 0;
}
guess what this prints...?
In Python, you can choose between calling the meant-to-be-internal
method _amethod (the user then must KNOW not to use it) or avoiding
accidental clashes by calling it __amethod (so it gets name-mangled
by the Python compiler to help avoid _accidental_ clashes). Not
foolproof, yet (homonymous classes in different modules are still
*possible*, unless the designer takes care about that...), but quite
a bit better than what pure *accessibility* restrictions achieve in C++
in cases such as this.
> On the issue of working around bugs in a library via runtime protected
> attributes, it would be better if the skillful coder
> fixed the bug ;-) or use another library. This is just what I would do.
Modifying source code inside a component that is maintained by
another party -- basically "forking" that component -- even where
_possible_ at all because you do get source, is a grievous mistake
(correctly identified as such in the important book "Antipatterns").
By making your own fixes (forking) the component, you make your
situation *UNTENABLE* when new releases of that component come out --
you're contemned to keep maintaining your fixes forever, porting
them to each successive release, rebuilding from source every time,
distributing the .DLL for YOUR fixed component (when you can even
have the LICENSE to do so...) rather than relying on system-provided
versions...
The option to avoid buggy libraries isn't always there - sometimes
one IS condemned to e.g. develop on top of Microsoft MFC, because
that's part of the specs for your component. Then, you start cursing
a language which has let the designer of those libraries impede you
from access to a zillion things you desperately DO need to access
(or, you start doing heavy-duty casting, but there are still some
limitations that will stump you quite badly, such as the use of
concrete classes rather than factory-idioms where you NEED to be
able to interpose your own subclass of some widget and the deuced
framework insists on generating the concrete baseclass instead...).
Alex