Paul Rubin <http://phr...@NOSPAM.invalid> writes: > Mike Meyer <m...@mired.org> writes: >> > What convention? It just makes it possible to write code with some >> > specific invariants if there's a need to do so. >> That you don't pass private variables to a function unless it's a builtin. > No, I don't see that as "convention", it's just something that you do > if you want to make sure that all the references are local or > builtins.
Semantics...
> It's like saying that if you if you want to be sure that x > is always positive, then don't execute statements like "x = -3". I > wouldn't call something like that a "convention".
Right - that's not a convention. That behavior would be a convention if you did somehing like prefixing the variable name with a "p" to indicate that you don't do that.
> maybe I'd like to be sure that x never yields negative numbers. > CPython happens to have some features that stop me from guaranteeing > that invariant.
I'd say CPython was missing the features that you need to guarantee that. Missing quite a *lot* of features, in fact. But Python has never been about keeping people from writing bad code - it's about helping people write good code.
>> It wouldn't catch overriding things in __builtins__ or >> overriding builtins in a module, or things poking at the variable >> through __dict__, or - well, there are probably lots of things that >> need to be dealt with. > Yes, but I see all of those as implementation artifacts, not anything > fundamental.
Right - those aren't fundamental. It's things like Python allowing programmers to use whatever style is appropriate to the problem at hand, rather than insisting on an OO style. Once you get outside the OO style, "private" attributes becomes problematic.
>> Preventing accidents is all "private" does - without fundamental >> changes to the implementation of the language. You have to catch every >> mechanism that can be used to find a reference to an attribute, like >> references to __dict__ and to the class variable.
> The Python stdlib for a long time had a module called Bastion that > attempted to do exactly that, so you can't say that the desire is > un-Pythonic.
Of course I can say it's unpythonic. I might even be right: just because the standard library does something doesn't mean that something is automatically pythonic. But I haven't said that "private" is unpythonic. I will say that the things you need to program effectively with private variables - like having to inherit all your utility functions - is unpythonic.
> Bastion was only removed because implementation issues > kept it from working properly. Those issues probably can't bbe > resolved in the 2.x series but as the language evolves, maybe > something can be done to bring it back.
Pretty much every attempt to restrict what other programmers do in Python has failed - for "implementation issues". I think that's a good sign that this kind of thing isn't going to work without some serious work on the interpreter.
<mike -- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Paul Rubin <http://phr...@NOSPAM.invalid> writes: > Mike Meyer <m...@mired.org> writes: >> >> Compile-time restrictions don't matter for squat - you need >> >> serious restrictions on what the program can do at runtime. >> > You need both. >> Yup. Any language besides Java even *try* to provide both for a >> production environment? > Yes. Python tried. It had a module called rexec for that purpose. > I keep mentioning that, and you keep ignoring it. Rexec was around > for a long time, and was removed for technical reasons with some > reluctance. There is nothing un-Pythonic about the idea.
If you've mentioned it before, it wasn't to me. Or maybe my news server dropped it.
Rexec was removed because it didn't work. Just like bastion and every other attempt to create a "safe" environment in Python. Any security wonk worth his pay will tell you that you don't add security to something after the fact if you want good security. You design it in from the beginning.
Of course, what rexec tried to do and what "private" do are orthogonal issues.
<mike
-- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Mike Meyer <m...@mired.org> writes: > I'd say CPython was missing the features that you need to guarantee > that. Missing quite a *lot* of features, in fact. But Python has never > been about keeping people from writing bad code - it's about helping > people write good code.
Privilege separation is considered a good coding practice. How does Python help it? That's what got this started. In the first nontrivial Python program I wrote, I tried to use private variables (which didn't exist) and ended up using using an RPC kludge (described in other post).
> Pretty much every attempt to restrict what other programmers do in > Python has failed - for "implementation issues". I think that's a good > sign that this kind of thing isn't going to work without some serious > work on the interpreter.
You could take it as a sign that the interpreter could benefit from some serious work. It's probably hopeless in CPython. I don't know the situation in Jython. I'll be interested to find out what happens with PyPy. Reimplementating the interpreter in Python surely qualifies as serious work either way.
Paul Rubin <http://phr...@NOSPAM.invalid> writes: > Mike Meyer <m...@mired.org> writes: >> Which brings me to my point. Rather than trying to bandage Python to >> do what you want - and what, based on this thread, a lot of other >> people *don't* want - you should be building a system from the ground >> up to support the kind of B&D environment you want. > Heh, that goes against the principle that Python is supposed to be > good for everything.
And where did you run into that principle? I've certainly never heard it before. It's pretty silly principle, as no language is good for everything. Python is general-purpose, meaning that it's not limited to a small set of problem areas, and that it probably won't be to bad for most uses. But that's not the same thing as being "good" for everything.
Now, one of Python's strengths is that it can glue other applications together. So you can, for instance, write your application-specific code in a language suitable for that problem domain, then wrap those objects for use in a python interpreter - which is very popular in scientific circles - and optionally embed the interpeter in your application so you can invoke scripts as a fundamental part of your application. This makes it possible to deal with that application area with Python - but it doesn't mean Python is good for writing code for that application; just that it's good for wrapping other languages.
<mike
-- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Paul Rubin <http://phr...@NOSPAM.invalid> writes: > Mike Meyer <m...@mired.org> writes: >> I'd say CPython was missing the features that you need to guarantee >> that. Missing quite a *lot* of features, in fact. But Python has never >> been about keeping people from writing bad code - it's about helping >> people write good code. > Privilege separation is considered a good coding practice. How does > Python help it?
With conventions and name mangling. Which are only slightly less effective than the C++/Java technic for doing the same thing.
>> Pretty much every attempt to restrict what other programmers do in >> Python has failed - for "implementation issues". I think that's a good >> sign that this kind of thing isn't going to work without some serious >> work on the interpreter. > You could take it as a sign that the interpreter could benefit from > some serious work.
If you want it to become a secure environment to run untrusted code in, then it definitely neede some serious work. I'd recommend starting by copying /dev/null over all the .c and .h files. Of course, not everyone wants that from Python, so they don't get any benefit from such work.
> I don't know the situation in Jython.
I was going to suggest Jython as a better bet for getting something rexec-like to work. Java was at least intended to provide a secure environment to run untrusted code in, so you're not building on quicksand. IronPython might also be worth a look, based on what little (and it's very little) I know about .NET.
<mike -- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
On Sun, 02 Oct 2005 16:42:49 -0400, Mike Meyer <m...@mired.org> wrote: >Paul Rubin <http://phr...@NOSPAM.invalid> writes: >> Well, it's a discussion of why a certain feature might be useful, not >> that it's required. Mike Meyer points out some reasons it might be >> hard to do smoothly without changing Python semantics in a deep way >> (i.e. Python 3.0 or later).
>Actually, I think that the semantic changes required to make private >do what you want are deep enough that the resulting language wouldn't >be Python any longer. It has deep implications from the interpeter >implementation all the way out to the design of the standard library, >all of which would have to be reworked to make private do "the right >thing."
>Not that I think that private is a bad idea. If I'm not writing >python, then I'm probably writing Eiffel. Eiffel has facilities for >protecting features, though the design is more consistent than the >mishmash one gets in C++/Java. Nuts - in Eiffel, you can't write >"instance.attribute = value"; assignment to an attribute has to be >done in a method of the owning instance.
Actually, ISTM that's true of python too, considering that
instance.attribute = value
is sort of syntactic sugar for
instance.__setattr__('attribute', value)
so it's a (normally inherited) "method of the owning instance" (taking that to mean a method of its class) that does the assignment.
>Which brings me to my point. Rather than trying to bandage Python to >do what you want - and what, based on this thread, a lot of other >people *don't* want - you should be building a system from the ground >up to support the kind of B&D environment you want.
I agree, but I wouldn't agree if I thought you were saying it's useless to define exactly what it is we're talking about before deciding on what needs hw/os/gil/intepreter/convention level support, and how python would make use of such a capability, however implemented.
Paul summarised three levels, of which at least fixing unintentional name-mangling collisions could be solved, IWT.
>Of course, you do realize that in practice you can *never* get what >you want. It assumes that the infrastructure is all bug-free, which >may not be the case.
Yeah, but IRL how close to *never* do you in practice demand to bet your life on it? ;-)
>For example, I once had a system that took a kernel panic trying to >boot an OS upgrade. It had been running the previous versionn of the >OS for most of a year with no problems. Other basically identical >systems ran the upgraded OS just fine. I finally wound up stepping >through the code one instruction at a time, to find that the >subroutine invocation instruction on this machine was setting a bit in >a register that it wasn't supposed to touch, but only in kernel >mode. An internal OS API change meant it only showed up in the >upgraded OS.
>The infamous Pentium floating point bug shows that this case isn't >restricted to failing hardware.
Was that software? I've forgotten the details and am too lazy to google ;-/
Mike Meyer <m...@mired.org> writes: > > Privilege separation is considered a good coding practice. How does > > Python help it?
> With conventions and name mangling. Which are only slightly less > effective than the C++/Java technic for doing the same thing.
That's not what privilege separation means. It means that the privileged objects stay secure even when the unprivileged part of the program is completely controlled by an attacker. It's not used enough. It's typically done with separate processes and maybe separate user accounts:
and in the case of host-security modules (used in banking), it's done with separate hardware. Java (but not C++) can do it with private variables and so forth (applet security depends on that). It's a reasonable style to use in any program that uses files, sockets, etc., and has to cope with possibly-malicious data.
> I was going to suggest Jython as a better bet for getting something > rexec-like to work.
Yeah, that's the thing, we should not think of Python as meaning one particular implementation.
On 2 Oct 2005 10:31:07 -0700, "El Pitonero" <piton...@gmail.com> wrote:
>Bengt Richter wrote:
>> I decided to read this thread today, and I still don't know exactly >> what your requirements are for "private" whatevers.
>No name collision in subclassing. Notice that even if you use
>self._x = 3
>in a parent class, it can be overriden in a sub-sub-class accidentally.
I noticed that, and suggested a GUID-based possible approach ;-)
>> Or maybe, what are your real requirements? >> ;-)
>No need for this type of joke.
I'm not sure what you mean, but I meant no offence, just a nudge to refine the requirements statement, and it seems (and I hope) Paul took it that way. [...]
>Would any Python programmer trade the benefits of a highly dynamic >language with an unessential feature like Java-style "private" data >members? My guess is not.
ISTM it's not a case of either-or here ;-)
>---------------
>What do I say Java-style "private" is unessential?
>If your Python class/object needs a real Java-style private working >namespace, you have to ask yourself: do the private variables REALLY >belong to the class?
>In my opinion, the answer is: NO. Whenever you have Java-style private >variables (i.e, non-temporary variables that need to persist from one >method call to the next time the class node is accessed), those >variables/features may be better described as another object, separate >from your main class hierarchy. Why not move them into a foreign worker >class/object instead, and let that foreign worker object hold those >private names, and separate them from the main class hierarchy? (In >Microsoft's jargon: why not use "containment" instead of >"aggregation"?)
>That is, the moment you need Java-style private variables, I think you >might as well create another class to hold those names and >functionalities, since they do not belong to the core functionality of >the main class hierarchy. Whatever inside the core functionality of the >main class, should perhaps be inheritable, sharable and modifiable.
Such "another class" is what I was suggesting the automatic hidden creation of when I said """ Alternatively, it might be possible to define (in the metaclass call) __getattribute__ for the class so that it notices when method_foo or method_bar are being accessed to create bound methods, and then bind in a proxy "self" instead that would have self.private_var on its own "self" and delegate public attribute accesses to the normal self. Maybe this could get around byte-code-munging at the cost of easier breakin than via inspect etc., and less run time efficiency. Just musing ... """ What kind of implementation were you thinking of?
>If you use containment instead of aggregation, the chance for name >collision reduces dramatically. And in my opinion, it's the Pythonic >way of dealing with the "private" problem: move things that don't >belong to this object to some other object, and be happy again.
Yes, but how do you propose to implement access the "private" variables without changing the the "spelling" too much?
>> Op 2005-09-29, Bill Mill schreef <bill.m...@gmail.com>:
>>> But, if your users can't figure out that they shouldn't be changing >>> the variable called t._test__i without expecting side effects, what do >>> you think of the users of your class?
>>> Python is for consenting adults.
>> No it is not. Consenting means you had the choice. Python doesn't >> give you the choice not to consent.
> Damn straight. I used to be a perfectly happy Pascal programmer, until > Guido and the Timbot kicked down my front door and forced me at gun point > to start programming in Python.
I don't know about you, but when I hear someone saying, this is to be used between consenting adults, I don't expect the choice to be limited between using and not using. Those adults also have to consent on how to use it.
If you limit this "consenting adults" to using python or not, you are making my point.
> Antoon Pardon wrote: >> Op 2005-09-30, Steve Holden schreef <st...@holdenweb.com>:
>>>Antoon Pardon wrote:
>>>>Op 2005-09-29, Bill Mill schreef <bill.m...@gmail.com>:
>>>>>But, if your users can't figure out that they shouldn't be changing >>>>>the variable called t._test__i without expecting side effects, what do >>>>>you think of the users of your class?
>>>>>Python is for consenting adults.
>>>>No it is not. Consenting means you had the choice. Python doesn't >>>>give you the choice not to consent. Unless of course you write >>>>it as a C-extension, then you can hide all you want.
>>>Good grief, the ultimate choice is to use Python because you like it, or >>>not to use it because you don't. Enough with the picking every available >>>nit, please. Consent or stop complaining :-)
>> This is IMO not a nit. IMO people are redefining words. We are also >> talking within a certain context. When people use this slogan, they >> don't mean that people have the choice to not use python.
> Quite true, but people do none the less have that choice. Some days I > wish a few more would exercise it.
Yes they have that choice and the problem is that if too many make that choice, python wouldn't evolve. A lot of progress python made, is because people who didn't like some aspects of it, discussed them and either implemeted a change themselves or got someone else interested enough to implement a change.
"El Pitonero" <piton...@gmail.com> writes: > The thing is, there are two sides to every coin. Features surely can > be viewed as "goodies", or they can be viewed as "handcuffs".
Let's see, say I'm a bank manager, and I want to close my cash vault at 5pm today and set its time lock so it can't be opened until 9am tomorrow, including by me. Is that "handcuffs"? It's normal procedure at any bank, for good reason. It's not necessarily some distrustful policy that the bank CEO set to keep me from robbing the bank. I might have set the policy myself. Java lets me do something similar with instance variables. Why is it somehow an advantage for Python to withhold such a capability?
> Sure, in these very dynamic languages you can ACCIDENTALLY override > the default system behavior. How many Python programmers have once > upon a time done stupid things like: > list = 3
That's just a peculiarity, not any deep aspect of Python. Try it for 'None' instead of 'list':
>>> None = 3 SyntaxError: assignment to None
Why is 'list' somehow different from 'None'? I'd say there's a case to be made for having the compiler protect 'list' and other builtins the same way it protects 'None'. Python won't be any less dynamic because of it.
> The upside is exactly the same as the fact that you can override the > "list()" function in Python. Python is dynamic language.
That's not exactly an upside, and it has nothing to do with Python being dynamic. C is static but you can override 'printf'. Overriding 'list' in Python is pretty much the same thing.
> In Python, if you have a private variable:
> self._x = 3
> and you, for curiosity reasons and DURING runtime (after the program is > already up and running) want to know the exact moment the self._x > variable is accessed (say, print out the local time), or print out the > calling stack frames, you can do it. And I mean the program is running.
So let's see:
def countdown(): n = 3 while n > 0: yield n g = countdown() print g.next() # 3 print g.next() # 2
where's the Python feature that lets me modify g's internal value of n at this point? How is that different from modifying a private instance variable? "Python feature" means something in the language definition, not an artifact of some particular implementation. Is Python somehow deficient because it doesn't give a way to do that? Do you want to write a PEP to add a way? Do you think anyone will take it seriously?
>"El Pitonero" <piton...@gmail.com> writes: >> The thing is, there are two sides to every coin. Features surely can >> be viewed as "goodies", or they can be viewed as "handcuffs".
>Let's see, say I'm a bank manager, and I want to close my cash vault >at 5pm today and set its time lock so it can't be opened until 9am >tomorrow, including by me. Is that "handcuffs"? It's normal >procedure at any bank, for good reason. It's not necessarily some >distrustful policy that the bank CEO set to keep me from robbing the >bank. I might have set the policy myself. Java lets me do something >similar with instance variables. Why is it somehow an advantage for >Python to withhold such a capability?
>> Sure, in these very dynamic languages you can ACCIDENTALLY override >> the default system behavior. How many Python programmers have once >> upon a time done stupid things like: >> list = 3
>That's just a peculiarity, not any deep aspect of Python. Try it for >'None' instead of 'list':
> >>> None = 3 > SyntaxError: assignment to None
>Why is 'list' somehow different from 'None'? I'd say there's a case >to be made for having the compiler protect 'list' and other builtins >the same way it protects 'None'. Python won't be any less dynamic >because of it.
I think I can write you a custom import that will prevent the assignment of a list of names you specify in the code of the imported module. Would that be useful? Or would it be more useful to put that detection in py/lint/checker/etc (where it probably already is?)?
Would you want to outlaw 'None' as an attribute name? Python seems to be straddling the fence at this point:
>>> class C(object): pass ... >>> c = C() >>> c.None = 'c.None' SyntaxError: assignment to None >>> vars(c)['None'] = 'c.None' >>> c.None 'c.None'
>> The upside is exactly the same as the fact that you can override the >> "list()" function in Python. Python is dynamic language.
>That's not exactly an upside, and it has nothing to do with Python >being dynamic. C is static but you can override 'printf'. Overriding >'list' in Python is pretty much the same thing.
>> In Python, if you have a private variable:
>> self._x = 3
>> and you, for curiosity reasons and DURING runtime (after the program is >> already up and running) want to know the exact moment the self._x >> variable is accessed (say, print out the local time), or print out the >> calling stack frames, you can do it. And I mean the program is running.
>So let's see:
> def countdown(): > n = 3 > while n > 0: > yield n > g = countdown() > print g.next() # 3 > print g.next() # 2
>where's the Python feature that lets me modify g's internal value of n >at this point? How is that different from modifying a private >instance variable? "Python feature" means something in the language >definition, not an artifact of some particular implementation. Is >Python somehow deficient because it doesn't give a way to do that? Do >you want to write a PEP to add a way? Do you think anyone will take >it seriously?
I could see it as part of a debugging interface that might let you mess more with frames in general. I wouldn't be surprised if a lot of the under-the-hood access we enjoy as it is was a byproduct of scratching debugging-tool-need itches.
>> Well I have the following reasons not to like the current python way:
>> 1) Beginning all your private variables with an underscore is like >> starting all your integers with an 'i' or all your dictionary with >> a 'd' etc.
> Three points:
> (1) It is utterly pointless in a statically typed language like C to name > integer variables starting with an i, because both you and the compiler > already know it is an integer. However, in a dynamically typed language > like Python, it may in some circumstances make sense, since you have no > other clue as to the expected type of object a variable has.
> (Descriptive names like "arglist" are better, but when the descriptive > name is ambiguous, or there is no sensible descriptive name, this > convention can be helpful.)
I don't think such a convention is that helpfull.
> (2) Hungarian notation wasn't supposed to be about starting integer > variables' names with an i, that was a misunderstanding of the Microsoft > OS division. See here for further details:
> (3) Let's do a small thought experiment. Suppose that Python introduces > "real" private variables. Now that Python has private variables, tens of > thousands of C++ and Java developers immediately rush to use Python in > huge collaborative projects. You're working on one of these huge projects, > and reading the source code to a class that takes 45 pages for its > definition. You're on page 33, and you see these lines:
> # cache the expensive lookup for extra speed > usercache = self.longcomplexcalculation()
> "Ahah!" you say to yourself, "That's exactly what I need to make my code > run faster. Instead of calling Klass.longcomplexcalculation() every time I > need it, I can just look at usercache."
> Quick: was usercache a private variable? How can you tell, short of > searching through those 45 pages of code? You can't.
What's the problem, my editor will find that soon enough.
> Of course, in real Python, as soon as you see _usercache with a leading > underscore, you know it is a private variable, and you don't have to > search the source code to find out. So that's an advantage to the > existing Python system.
But you don't know if it is a private variable of this module or a private variable imported from another module. So you don't know if it is something that you can touch freely but clients should be cautious about or something you should be cautious about.
So take a module with tens of private variables, which are the one that are imported and which are the local ones. You can search all those pages too.
> It seems to me that much of this argument is about terminology, not > reality. We've made a mistake in describing Python as "not having private > variables, only semi-private by convention". Bad bad bad.
> What we should have said is that Python DOES have private variables. In > the same way that Python forces you to use a consistent indentation > scheme, Python forces you to name all your private attributes with a > leading underscore. And just like C++ lets you sneakily access private > variables by defining private as public, so Python lets you sneakily > access private variables by mangling the name.
But private variable are not mangled. Just putting one underscore is apparantlty enough to flag a variable a private, but you need two starting underscores for name mangling. At least that is how I understand:
> Then we'd all be happy, the language zealots would take note that Python's > implementation of private variables has a gotcha to watch out for, and > we'd all be happy.
>> 2) The editor and font I use make it hard to see underscores. They >> usually seem to belong more to the line below than to the actual >> lines.
> That's a bug in the editor/font combination. I've seen some versions of > Abiword cut off the bottom pixel from lines, including underscores. If > your editor made y look like v or u, you'd call it a bug, and if it makes > an underscore disappear or look like part of the next line, that's a bug > too. (Just like Ariel has the bug that the letters r n together look like > the letter m. darn vs dam.
IMO that an underscore looks like it belong to the next line is not a bug. An underscore doesn't belong to this line, it belongs under it.
>> My idea as somekind of compromise between what happens in languages >> like C++ and currently in python would be the following:
>> 1) Allow keywords like private (or implemetation) to mark certain >> variables, functions or classes as an implementation detail. >> Personnally I would prefer the opposite such as a interface >> to mark objects which are not private, but that would break too >> much code.
>> 2) Allow the client access to these private variables, through >> a special construct. Maybe instead of "from ... import ..." >> "from ... spy ...".
> What you are suggesting is that you have private variables that are only > private by convention, since anyone can simply call use spy to treat > them as public. In other words, no different from what Python already > does, except it avoids underscores and introduces at least one new keyword > (spy) and one new syntax element (something to flag a variable as private).
> Yeah, that will make a huge difference.
IMO this convention doesn't make sense, because it doesn't distinghuish between private variables of this module, which the developer is freely to use however it fits and private variables from other modules you should be cautious about. With my suggestion it would be easier to use a convention like:
from module spy var as _var
This would would make the '_' much more usefull as a flag, because it would now stand for an imported private variable.
Paul Rubin <http://phr...@NOSPAM.invalid> writes: > Mike Meyer <m...@mired.org> writes: >> > Privilege separation is considered a good coding practice. How does >> > Python help it? >> With conventions and name mangling. Which are only slightly less >> effective than the C++/Java technic for doing the same thing. > That's not what privilege separation means. It means that the > privileged objects stay secure even when the unprivileged part of the > program is completely controlled by an attacker.
In which case, what's "private" got to do with this? The examples I've seen of it don't give you privilege seperation any more than python does.
Of course, while we're adding things to Python to support what people consider good coding practices, let's not forget:
Design by contract. Covariant method specialization. Class invariants. Avoiding variable aliasing. Hungarian Notation. The Law of Demeter. Loop invariants. Avoiding mixed mode arithmetic. The telephone test. Procedure/function sepration. Type discipline. Contravariant method specialization.
and so on.
<mike
-- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
>>>What if the class author removes a non-private variable or changes a >>>method's documented parameters in the next version of the class, because >>>he think it'll work better, or just because he can?
>> Changing an interface is different from changing the implementation. > > A (documented) interface is like a contract. The implementation is > > just one way to follow that contract.
> Agreed. However, there is also a difference between an interface and > "non-private variables."
> E.g. you have a library, and playing with the code, you notice that by > passing an empty string as a filename, you get the last file accessed. > Cool. You write your program using this feature. Problem is, it's a > quirk of the implementation, and in the next version, the library author > fixes this "bug". Preventing access to private variables wouldn't help - > the only thing you touched was the public parameter to a public function.
But this behaviour of a particular client can't break your library.
> Of course, you could have avoided this by only using the documented > interface, but if we go that route, you wouldn't have to worry about > people accessing private variables, as they wouldn't be documented.
I'm not so much worried about client software that breaks, as I'm worried about server code that breaks because a client fiddled with the private stuff.
> There is little in the way of technical problems that are solved by > language level enforcement of private variables. The issues in question > are mostly social ones, and if you're not reading and following the > documented interface, stopping private variable access is not going to > prevent most of your problems.
It will prevent problems for others. If you are in a project with multiple authors, your usage of private variables can break code that other people rely on.
I you just use undocumented features, you will mostly only create problems in your own code.
>>> People who think that forbidding access to private variables/methods >>> will save themselves from upgrade woes are deluding themselves.
>> It helps, just as locks wont save you from burglars if they really >> want to rob you, but the locks do help.
> Right, but like doors that automatically lock when they close, items > which are there to protect you can be a nusaince, especially when you've > left your keys on the dining room table.
It is always a trade off. Sometimes you settle for more protection and less comfort other times you do the opposite. If private variables would be possible, nobody would force you to use them if you didn't want to.
Mike Meyer <m...@mired.org> writes: > > That's not what privilege separation means. It means that the > > privileged objects stay secure even when the unprivileged part of the > > program is completely controlled by an attacker.
> In which case, what's "private" got to do with this? The examples I've > seen of it don't give you privilege seperation any more than python does.
If you have a java class instance with a private member that's (say) a network socket to a special port, access to the port is controlled entirely by that class. Calling classes can't reach into the instance to get at the socket. Similarly, private variables might hold database passwords, crypto keys, or whatever.
b...@oz.net (Bengt Richter) writes: > Would you want to outlaw 'None' as an attribute name? > Python seems to be straddling the fence at this point: > >>> c.None = 'c.None' > SyntaxError: assignment to None
Heehee, I think that's just a compiler artifact, the lexer is treating None as a keyword instead of a normal lexical symbol that the compiler treats separately. That's also why it raises SyntaxError instead of some other type of error. Yes, None should be ok as an attribute name.
> I could see it as part of a debugging interface that might let you > mess more with frames in general. I wouldn't be surprised if a lot > of the under-the-hood access we enjoy as it is was a byproduct of > scratching debugging-tool-need itches.
Such an interface should probably work like the Java one, i.e. it would have a special socket listener that you'd poke at the program through, not have debugging code running in the same interpreter space as the target app.
On 03 Oct 2005 04:47:26 -0700, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
>b...@oz.net (Bengt Richter) writes: >> Would you want to outlaw 'None' as an attribute name? >> Python seems to be straddling the fence at this point: >> >>> c.None = 'c.None' >> SyntaxError: assignment to None
>Heehee, I think that's just a compiler artifact, the lexer is treating >None as a keyword instead of a normal lexical symbol that the compiler >treats separately. That's also why it raises SyntaxError instead of >some other type of error. Yes, None should be ok as an attribute name.
Not sure whether which compiler. This one seems to differ from the C version.
So the compiler module is happy to generate code that you can execute, but the builtin compiler seems not to be:
>>> c.None = 'c.None' SyntaxError: assignment to None
and definitely not run-time: >>> def foo(): ... c.None = 'c.None' ... File "<stdin>", line 2 SyntaxError: assignment to None
Seems like a bug wrt the intent of making compiler.compile work exactly like the builtin C version. But maybe it has been fixed -- I am still running 2.4 from the beta I built with mingw (because the new microsoft msi loader won't run on my version of NT4 without upgrading that I've got too much dll hell to do on this box. I should also upgrade mingw/msys and recompile, but I spend time here instead ;-/ )
Python 2.4b1 (#56, Nov 3 2004, 01:47:27) [GCC 3.2.3 (mingw special 20030504-1)] on win32 Type "help", "copyright", "credits" or "license" for more information.
On Mon, 03 Oct 2005 09:14:34 +0000, Antoon Pardon wrote: > If you are in a project with > multiple authors, your usage of private variables can break code > that other people rely on.
If you are in a project with multiple authors, your usage of PUBLIC variables can break code that other people rely on.
That's why you have testing. You do test, don't you?
Paul Rubin wrote: > Steven D'Aprano <st...@REMOVETHIScyber.com.au> writes:
>>No, but that is precisely why Python's semi-private variables are >>usually better. Names like _X and class.__X are warnings to the developer >>"use these at your own risk", without preventing developers who need them >>from using them. You have most of the benefits of private variables with >>none of the disadvantages.
> I'm sorry but I thought the idea was to actually reduce the risk of > bugs, not merely attach the risk to the right person.
I'm sorry but you're just plain wrong - at least according to Python's "we're all consenting adults here" philosophy.
> If changing the > way a class uses its own private variables breaks an application > because another class was using the private variable unexpectedly, > then that's bad,
No, it's just an obvious side-effect of the programmer messing with implementation stuff. He knew from the start it was implementation detail, so he accepted that this may change without notice and break its code. period.
> regardless of whether the other class's author was > notified or not.
> It's better to let the compiler automatically flag > these things, than to depend on conventions.
Nope. *you* think it's better. Most of us here obviously think that relying on convention is a good (and even better) solution - and the fact is that it works just fine.
-- bruno desthuilliers python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for p in 'on...@xiludom.gro'.split('@')])"
Paul Rubin <http://phr...@NOSPAM.invalid> writes: >> > That's not what privilege separation means. It means that the >> > privileged objects stay secure even when the unprivileged part of the >> > program is completely controlled by an attacker. >> In which case, what's "private" got to do with this? The examples I've >> seen of it don't give you privilege seperation any more than python does. > If you have a java class instance with a private member that's (say) a > network socket to a special port, access to the port is controlled > entirely by that class.
Are you sure? My understanding was that Java's introspection mechanism could be used to access private variables.
A couple of other things to think about:
Are you sure you want to use the C++ model for privilege separation? C++'s design doesn't exactly inspire confidence in me. I'd recommend checking languages that were designed to be OO from scratch, rather than as extensions or rewrites of other languages. I'd also check dynamic languages to see if any of them do this - other than PHP, which apparently adopted the C++ model, and is another language I wouldn't trust for inspiration.
In static languages, information of this kind is normally attached to variables. In Python, the only thing a variable knows is the object it references. So do you want the privilege information attached to the variable or the object it references? If you attach it to the variable, you're again making what appears to be a fundamental change in Python, and possibly invoking serious implementation headaches. If you attach it to the object, you solve a lot of the problems Pythons reference model creates, but you also leave open the possibility of simple assignment changing an attribute.
Finally, another hole to fix/convention to follow to make this work properly in Python. This one is particularly pernicious, as it allows code that doesn't reference your class at all to violate the private variables. Anyone can dynamically add methods to an instance, the class it belongs to, or to a superclass of that class. This means code in one place can add a method to a superclass of your class that clobbers your private variable, which can then be invoked on an instance of your class to surprise you. So you may have to examine code that doesn't reference your class at all to find the statement that is clobbering your private variable.
<mike -- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Mike Meyer wrote: > Paul Rubin <http://phr...@NOSPAM.invalid> writes:
>>>>That's not what privilege separation means. It means that the >>>>privileged objects stay secure even when the unprivileged part of the >>>>program is completely controlled by an attacker.
>>>In which case, what's "private" got to do with this? The examples I've >>>seen of it don't give you privilege seperation any more than python does.
>>If you have a java class instance with a private member that's (say) a >>network socket to a special port, access to the port is controlled >>entirely by that class.
> Are you sure? My understanding was that Java's introspection mechanism > could be used to access private variables.
Yes, in a Java application with the default security manager it is trivial to access a private variable of another class using introspection. For example:
/* HasPrivate.java */
public class HasPrivate { private int myPrivate = 42;
}
/* HackPrivate.java */
import java.lang.reflect.Field;
public class HackPrivate { public static void main(String[] argv) throws Exception { HasPrivate hp = new HasPrivate(); Field notSoPrivate = hp.getClass().getDeclaredField("myPrivate"); notSoPrivate.setAccessible(true); System.out.println("myPrivate = " + notSoPrivate.getInt(hp)); }
Mike Meyer <m...@mired.org> writes: > > If you have a java class instance with a private member that's (say) a > > network socket to a special port, access to the port is controlled > > entirely by that class.
> Are you sure? My understanding was that Java's introspection mechanism > could be used to access private variables.
Right, I should have been more specific, if I understand correctly, there are some JVM settings that turn that on and off (I'm not any kind of expert on this). For sandbox applets, it's turned off, for example. This came up in a huge discussion thread on sci.crypt a few months ago. Apparently the default is for it to be turned on, to the surprise and disappointment of some:
> A couple of other things to think about: > Are you sure you want to use the C++ model for privilege separation?
I'm not sure what you mean by the C++ model. If you mean the Java model, as I keep saying, applet sandbox security relies on it, so it may not be perfect but it's not THAT bad. Using it routinely in normal applications would be a big improvement over what we do now. High-security applications (financial, military, etc.) would still need special measures.
> C++'s design doesn't exactly inspire confidence in me.
Me neither, "C++ is to C as lung cancer is to lung".
> Finally, another hole to fix/convention to follow to make this work > properly in Python. This one is particularly pernicious, as it allows > code that doesn't reference your class at all to violate the private > variables. Anyone can dynamically add methods to an instance, the > class it belongs to, or to a superclass of that class.
Yes, Python isn't even type-safe any more:
class A(object): pass class B(object): pass a = A() print type(a) a.__class__ = B print type(a) # oops
IMHO that "feature" you describe is quite inessential in Python. The correct way to override or extend the operations on a class is to subclass it. I can't think of a single place where I've seen Python code legitimately go changing operations by jamming stuff into the class object. I'd consider the stdlib's socket.py to be illegitimate and it cost me a couple of hours of debugging one night:
and even that is only messing with specific instances, not classes. Make sure to have a barf bag handy if you look at the socket.py code. I really should open a sf bug about it.
Paul Rubin <http://phr...@NOSPAM.invalid> writes: >> A couple of other things to think about: >> Are you sure you want to use the C++ model for privilege separation? > I'm not sure what you mean by the C++ model. If you mean the Java > model, as I keep saying, applet sandbox security relies on it, so it > may not be perfect but it's not THAT bad. Using it routinely in > normal applications would be a big improvement over what we do now. > High-security applications (financial, military, etc.) would still > need special measures.
I assumed the Java model was based on the C++ model because it seems that everything in Java is based on C++, and they share the same vocabulary. If I'm wrong - well, that means you considered another language already.
Sure, Java might be a big improvement. But Python isn't Java. Rather than just throwing in something that works, do the legwork to verify that you're going to propose a best-of-breed solution. If something perfect is available, wouldn't you feel awful if Python got saddled with something that wasn't perfectd?
>> Finally, another hole to fix/convention to follow to make this work >> properly in Python. This one is particularly pernicious, as it allows >> code that doesn't reference your class at all to violate the private >> variables. Anyone can dynamically add methods to an instance, the >> class it belongs to, or to a superclass of that class.
> Yes, Python isn't even type-safe any more:
> class A(object): pass > class B(object): pass > a = A() > print type(a) > a.__class__ = B > print type(a) # oops
No, that's not the feature I was talking about. But it is *yet another* hole you have to deal with.
> IMHO that "feature" you describe is quite inessential in Python. The > correct way to override or extend the operations on a class is to > subclass it. I can't think of a single place where I've seen Python > code legitimately go changing operations by jamming stuff into the > class object. I'd consider the stdlib's socket.py to be illegitimate > and it cost me a couple of hours of debugging one night:
Note that you can extend an instance as well as a class, though it's a bit more baroque to do so. But people use the ability to add a method to a class dynamically when they want to switch between behaviors for all objects in the class at run time. Personally, I would use a dictionary of methods, but I don't see anything particulary wrong with the latter approach - unless you just dislike dynamic code. Yanking a feature because it gets misued in one place means we should probably pre-emptively yank private to keep people from declaring things private when they shouldn't be.
<mike -- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
|
