Python Interview Questions

[Krypto]

Hi,

I have used Python for a couple of projects last year and I found it
extremely useful. I could write two middle size projects in 2-3 months
(part time). Right now I am a bit rusty and trying to catch up again
with Python.

I am now appearing for Job Interviews these days and I am wondering if
anybody of you appeared for a Python Interview. Can you please share
the questions you were asked. That will be great help to me.

Thanks,

[snd...@gmail.com]

assert() in the c functions extending python ends the interview rather
quickly.
or at least it did for me @IronPort ages ago.
To be honest they have massively multithreaded app
and the last thing they needed was an appliance down because
some programmer was not sure what to do when a certain condition
arises.

[Tim Chase]

While I haven't interviewed precisely for Python, I've been
on the other (interviewing) end and can offer a few of the
sorts of things I ask. I don't expect perfect answers to
all of them, but they show me a range of what the
interviewee knows. I try and give a scattershot of
questions from the following areas to try and narrow down
where they fall in terms of pythonability, and then grill
more deeply around the edges that I find.

Basic Python:
=============
- do they know a tuple/list/dict when they see it?

- when to use list vs. tuple vs. dict. vs. set

- can they use list comprehensions (and know when not to
abuse them? :)

- can they use tuple unpacking for assignment?

- string building...do they use "+=" or do they build a list
and use .join() to recombine them efficiently

- truth-value testing questions and observations (do they
write "if x == True" or do they just write "if x")

- basic file-processing (iterating over a file's lines)

- basic understanding of exception handling

Broader Basic Python:
=====================
- questions about the standard library ("do you know if
there's a standard library for doing X?", or "in which
library would you find [common functionality Y]?") Most
of these are related to the more common libraries such as
os/os.path/sys/re/itertools

- questions about iterators/generators

- questions about map/reduce/sum/etc family of functions

- questions about "special" methods (__<foo>__)

More Advanced Python:
=====================
- can they manipulate functions as first-class objects
(Python makes it easy, but do they know how)

- more detailed questions about the std. libraries (such as
datetime/email/csv/zipfile/networking/optparse/unittest)

- questions about testing (unittests/doctests)

- questions about docstrings vs. comments, and the "Why" of
them

- more detailed questions about regular expressions

- questions about mutability

- keyword/list parameters and unpacked kwd args

- questions about popular 3rd-party toolkits (BeautifulSoup,
pyparsing...mostly if they know about them and when to use
them, not so much about implementation details)

- questions about monkey-patching

- questions about PDB

- questions about properties vs. getters/setters

- questions about classmethods

- questions about scope/name-resolution

- use of lambda

Python History:
===============
- decorators added in which version?

- "batteries included" SQL-capible DB in which version?

- the difference between "class Foo" and "class Foo(object)"

- questions from "import this" about pythonic code

Python Resources:
=================
- what do they know about various Python web frameworks
(knowing a few names is usually good enough, though
knowledge about the frameworks is a nice plus) such as
Django, TurboGears, Zope, etc.

- what do they know about various Python GUI frameworks and
the pros/cons of them (tkinter, wx, pykde, etc)

- where do they go with Python related questions (c.l.p,
google, google-groups, etc)

Other Process-releated things:
==============================
- do they use revision control
(RCS/CVS/Subversion/Mercurial/Git...anything but VSS) and
know how to use it well

- do they write automated tests for their code

Touchy-feely things:
====================
- tabs vs. spaces, and their reasoning

- reason for choosing Python

- choice of editor/IDE

Good luck with your interviewing and hope this helped,

-tkc

[Krypto]

> Good luck with your interviewing and hope this helped,
>
> -tkc

Well, I was looking exactly for this. Many thanks to you Tim. After
going through your list I came to know that I know nothing in Python
and have to catch up a whole lot.

[Tim Chase]

It was certainly not an exhaustive list of "you must know
everything on this list or else you'd never get hired", or
conversely, it's also not a "if you don't know everything on this
list, you're not worthy to call yourself a Python programmer".

It's a way I've found to gauge what somebody means when they say
the program in Python. I've had the gamut of folks where that
phrase means anything from "I glanced at some Python code once"
to "I'm Guido" (okay, so that's a bit of hyperbole, but you get
the idea). My goal is to use as few questions as possible to
flush out just what an interviewee mean by "I program in Python".

Hanging out here on the c.l.p list will introduce you to a lot of
these ideas on the sly. For my "basic" categories,
lists/tuples/dicts/sets as well as list-comprehensions show up
here regularly; I've answered several on basic file processing in
the last day or two (iterate over a file object, doing something
with each line); you see truth-testing regularly (and
chastizement when folks do things like "if foo == True"); you see
basic exception handling...all the basic stuff is regularly
covered/used here.

Knowledge of the available system libraries comes with using them
and having need for them. I'm still learning it. I finally
tired of writing my own command-line parsers and investigated
what the std. lib had to offer, only to find that optparse did
everything I needed: cleanly, readably, and extensibly. So now I
use that instead. That sort of experience only comes from time
emersed in using Python to solve problems.

All that to say, don't sweat it too badly, and that fortunately
Python is an easy language to work with.

-tkc

[Ben Finney]

Krypto <krypto...@gmail.com> writes:

> I am now appearing for Job Interviews these days and I am wondering
> if anybody of you appeared for a Python Interview. Can you please
> share the questions you were asked. That will be great help to me.

I've given some interviews for programming positions. I find it
worthless to ask questions about the language; instead, I ask the
applicant *what they've done* with the language.

Then, I ask them to write some code, usually something simple but
poorly-specified, and observe their approach to the task. That tells
me far more about their knowledge of the language than answers to quiz
questions; it also tells me far more about them as a programmer, which
is what I actually care about.

--
\ "Any intelligent fool can make things bigger and more |
`\ complex... It takes a touch of genius – and a lot of courage |
_o__) – to move in the opposite direction." —Albert Einstein |
Ben Finney

[konryd]

> - string building...do they use "+=" or do they build a list
> and use .join() to recombine them efficiently

I'm not dead sure about that, but I heard recently that python's been
optimized for that behaviour. That means: using += is almost as fast
as joining list. Besides, "+=" is more obvious than the other choice
and since performance is not a problem until it's a problem, I'd
rather use it. Anyway: I wouldn't pick it for an interview question.

regards
Konrad

[Steven Bethard]

konryd wrote:
>> - string building...do they use "+=" or do they build a list
>> and use .join() to recombine them efficiently
>
>
> I'm not dead sure about that, but I heard recently that python's been
> optimized for that behaviour. That means: using += is almost as fast
> as joining list.

For some simple cases, this is true. But it only works in CPython, not
say Jython. So it's a better practice to continue using .join().

STeVe

[Rhamphoryncus]

Concatenating strings using += will often perform quadratically with
the number of concatenations. Your testing will likely use a small
number and not take a noticeable amount of time. Then it'll get
deployed and someone will plug in a much larger number, wondering why
the program is so slow. The recent optimizations just make it more
obscure.

+= shouldn't be an obvious choice for sequences. If it's mutable,
use .append(). If it's immutable, build up in a mutable sequence,
then convert.

--
Adam Olsen, aka Rhamphoryncus

[Paul Rubin]

Rhamphoryncus <rha...@gmail.com> writes:
> += shouldn't be an obvious choice for sequences. If it's mutable,
> use .append(). If it's immutable, build up in a mutable sequence,
> then convert.

I generally prefer to do this with generators rather than mutation.
I.e. instead of

blech = []
while some_condition():
yuck = some mess
blech.append(yuck)
result = ''.join(blech)

I like

def gen_blech():
while some_condition():
yield (some mess)
result = ''.join(gen_blech())

It just seems cleaner. YMMV. Of course when "some mess" is a single
expression instead of multiple statements, it may be possible to use
a genexp without the auxiliary function.

[yeryom...@gmail.com]

[Tim Chase]

On 07/09/12 01:39, yeryom...@gmail.com wrote:
> On Tuesday, 30 October 2007 21:24:04 UTC+2, Tim Chase wrote:

yes, yes I did, almost 5 years ago. :-)

You didn't include any questions/comments on my email, so it's a bit
hard to respond.

>> While I haven't interviewed precisely for Python, I've been
>> on the other (interviewing) end and can offer a few of the
>> sorts of things I ask.

[snip]

>> Python History:
>> ===============
>> - decorators added in which version?
>>
>> - "batteries included" SQL-capible DB in which version?

I've long wanted to update my original post in this department to
include things like the "with" statement as well as my reasoning.
Some of our deployments were stuck on earlier versions (currently
our oldest is a 2.4 deployment that is merrily chugging along,
somewhat stuck due to an external binary dependency for which terms
changed between versions). At the time I wrote that, we had some
2.2 and 2.3 code in production that couldn't use decorators, and
sqlite wasn't officially added until 2.5 (IIRC). Now that
everything is at least 2.4, code can now use decorators freely.

I'd also likely include questions about integer division and other
stuff added in 3.x (such as the default new-class behavior).

So my main motivation was to make sure applicants knew that, in some
of our environments, certain features might not be available. Not
that I wanted to hire a Python historian.

>> Other Process-releated things:
>> ==============================
>> - do they use revision control
>> (RCS/CVS/Subversion/Mercurial/Git...anything but VSS) and
>> know how to use it well
>>
>> - do they write automated tests for their code

Don't let the importance of these two escape. It's SOOOOOOO easy to
grab any of the freely available VCS tools and learn the ropes, or
to write some test code. Failure to do so just seems like
negligence these days.

-tkc

[Roy Smith]

In article <3e0ef383-9615-4b4d...@googlegroups.com>,

yeryom...@gmail.com wrote:

> On Tuesday, 30 October 2007 21:24:04 UTC+2, Tim Chase wrote:

> > - more detailed questions about the std. libraries (such as
> > datetime/email/csv/zipfile/networking/optparse/unittest)

You need to be careful when you ask questions like this. I would expect
somebody to be aware of those and have a high-level understanding of
what they do, but certainly not remember the details of the exact syntax
and argument order. Even with stuff I use everyday (like unittest and
datetime), I have a browser open to the reference manual most of the
time.

> > - questions about PDB

Heh. I would answer that with, "Python Debugger? I've never used it".

> > Python History:
> > ===============
> > - decorators added in which version?
> >
> > - "batteries included" SQL-capible DB in which version?
> >
> > - the difference between "class Foo" and "class Foo(object)"
> >
> > - questions from "import this" about pythonic code

With the exception of the question about new-style classes, these are
silly questions. I was around when both decorators and sqlite3 were
added. I couldn't possible tell you when to any precision better than
"2 dot something". As for the zen of python, it's cute, and a piece of
python folklore, but hardly an essential part of being a good python p

[Tim Chase]

On 07/09/12 08:25, Roy Smith wrote:
>> On Tuesday, 30 October 2007 21:24:04 UTC+2, Tim Chase wrote:
>
>>> - more detailed questions about the std. libraries (such as
>>> datetime/email/csv/zipfile/networking/optparse/unittest)
>
> You need to be careful when you ask questions like this. I would expect
> somebody to be aware of those and have a high-level understanding of
> what they do, but certainly not remember the details of the exact syntax
> and argument order. Even with stuff I use everyday (like unittest and
> datetime), I have a browser open to the reference manual most of the
> time.

Yeah, the aim isn't to grill them on the minutia, but to get a
feeling that they know the basics. The zipfile module offers a
ZipFile object for reading/writing zip files with or without
compression. The CSV file allows for reading/writing CSV files with
definable delimiters and quoting/escaping. Etc.

>>> - questions about PDB
>
> Heh. I would answer that with, "Python Debugger? I've never used it".

The ability to know off the top of your head that it's the "Python
Debugger" is more than enough :-) That's just first-order
ignorance: you know what you don't know and can spend a few minutes
reading up on it if you need it. The second[or higher]-order
ignorance of not knowing what pdb is (or, if you need more powerful
debugging, how to do it) is sign the person hasn't been programming
in Python much.

>>> Python History:
>>> ===============
>>> - decorators added in which version?
>>>
>>> - "batteries included" SQL-capible DB in which version?
>>>
>>> - the difference between "class Foo" and "class Foo(object)"
>>>
>>> - questions from "import this" about pythonic code
>
> With the exception of the question about new-style classes, these are
> silly questions. I was around when both decorators and sqlite3 were
> added. I couldn't possible tell you when to any precision better than
> "2 dot something".

I'd even be satisfied if a person just knew that such features
weren't there all along and might need to be worked around for older
deployments.

> As for the zen of python, it's cute, and a piece of python
> folklore, but hardly an essential part of being a good python p

[Ed: something appears to have gotten truncated there] Yeah, it's
more about a person being sufficiently steeped in python to know
bits and pieces of the zen, and their ability to recognize/create
pythonic code. I've seen enough Java-written-in-Python to know what
I don't want :-)

-tkc

[Rick Johnson]

On Jul 9, 12:40 pm, Tim Chase <python.l...@tim.thechases.com> wrote:
> The second[or higher]-order
> ignorance of not knowing what pdb is (or, if you need more powerful
> debugging, how to do it) is sign the person hasn't been programming
> in Python much.

So guru knowledge of pdb is prerequisite to being accepted as a
Pythonista? I find that ridiculous since *real* programmers don't use
debuggers anyway.

> [Ed: something appears to have gotten truncated there]  Yeah, it's
> more about a person being sufficiently steeped in python to know
> bits and pieces of the zen, and their ability to recognize/create
> pythonic code.  I've seen enough Java-written-in-Python to know what
> I don't want :-)

I know you are a member of the group who has an aversion to strict OOP
paradigm but is this a justified aversion, or just fear of OOP due to
static evolution? Look, i don't like java's strict approach either,
however, i do not have an aversion to OOP.

[Demian Brecht]

Definitely appreciate your approach, I've asked similar questions when interviewing.

I also usually like to ask what a candidate likes and dislikes about Python, hoping for the GIL to creep up, along with an explanation as to what it is, implementations that don't have it along with methods of getting around the lock (although that would be a fairly advanced topic IMHO). If it doesn't come up, sometimes I'll pop it in depending on their level of experience.

[Peter]

One of my favourite questions when interviewing - and it was 100% reliable :-) - "what are your hobbies?"

If the answer included programming then they were hired, if not, then they went to the "B" list.

In my experience, anybody who is really interested in programming will have it as a hobby (and is keen to learn even if they don't currently have the knowledge you require) - otherwise it is "just a job". Every job has a learning curve - whether it is just the particular domain or even a new language, the individual who sees programming as more "than a job" will come up to speed much faster and be more productive in both the short and long term.

Every programmer I have ever hired using this criteria worked out well.

[Peter]

[Devin Jeanpierre]

On Mon, Jul 9, 2012 at 5:22 PM, Peter <peter.m...@gmail.com> wrote:
> One of my favourite questions when interviewing - and it was 100% reliable :-) - "what are your hobbies?"
> If the answer included programming then they were hired, if not, then they went to the "B" list.

Woe is the poor college grad, who wants to appear like a well-rounded
individual and lists capoeira and gardening, instead.

-- Devin

[Cameron Simpson]

On 09Jul2012 11:44, Rick Johnson <rantingri...@gmail.com> wrote:
| On Jul 9, 12:40 pm, Tim Chase <python.l...@tim.thechases.com> wrote:
| > The second[or higher]-order
| > ignorance of not knowing what pdb is (or, if you need more powerful
| > debugging, how to do it) is sign the person hasn't been programming
| > in Python much.
|
| So guru knowledge of pdb is prerequisite to being accepted as a
| Pythonista? I find that ridiculous since *real* programmers don't use
| debuggers anyway.

You've misread him. He's saying not knowing what PDB is and what it may
be used for is a sign of low experience. Whether one uses it or not
isn't what he's measuring, it's whether one knows what it is for and how
it may be used.

| > [...]  I've seen enough Java-written-in-Python to know what

| > I don't want :-)
|
| I know you are a member of the group who has an aversion to strict OOP
| paradigm but is this a justified aversion, or just fear of OOP due to
| static evolution? Look, i don't like java's strict approach either,
| however, i do not have an aversion to OOP.

More misreading. "Java-written-in-Python" (and its variants) means non-python
code written in python, in this case from someone with a strong (or
rigid) Java background who is not adept with Python idioms. It has
nothing to do with OOP one way or the other. Surely we've all seen
(and doubtless written) clumsy python code; this is an example.

Cheers,
--
Cameron Simpson <c...@zip.com.au>

A strong conviction that something must be done is the parent of many
bad measures. - Daniel Webster

[Roy Smith]

In article <mailman.1959.1341868...@python.org>,

Peter <peter.m...@gmail.com> wrote:

> One of my favourite questions when interviewing - and it was 100% reliable
> :-) - "what are your hobbies?"

"My hobby happens to be gardening, for which I don't expect to be paid."

[Cameron Simpson]

On 09Jul2012 18:53, Devin Jeanpierre <jeanpi...@gmail.com> wrote:

| On Mon, Jul 9, 2012 at 5:22 PM, Peter <peter.m...@gmail.com> wrote:
| > One of my favourite questions when interviewing - and it was 100% reliable :-) - "what are your hobbies?"

| > If the answer included programming then they were hired, if not, then they went to the "B" list.
|
| Woe is the poor college grad, who wants to appear like a well-rounded
| individual and lists capoeira and gardening, instead.

A new word! A quick google search confused me as to whether this was
martial art or a form of dance. The GF suggested it was both. Having
seen this:

http://www.youtube.com/watch?v=Z8xxgFpK-NM

I am now convinced :-)
--
Cameron Simpson <c...@zip.com.au>

Hit the button Chewie! - Han Solo

[Christian Heimes]

Am 09.07.2012 23:22, schrieb Peter:
> One of my favourite questions when interviewing - and it was 100% reliable :-) - "what are your hobbies?"
>
> If the answer included programming then they were hired, if not, then they went to the "B" list.

on the contrary! When a potential candidate has computer stuff as her
main hobby then she goes on the no-hire list. I prefer people that can
cope with stress and pressure as well as people who can think outside
the box. When you work with computers all day at work *and* at home then
you are unable to shut off mentally.

Gardening is great hobbies for a developer. You need to be patient,
reliable and provide constantly good work to grow your own vegetables.

Christian

[Tim Chase]

On 07/09/12 17:53, Devin Jeanpierre wrote:
>> One of my favourite questions when interviewing - and it was
>> 100% reliable :-) - "what are your hobbies?" If the answer
>> included programming then they were hired, if not, then they
>> went to the "B" list.
>

> Woe is the poor college grad, who wants to appear like a
> well-rounded individual and lists capoeira and gardening,
> instead.

The problem is the "instead...if your list of hobbies includes
capoeira and gardening in addition to programming, you're at least
considered. :-)

-tkc

[Roy Smith]

In article <mailman.1965.1341876...@python.org>,

Christian Heimes <li...@cheimes.de> wrote:

> Am 09.07.2012 23:22, schrieb Peter:
> > One of my favourite questions when interviewing - and it was 100% reliable
> > :-) - "what are your hobbies?"
> >
> > If the answer included programming then they were hired, if not, then they
> > went to the "B" list.
>
> on the contrary! When a potential candidate has computer stuff as her
> main hobby then she goes on the no-hire list.

I think this points out the silliness of these kinds of questions.
There is no "right" answer. More to the point, the interviewee, when
he/she gets one of these questions, probably goes into hyper-analysis
mode: "Now, just what did he mean by that question?" He's likely to
give the answer he thinks you want to hear.

Do you really want to make hire/no-hire decisions based on somebody's
ability to second-guess what you probably wanted to hear when you asked
a pointless question?

[Tim Chase]

On 07/09/12 18:12, Cameron Simpson wrote:
> On 09Jul2012 18:53, Devin Jeanpierre <jeanpi...@gmail.com> wrote:
> | On Mon, Jul 9, 2012 at 5:22 PM, Peter <peter.m...@gmail.com> wrote:

> | > One of my favourite questions when interviewing - and it was 100% reliable :-) - "what are your hobbies?"
> | > If the answer included programming then they were hired, if not, then they went to the "B" list.
> |

> | Woe is the poor college grad, who wants to appear like a well-rounded
> | individual and lists capoeira and gardening, instead.
>

> A new word! A quick google search confused me as to whether this was
> martial art or a form of dance. The GF suggested it was both.

You were unfamiliar with the word "gardening" and unsure whether it
was a martial art or a form of dance?

http://groovewitch.files.wordpress.com/2011/07/ninja-garden-gnome-1.jpeg

«grins, ducks & runs»

-tkc

[Emile van Sebille]

On 7/9/2012 2:22 PM Peter said...

Hence the age bias.

Personally, I'm quite happy now that I've divorced my hobby from my
career. And my family likes me better too.

Emile

[dnc...@gmail.com]

Tim,

I've read your list and with one exception it all looks very reasonable. (As an hobbiest, I'm amazed at just how much I have picked up.)

The set of questions I'm not sure I understand is the 'What version did ... appear?' questions. This, to me, doesn't seem to indicate any programming experience or expertise. A question asking 'Do you understand different versions?' and 'How would you find out whether a particular version can do a particular thing?' (i.e. which version can you use on GAE?) would seem to give good information. 'How do decorators work?' would seem to give good information.

So my question is: what information are you looking for by asking which version introduced decorators?

With respect,

Den

[Christian Heimes]

Am 10.07.2012 01:40, schrieb Roy Smith:
> Do you really want to make hire/no-hire decisions based on somebody's
> ability to second-guess what you probably wanted to hear when you asked
> a pointless question?

I don't want her/him to second-guess at all. I expect a straight and
honest answer. Second-guessing leads to cheating and lying which doesn't
work in the long run.

"I don't like to disclose my personal life" is also a good answer as it
shows that the candidate self-confidence and doesn't sell privacy for a job.

[dnc...@gmail.com]

[Tim Chase]

On 07/09/12 19:01, dnc...@gmail.com wrote:
> The set of questions I'm not sure I understand is the 'What
> version did ... appear?' questions. This, to me, doesn't seem to
> indicate any programming experience or expertise. A question
> asking 'Do you understand different versions?' and 'How would you
> find out whether a particular version can do a particular thing?'
> (i.e. which version can you use on GAE?) would seem to give good
> information.

The reason *I* ask them is that we have some 2.4 installations
(where things like "with" aren't available) and at the time I typed
up the list, there was some earlier 2.2 and 2.3 code out there where
decorators or sqlite[*] didn't work.

So I guess it's a bit of a "how long have they been programming in
python" experience aspect. Programmers that have been around a
while often remember the frustration of $FEATURE_LACK and then the
relief of a much better way to do it. The functionality of
decorators was around far earlier, but the clean syntactic sugar
made it much nicer to use. The sqlite/sqlite3 libraries were
around, but you had to install them yourself (whether from source, a
custom installer, or your package manager).

As mentioned in another branch of this thread, I don't require
python historians, but do prefer folks that know which features to
check availability for deployment.

-tkc

[*] without installing an add-on

[Roy Smith]

In article <mailman.1972.1341879...@python.org>,

Tim Chase <pytho...@tim.thechases.com> wrote:

> As mentioned in another branch of this thread, I don't require
> python historians, but do prefer folks that know which features to
> check availability for deployment.

Heh. Tell me, when did strings get methods? :-)

[Tim Chase]

On 07/09/12 19:27, Roy Smith wrote:
>> prefer folks that know which features to check availability for
>> deployment.
>
> Heh. Tell me, when did strings get methods? :-)

IIRC, ~2.0? I'm cognizant of the shift happening from the string
module to string methods, but I wouldn't expect deep history
knowledge--last I checked, RedHat still supports a RHEL version that
ships with 2.4, so that's about as far back as I'd probe these days
(so I'd drop the decorators question now). Certainly not a
deal-breaker either way. Just more data points.

-tkc

[Mark Lawrence]

On 10/07/2012 00:33, Christian Heimes wrote:
> Am 09.07.2012 23:22, schrieb Peter:

>> One of my favourite questions when interviewing - and it was 100% reliable :-) - "what are your hobbies?"
>>
>> If the answer included programming then they were hired, if not, then they went to the "B" list.
>

> on the contrary! When a potential candidate has computer stuff as her

> main hobby then she goes on the no-hire list. I prefer people that can
> cope with stress and pressure as well as people who can think outside
> the box. When you work with computers all day at work *and* at home then
> you are unable to shut off mentally.
>
> Gardening is great hobbies for a developer. You need to be patient,
> reliable and provide constantly good work to grow your own vegetables.
>
> Christian
>

I guess that's why I detest gardening :-)

--
Cheers.

Mark Lawrence.

[Shambhu Rajak]

I agree with Christian, a developer should have hobbies other than computer stuffs. Versatile environment give more
Ability to think differently.
I like playing guitar :-)

Be enthu, run foolishly and learn intelligently.

-Shambhu

-----Original Message-----
From: Christian Heimes [mailto:li...@cheimes.de]
Sent: 10/07/2012 5:03 AM
To: pytho...@python.org
Subject: Re: Python Interview Questions

Am 09.07.2012 23:22, schrieb Peter:

> One of my favourite questions when interviewing - and it was 100% reliable :-) - "what are your hobbies?"
>
> If the answer included programming then they were hired, if not, then they went to the "B" list.

[rusi]

On Jul 10, 4:40 am, Roy Smith <r...@panix.com> wrote:
> In article <mailman.1965.1341876813.4697.python-l...@python.org>,

Add to that the Heisenberging that happens to interviewees (and
interviewers) from reading this thread

[Steven D'Aprano]

On Tue, 10 Jul 2012 01:33:06 +0200, Christian Heimes wrote:

> Am 09.07.2012 23:22, schrieb Peter:
>> One of my favourite questions when interviewing - and it was 100%
>> reliable :-) - "what are your hobbies?"
>>
>> If the answer included programming then they were hired, if not, then
>> they went to the "B" list.
>
> on the contrary! When a potential candidate has computer stuff as her
> main hobby then she goes on the no-hire list. I prefer people that can
> cope with stress and pressure as well as people who can think outside
> the box. When you work with computers all day at work *and* at home then
> you are unable to shut off mentally.

...because the stuff you do on computers at work, and the stuff you do on
computers at home, are obviously exactly the same thing.

If they read for their job, and read at home, does that also mean they
can't shut off?

This is why I hate job interviews. You have like 30 minutes, or even as
little as 30 seconds, to make a good impression on somebody who may or
may not be capable of telling the difference between a cheese sandwich
and a box of hair -- and even the *good* interviewers are probably making
their judgement on the basis of subjective factors with no right or wrong
answers.

> Gardening is great hobbies for a developer. You need to be patient,
> reliable and provide constantly good work to grow your own vegetables.

And live in a house in the suburbs with enough room for a garden, good
soil, and not in the shadow of buildings. And work hours where you are
home during daylight hours.

You know what else is a great hobby for a developer? Anything, or
nothing, at all.


--
Steven

[Chris Angelico]

On Tue, Jul 10, 2012 at 3:08 PM, Shambhu Rajak
<Shambh...@kpitcummins.com> wrote:
> I agree with Christian, a developer should have hobbies other than computer stuffs. Versatile environment give more
> Ability to think differently.
> I like playing guitar :-)

Music and programming do go VERY well together. My hobbies include
online roleplaying (Dungeons & Dragons, etc), writing/managing a MUD,
playing the church organ, and arranging 19th-century music. It's not
at all an uncommon pairing.

But would a job interviewer REALLY care that I spend my Sunday
mornings up front, hiding behind two manuals and a set of faulty
pedals? Or would it be of interest that I play the odd video game (and
believe you me, some of the games I play are VERY odd)? If so, I
hereby resign all hope of comprehending job interviews, and will fall
back on Mr Hall Pycroft's notion[1] that there's absolutely no logic
to them at all.

ChrisA

[1] cf Sherlock Holmes: The Adventure of the Stock-Broker's Clerk

[Christian Heimes]

Am 10.07.2012 09:33, schrieb Steven D'Aprano:
> This is why I hate job interviews. You have like 30 minutes, or even as
> little as 30 seconds, to make a good impression on somebody who may or
> may not be capable of telling the difference between a cheese sandwich
> and a box of hair -- and even the *good* interviewers are probably making
> their judgement on the basis of subjective factors with no right or wrong
> answers.

IMHO one category of answers is always wrong: lies. You may oversell
yourself a bit, you can (and should) keep private matters to yourself
but don't lie.

> And live in a house in the suburbs with enough room for a garden, good
> soil, and not in the shadow of buildings. And work hours where you are
> home during daylight hours.

Almost everybody can garden under ideal conditions. I grow about 15
herbs, strawberries, tomatoes, chillies and flowers on a small balcony
in the middle of the city. This year I'm going to harvest at least 200
tomatoes from two plants in a 1m * 40cm * 40cm box of soil. I even have
a calabash plant that grows like crazy. See? :)

Christian

[Jean-Michel Pichavant]

Tim Chase wrote:
> On 07/09/12 19:27, Roy Smith wrote:
>

>>> prefer folks that know which features to check availability for
>>> deployment.
>>>
>> Heh. Tell me, when did strings get methods? :-)
>>
>

> IIRC, ~2.0? I'm cognizant of the shift happening from the string
> module to string methods, but I wouldn't expect deep history
> knowledge--last I checked, RedHat still supports a RHEL version that
> ships with 2.4, so that's about as far back as I'd probe these days
> (so I'd drop the decorators question now). Certainly not a
> deal-breaker either way. Just more data points.
>
> -tkc
>
>
>
>

Why would you want to hire someone that knows something pointless as the
version where feature X has been introduced ? Just tell him that feature
X has been introducted in version Y, costless 2.5sec training. Don't you
want to hire someone that knows things you don't and benefit from each
others abilities, learning from each others, improving the company
global skill range ?

JM

[Mark Lawrence]

Surely the purpose of asking questions about hobbies or similar is to
establish whether or not the person is likely to fit in?

Slightly different tack, you have to get into the interview, i.e. pass
the first thirty seconds test. I recall reading in a book in the local
library of a manager that wouldn't employ people unless they were
wearing a new pair of shoes. Guess they didn't take many people on.

--
Cheers.

Mark Lawrence.

[Mark Lawrence]

On 10/07/2012 09:11, Christian Heimes wrote:
> Almost everybody can garden under ideal conditions. I grow about 15
> herbs, strawberries, tomatoes, chillies and flowers on a small balcony
> in the middle of the city. This year I'm going to harvest at least 200
> tomatoes from two plants in a 1m * 40cm * 40cm box of soil. I even have
> a calabash plant that grows like crazy. See? :)
>
> Christian
>

Big 'ead :)

--
Cheers.

Mark Lawrence.

[rusi]

On Jul 10, 12:33 pm, Steven D'Aprano <steve

+comp.lang.pyt...@pearwood.info> wrote:
> This is why I hate job interviews. You have like 30 minutes, or even as
> little as 30 seconds, to make a good impression on somebody who may or
> may not be capable of telling the difference between a cheese sandwich
> and a box of hair -- and even the *good* interviewers are probably making
> their judgement on the basis of subjective factors with no right or wrong
> answers.

You make it sound terrible...
But just think which is worse: jobs decided in 30 minutes or marriages
decided in 30 seconds?

[BartC]

"Peter" <peter.m...@gmail.com> wrote in message
news:35e7a860-fd41-4018...@googlegroups.com...

> One of my favourite questions when interviewing - and it was 100% reliable
> :-) - "what are your hobbies?"
>
> If the answer included programming then they were hired, if not, then they
> went to the "B" list.
>

> In my experience, anybody who is really interested in programming will
> have it as a hobby (and is keen to learn even if they don't currently have
> the knowledge you require) - otherwise it is "just a job".

Won't they be tempted to work on their pet project instead of what they're
being paid for? There's also the risk of mixing up software created at home,
with that done at work, with all the intellectual property issues that might
arise.

--
Bartc

[Ethan Furman]

Jean-Michel Pichavant wrote:
> Why would you want to hire someone that knows something pointless as the
> version where feature X has been introduced ?

As an example from today, if someone claimed to have 5+ years of Python
experience, but didn't know that 'with' was standard in 2.6 (or at least
the end of the 2.x cycle) I would be suspicious that they actually had
the experience they claimed.

~Ethan~

[Chris Angelico]

On Wed, Jul 11, 2012 at 1:55 AM, BartC <b...@freeuk.com> wrote:
> There's also the risk of mixing up software created at home, with that done
> at work, with all the intellectual property issues that might arise.

You just make the matter clear from the beginning, for instance:
what's done at work stays at work, and copyright is assigned by the
act of pushing to the repository. I've lifted oddments of code from my
home projects to use at work; it's no different from using skills
learned at home, which is exactly what a programmer is being paid for.

This is another good reason to make license terms clear and explicit
on every project you ever put a hand to. Doesn't matter who's lifting
code from where to where, it's easy to work out whether it's
permissible or not.

ChrisA

[Steven D'Aprano]

Be careful of jumping to conclusions though. Perhaps they have five years
experience with Python 1.5, 2.3 and 2.4 on Centos systems. Of course, if
they try to sell themselves as having five years experience with Python
3.2 and they don't know anything about the with statement, that tells you
everything you need to know.



--
Steven

[Chris Angelico]

On Wed, Jul 11, 2012 at 2:34 AM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> Of course, if they try to sell themselves as having

> five years experience with Python 3.2...

... then they've been borrowing Guido's time machine for personal purposes.

ChrisA

[Steven D'Aprano]

On Tue, 10 Jul 2012 11:29:24 +0200, Jean-Michel Pichavant wrote:

> Why would you want to hire someone that knows something pointless as the
> version where feature X has been introduced ? Just tell him that feature
> X has been introducted in version Y, costless 2.5sec training. Don't you
> want to hire someone that knows things you don't and benefit from each
> others abilities, learning from each others, improving the company
> global skill range ?

The reason for the question is to get some idea of how well the candidate
actually knows Python. If you ask them questions that you don't know the
answer to, how will you tell if they're right?

I certainly wouldn't disqualify a candidate if they didn't know what
version introduced (say) decorators. If they said "what's a decorator?"
or "version 10", that would be a hint that they don't actually know much
about Python. If they said "I don't know, I'm still stuck on Python 2.3",
they would get a point for honesty and lose a point for being way out of
date. If they said version 2.3 or 2.5 (it's actually 2.4), well, that's
close enough.

Of course, an acceptable answer would be "buggered if I know, but if you
give me a minute, I'll google it for you".


--
Steven

[Steven D'Aprano]

On Tue, 10 Jul 2012 10:11:22 +0200, Christian Heimes wrote:

> Am 10.07.2012 09:33, schrieb Steven D'Aprano:
>> This is why I hate job interviews. You have like 30 minutes, or even as
>> little as 30 seconds, to make a good impression on somebody who may or
>> may not be capable of telling the difference between a cheese sandwich
>> and a box of hair -- and even the *good* interviewers are probably
>> making their judgement on the basis of subjective factors with no right
>> or wrong answers.
>
> IMHO one category of answers is always wrong: lies. You may oversell
> yourself a bit, you can (and should) keep private matters to yourself
> but don't lie.

If only that were true. I know quite a few people who looked the
interviewer straight in the eye and told the most bare-faced lies without
a trace of shame, and got the job. Ten years on, at least one of them is
making something around $300,000 a year, based entirely on his ability to
smile and tell customers plausible lies.

I can't lie to save my life, which is why I have trouble in interviews.
But of course all good liars would say the same thing.



--
Steven

[Ethan Furman]

Chris Angelico wrote:
> On Wed, Jul 11, 2012 at 2:34 AM, Steven D'Aprano
> <steve+comp....@pearwood.info> wrote:

>> Of course, if they try to sell themselves as having

>> five years experience with Python 3.2...
>
> ... then they've been borrowing Guido's time machine for personal purposes.

Reminds me of a job posting a few years ago where the prospective
employer wanted three plus years experience in some language, and that
language had only been created a year and a half before.

~Ethan~

[Chris Angelico]

On Wed, Jul 11, 2012 at 2:51 AM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> If only that were true. I know quite a few people who looked the
> interviewer straight in the eye and told the most bare-faced lies without
> a trace of shame, and got the job. Ten years on, at least one of them is
> making something around $300,000 a year, based entirely on his ability to
> smile and tell customers plausible lies.

So he's either a politician, a salesman, a lawyer, a counselor, a
manager, a thespian, or a venture capitalist. And maybe a few other
possibilities. Professional liars, all. :)

ChrisA

[Jean-Michel Pichavant]

Must be a cultural thing. We don't question people experience that much
here. They'll be challenged anyway during the trial period (6 months
during which the contract can be cancelled anytime without any reason).
Actually I think it would be considered quite rude to challenge someone
with questions right after he told you he worked 5 years as technical
leader on a software developped in python for instance.

I've never been asked nor did I asked to go into such technical details.
Interviews are more about years of experience, projects, working with
teams, carreer expectations, distance between home and workplace,
willingness to work weekends when required.

I'm no saying one way is better than another. I'm making an observation
on how different can be an interview from one location to another.

JM

[Steven D'Aprano]

Actually, he's a senior software developer for a major international
software company whose name Might Seem familiar to you.

To be honest, I can't tell you too much more about his job, as I've made
it a practice not to learn too many details.


--
Steven

[Mark Lawrence]

On 10/07/2012 18:12, Dennis Lee Bieber wrote:
> On 10 Jul 2012 07:33:59 GMT, Steven D'Aprano
> <steve+comp....@pearwood.info> declaimed the following in
> gmane.comp.python.general:

>
>
>> may not be capable of telling the difference between a cheese sandwich
>> and a box of hair -- and even the *good* interviewers are probably making
>

> They are both containers holding samples of protein
>

Does the hair contain much more roughage?

--
Cheers.

Mark Lawrence.

[Demian Brecht]

I also judge candidates on their beards (http://www.wired.com/wiredenterprise/2012/06/beard-gallery/). If the beard's awesome enough, no questions needed. They're pro.

[Ian Kelly]

On Tue, Jul 10, 2012 at 4:26 PM, Dennis Lee Bieber
<wlf...@ix.netcom.com> wrote:
> On Tue, 10 Jul 2012 09:05:50 -0700, Ethan Furman <et...@stoneleaf.us>

> declaimed the following in gmane.comp.python.general:
>

>> As an example from today, if someone claimed to have 5+ years of Python
>> experience, but didn't know that 'with' was standard in 2.6 (or at least
>> the end of the 2.x cycle) I would be suspicious that they actually had
>> the experience they claimed.
>>
>

> >From the 2.5 help file:
> """
> 3.4.9 With Statement Context Managers
> New in version 2.5.
> """

In 2.5 the with statement requires a __future__ import, so can't be
considered "standard".

[David Robinow]

On Tue, Jul 10, 2012 at 1:02 PM, Ethan Furman <et...@stoneleaf.us> wrote:
...

>
> Reminds me of a job posting a few years ago where the prospective employer
> wanted three plus years experience in some language, and that language had
> only been created a year and a half before.

I saw several of those when Java was new.

[Rick Johnson]

On Jul 10, 4:29 am, Jean-Michel Pichavant <jeanmic...@sequans.com>
wrote:

> Why would you want to hire someone that knows something pointless as the
> version where feature X has been introduced ? Just tell him that feature
> X has been introducted in version Y, costless 2.5sec training. Don't you
> want to hire someone that knows things you don't and benefit from each
> others abilities, learning from each others, improving the company
> global skill range ?
>
> JM

Ha! Intelligent people are scary to bosses. They want robots Jean.
Robots that are *just* intelligent enough to reduce their own work
load whist NOT intelligent enough to render them obsolete.

[Stefan Behnel]

Mark Lawrence, 10.07.2012 11:42:

> I recall reading in a book in the local library
> of a manager that wouldn't employ people unless they were wearing a new
> pair of shoes. Guess they didn't take many people on.

Managers tend to like wasting resources. Buying a new pair of shoes for
each job interview sounds reasonable once you have a salary well beyond
your own capabilities.

Stefan

[Robert Miles]

On 7/10/2012 1:08 PM, Demian Brecht wrote:
> I also judge candidates on their beards (http://www.wired.com/wiredenterprise/2012/06/beard-gallery/). If the beard's awesome enough, no questions needed. They're pro.

You should hire me quickly, then, since I have a beard, already turning
partly gray. Never mind that the computer languages I have studied
enough to write even one program don't yet include Python.

[charvi...@gmail.com]

Hi Guys,

Finally I have decided to put best interview question and answers.

Please visit http://www.f2finterview.com/web/CorePython/ for core python and http://www.f2finterview.com/web/PythonAdvanced/ for advanced python

On Tuesday, October 30, 2007 11:44:01 PM UTC+5:30, Krypto wrote:
> Hi,
>
> I have used Python for a couple of projects last year and I found it
> extremely useful. I could write two middle size projects in 2-3 months
> (part time). Right now I am a bit rusty and trying to catch up again
> with Python.
>
> I am now appearing for Job Interviews these days and I am wondering if
> anybody of you appeared for a Python Interview. Can you please share
> the questions you were asked. That will be great help to me.
>
> Thanks,

[Kushal Kumaran]

On Wed, Sep 5, 2012 at 12:20 PM, <charvi...@gmail.com> wrote:
> On Tuesday, October 30, 2007 11:44:01 PM UTC+5:30, Krypto wrote:
>> Hi,
>>
>> I have used Python for a couple of projects last year and I found it
>> extremely useful. I could write two middle size projects in 2-3 months
>> (part time). Right now I am a bit rusty and trying to catch up again
>> with Python.
>>
>> I am now appearing for Job Interviews these days and I am wondering if
>> anybody of you appeared for a Python Interview. Can you please share
>> the questions you were asked. That will be great help to me.
>>
>
> Finally I have decided to put best interview question and answers.
>
> Please visit http://www.f2finterview.com/web/CorePython/ for core python and http://www.f2finterview.com/web/PythonAdvanced/ for advanced python
>

As I see from a quick glance, several of your answers seem to be
copied from the python faq at http://docs.python.org/faq/. The
copyright notice for the python.org website seems to be at
http://docs.python.org/copyright.html. Do you have the Python
Software Foundation's permission to copy large chunks of the
python.org website and claim it as your own content?

--
regards,
kushal

[Peter Otten]

charvi...@gmail.com wrote:

> Finally I have decided to put best interview question and answers.
>

> Please visit http://***/web/CorePython/ for core python
> and http://***/web/PythonAdvanced/ for advanced python

Hm, are you a reformed PHP programmer who has never heard of sql injection
attacks? The first "advanced" answer (and probably all the database-related
stuff) should definitely be withdrawn.

[Chris Angelico]

On Thu, Sep 6, 2012 at 12:21 AM, Peter Otten <__pet...@web.de> wrote:
> charvi...@gmail.com wrote:
>
>> Finally I have decided to put best interview question and answers.
>>

>> Please visit http://***/web/CorePython/ for core python
>> and http://***/web/PythonAdvanced/ for advanced python
>
> Hm, are you a reformed PHP programmer who has never heard of sql injection
> attacks? The first "advanced" answer (and probably all the database-related
> stuff) should definitely be withdrawn.

I wouldn't go that far. The 'name' parameter, I would expect, would be
a constant. However, this strikes me as encouraging some really
inefficient code, like iterating over all the rows in a table with N+1
queries (one to get the length, then a separate query for each row).
Plus, use of limit without order by is not guaranteed (although since
this is specific to MySQL, it's unlikely you'll run into trouble - but
PostgreSQL, with its MVCC storage system, frequently reorders rows in
a table).

As a general rule, I don't like SQL builders. I'd prefer to directly
embed SQL statements in my code. Although sometimes a class can
helpfully manage some things, it's seldom worth having something that
tries to pretend a table is iterable in that sort of way.

ChrisA

[Chris Angelico]

On Thu, Sep 6, 2012 at 12:34 AM, Chris Angelico <ros...@gmail.com> wrote:
> However, this strikes me as encouraging some really
> inefficient code, like iterating over all the rows in a table with N+1
> queries (one to get the length, then a separate query for each row).

Huh. And then I scroll down, and that's precisely what he's doing. Do
you understand why this is a bad thing to do?

ChrisA

[Ian Kelly]

On Wed, Sep 5, 2012 at 8:34 AM, Chris Angelico <ros...@gmail.com> wrote:
> I wouldn't go that far. The 'name' parameter, I would expect, would be
> a constant.

The 'item' parameter, though, is probably not a constant, and it's
interpolated just the same.

> However, this strikes me as encouraging some really
> inefficient code, like iterating over all the rows in a table with N+1
> queries (one to get the length, then a separate query for each row).

> Plus, use of limit without order by is not guaranteed (although since
> this is specific to MySQL, it's unlikely you'll run into trouble - but
> PostgreSQL, with its MVCC storage system, frequently reorders rows in
> a table).

The lack of an ORDER BY is the least of the problems with that SQL.
He's also using LIMIT without OFFSET, so the only thing that the
'item' argument changes is how many rows are returned (all but one of
which are ignored), not which one is actually fetched.

It's a bit sad that these are touted as answers to interview
questions. I wouldn't hire anybody who gave answers like these.

[Chris Angelico]

On Thu, Sep 6, 2012 at 1:22 AM, Ian Kelly <ian.g...@gmail.com> wrote:
> The lack of an ORDER BY is the least of the problems with that SQL.
> He's also using LIMIT without OFFSET, so the only thing that the
> 'item' argument changes is how many rows are returned (all but one of
> which are ignored), not which one is actually fetched.

No, he's using the two-arg form of LIMIT.

> It's a bit sad that these are touted as answers to interview
> questions. I wouldn't hire anybody who gave answers like these.

The code does not work as posted; there are args missing from the
INSERT example, for, uhh, example. It makes it hard to evaluate the
quality of the code, in some places. I'm not sure what these posts are
supposed to be, but I hope they're not being held up as model answers
to interview questions. For a start, I can't find any sort of clear
questions.

Or is the code itself the question and "How would you improve this"?

ChrisA

[Dave Angel]

Skip ahead to about page 13 to get more traditional questions, There,
many of the simplest questions have invalid answers, or confusing
explanations.

For example, on page 15, the question that says " ... if a list of words
is empty..." uses a="" as its source data to test with.

The first question on page 16 forgets to construct a loop, thus
processing only the first line in the file.

page 18 introduces new syntax to the language:
print n+=1 ##will work

and reassures us in the comment that it will work !!

page 18 also claims that values are passed to function by value.





--

DaveA

[Roy Smith]

In article <d4e47e64-91d3-4b9f...@googlegroups.com>,

charvi...@gmail.com wrote:

> Hi Guys,
>
> Finally I have decided to put best interview question and answers.
>
> Please visit http://www.f2finterview.com/web/CorePython/ for core python and
> http://www.f2finterview.com/web/PythonAdvanced/ for advanced python

I was going to comment on some of the specific items, but I got hung up
being unable to copy-paste text from your site so I could quote it.
That's usually done with some variation on "user-select: none" in the
CSS, but I didn't see that. I'm assuming it's some javascript trickery.

Why do you do that? It degrades the usability of the site, and doesn't
provide any real protection against people stealing content.

[Roy Smith]

In article <mailman.245.13468586...@python.org>,

Ian Kelly <ian.g...@gmail.com> wrote:

> It's a bit sad that these are touted as answers to interview
> questions. I wouldn't hire anybody who gave answers like these.

Over time, I've become convinced that most interview questions are crap.
The best programming interview questions always start with, "write a
program ...".

[Ian Kelly]

On Wed, Sep 5, 2012 at 9:34 AM, Chris Angelico <ros...@gmail.com> wrote:
> On Thu, Sep 6, 2012 at 1:22 AM, Ian Kelly <ian.g...@gmail.com> wrote:
>> The lack of an ORDER BY is the least of the problems with that SQL.
>> He's also using LIMIT without OFFSET, so the only thing that the
>> 'item' argument changes is how many rows are returned (all but one of
>> which are ignored), not which one is actually fetched.
>
> No, he's using the two-arg form of LIMIT.

My mistake. I didn't even know there was a two-arg form of LIMIT.
Must be a MySQL thing. :-)

Cheers,
Ian

[Roy Smith]

In article <mailman.255.13468632...@python.org>,

Ian Kelly <ian.g...@gmail.com> wrote:

> My mistake. I didn't even know there was a two-arg form of LIMIT.
> Must be a MySQL thing. :-)

What are you talking about? SQL is an ISO Standard. Therefore, all
implementations work the same way. Didn't you get the memo?

[Chris Angelico]

On Thu, Sep 6, 2012 at 2:40 AM, Ian Kelly <ian.g...@gmail.com> wrote:
> On Wed, Sep 5, 2012 at 9:34 AM, Chris Angelico <ros...@gmail.com> wrote:
>> On Thu, Sep 6, 2012 at 1:22 AM, Ian Kelly <ian.g...@gmail.com> wrote:
>>> The lack of an ORDER BY is the least of the problems with that SQL.
>>> He's also using LIMIT without OFFSET, so the only thing that the
>>> 'item' argument changes is how many rows are returned (all but one of
>>> which are ignored), not which one is actually fetched.
>>
>> No, he's using the two-arg form of LIMIT.
>

> My mistake. I didn't even know there was a two-arg form of LIMIT.
> Must be a MySQL thing. :-)

Yeah, it's not something I've used, but when my current job started,
we were using MySQL and I used to eyeball the logs to see what queries
were performing most suboptimally. (There were some pretty egregious
ones. Most memorable was rewriting a TEXT field several times a second
with several KB of PHP serialized array with status/statistical
information. Structured information, yes. Stored as a clob.) My first
databasing experience was DB2, with the uber-verbose "FETCH FIRST n
ROW[S] ONLY", but now I'm happily on Postgres.

Everyone who wants to use LIMIT without ORDER BY should try their code
on Postgres. You'll quickly discover the problem.

ChrisA

[Kushal Kumaran]

On Wed, Sep 5, 2012 at 3:11 PM, Stephen Anto <charvi...@gmail.com> wrote:

> On Wed, Sep 5, 2012 at 2:10 PM, Kushal Kumaran
> <kushal.kum...@gmail.com> wrote:
>>
>> On Wed, Sep 5, 2012 at 12:20 PM, <charvi...@gmail.com> wrote:

>> > On Tuesday, October 30, 2007 11:44:01 PM UTC+5:30, Krypto wrote:
>> >> Hi,
>> >>
>> >> I have used Python for a couple of projects last year and I found it
>> >> extremely useful. I could write two middle size projects in 2-3 months
>> >> (part time). Right now I am a bit rusty and trying to catch up again
>> >> with Python.
>> >>
>> >> I am now appearing for Job Interviews these days and I am wondering if
>> >> anybody of you appeared for a Python Interview. Can you please share
>> >> the questions you were asked. That will be great help to me.
>> >>
>> >
>> > Finally I have decided to put best interview question and answers.
>> >
>> > Please visit http://www.f2finterview.com/web/CorePython/ for core python
>> > and http://www.f2finterview.com/web/PythonAdvanced/ for advanced python
>> >
>>

>> As I see from a quick glance, several of your answers seem to be
>> copied from the python faq at http://docs.python.org/faq/. The
>> copyright notice for the python.org website seems to be at
>> http://docs.python.org/copyright.html. Do you have the Python
>> Software Foundation's permission to copy large chunks of the
>> python.org website and claim it as your own content?
>>
>

> Thank you for your information, I really sorry for this, if possible could
> you note which are the questions taken from faq, we will remove it as early
> as possible.
>

Since your website's Terms of Use state that you own the content, it
is your problem to make sure whatever's there is there legally.

Here's one example, though, to get you started: See the entry "Why
can’t I use an assignment in an expression?" at
http://docs.python.org/faq/design.html, and compare with the content
under the identical heading on page 3 of the CorePython section.

--
regards,
kushal

[MRAB]

On 06/09/2012 05:24, Kushal Kumaran wrote:
> On Wed, Sep 5, 2012 at 3:11 PM, Stephen Anto <charvi...@gmail.com> wrote:
>> On Wed, Sep 5, 2012 at 2:10 PM, Kushal Kumaran
>> <kushal.kum...@gmail.com> wrote:
>>>
>>> On Wed, Sep 5, 2012 at 12:20 PM, <charvi...@gmail.com> wrote:

>>> > On Tuesday, October 30, 2007 11:44:01 PM UTC+5:30, Krypto wrote:
>>> >> Hi,
>>> >>
>>> >> I have used Python for a couple of projects last year and I found it
>>> >> extremely useful. I could write two middle size projects in 2-3 months
>>> >> (part time). Right now I am a bit rusty and trying to catch up again
>>> >> with Python.
>>> >>
>>> >> I am now appearing for Job Interviews these days and I am wondering if
>>> >> anybody of you appeared for a Python Interview. Can you please share
>>> >> the questions you were asked. That will be great help to me.
>>> >>
>>> >
>>> > Finally I have decided to put best interview question and answers.
>>> >
>>> > Please visit http://www.f2finterview.com/web/CorePython/ for core python
>>> > and http://www.f2finterview.com/web/PythonAdvanced/ for advanced python
>>> >
>>>

>>> As I see from a quick glance, several of your answers seem to be
>>> copied from the python faq at http://docs.python.org/faq/. The
>>> copyright notice for the python.org website seems to be at
>>> http://docs.python.org/copyright.html. Do you have the Python
>>> Software Foundation's permission to copy large chunks of the
>>> python.org website and claim it as your own content?
>>>
>>
>> Thank you for your information, I really sorry for this, if possible could
>> you note which are the questions taken from faq, we will remove it as early
>> as possible.
>>
>
> Since your website's Terms of Use state that you own the content, it
> is your problem to make sure whatever's there is there legally.
>
> Here's one example, though, to get you started: See the entry "Why
> can’t I use an assignment in an expression?" at
> http://docs.python.org/faq/design.html, and compare with the content
> under the identical heading on page 3 of the CorePython section.
>

The section below it comes from here:

http://svn.effbot.org/public/stuff/sandbox/pyfaq/is-there-a-tool-to-help-find-bugs-or-perform-static-analysis.xml

[chinja...@gmail.com]

Hi I had one doubt.. I know very little bit of python .I wanted to know when to use list,tuple,dictionary and set? Please reply me asap

thanks

[Steven D'Aprano]

On Sat, 17 Nov 2012 10:01:01 -0800, chinjannisha wrote:

> Hi I had one doubt.. I know very little bit of python .I wanted to know
> when to use list,tuple,dictionary and set? Please reply me asap

Use a list when you want a list of items that should all be treated the
same way:

list_of_numbers = [1, 3, 5.1, 2, 6.5]

total = sum(list_of_numbers)


or when you need a collection of items where the order they are in is
important:

winners = ['george', 'susan', 'henry'] # 1st, 2nd, 3rd place

print('The winner is:', winners[0])


Use a tuple when you want a collection of items that mean different
things, a bit like a C struct or Pascal record:

a = (23, "henry", True, 'engineer')
b = (42, "natalie", False, 'manager')
c = (17, "george", True, 'student')


Use a dict when you need a collection of key:value mappings:

config = {'name': 'fred', 'pagesize': 'A4', 'verbose': True, 'size': 18}
address = {'fred': 'fr...@example.com', 'sally': 'sally...@example.com'}

if config['verbose']:
print('sending email...')
send_email_to(address['fred'], "Subject: Hello")


Use a set when you want to represent a collection of items and the order
is not important:

failed = {'fred', 'tom', 'sally'} # set literal syntax in Python 3 only
# use set(['fred', 'tom', 'sally']) in Python 2

if 'george' in failed:
print('George, you have failed!')



--
Steven

[Roy Smith]

In article <50a8acdc$0$29978$c3e8da3$5496...@news.astraweb.com>,

Steven D'Aprano <steve+comp....@pearwood.info> wrote:

> Use a list when you want a list of items that should all be treated the

> same way [...] or when you need a collection of items where the order they are in is
> important:
>

> Use a tuple when you want a collection of items that mean different
> things, a bit like a C struct or Pascal record:

That is certainly the right answer according to the One True Church Of
Pythonic Orthodoxy And Theoretical Correctness. But, let me give an
alternative answer which works for The Unwashed Masses Who Live In The
Trenches And Write Python Code For A Living:

Use a list when you need an ordered collection which is mutable (i.e.
can be altered after being created). Use a tuple when you need an
immutable list (such as for a dictionary key).

[Steven D'Aprano]

On Sun, 18 Nov 2012 08:53:25 -0500, Roy Smith wrote:

> In article <50a8acdc$0$29978$c3e8da3$5496...@news.astraweb.com>,
> Steven D'Aprano <steve+comp....@pearwood.info> wrote:
>
>> Use a list when you want a list of items that should all be treated the
>> same way [...] or when you need a collection of items where the order
>> they are in is important:
>>
>> Use a tuple when you want a collection of items that mean different
>> things, a bit like a C struct or Pascal record:
>
> That is certainly the right answer according to the One True Church Of
> Pythonic Orthodoxy And Theoretical Correctness.

Oh I'm sorry, did something I say suggest that the couple of examples I
gave are the *only* acceptable uses? My apologies for not giving an
exhaustive list of every possible use of lists and tuples, I'll be sure
to punish myself severely for the lapse.

> But, let me give an
> alternative answer which works for The Unwashed Masses Who Live In The
> Trenches And Write Python Code For A Living:
>
> Use a list when you need an ordered collection which is mutable (i.e.
> can be altered after being created). Use a tuple when you need an
> immutable list (such as for a dictionary key).

I keep hearing about this last one, but I wonder... who *actually* does
this? I've created many, many lists over the years -- lists of names,
lists of phone numbers, lists of directory search paths, all sorts of
things. I've never needed to use one as a dictionary key.

Under what sort of circumstances would somebody want to take a mutable
list of data, say a list of email addresses, freeze it into a known
state, and use that frozen state as a key in a dict? What would be the
point? Even if there was some meaningful reason to look up "this list of
12000 email addresses" as a single key, it is going to get out of sync
with the actual mutable list.

Sure, I have built a collection of items as a list, because lists are
mutable, then frozen it into a tuple, and *thrown the list away*, then
used the tuple as a key. But that's not the same thing, the intent is
different. In my case, the data was never intended to be a list, it was
always intended to be a fixed record-like collection, the use of list was
as a temporary data structure used for construction. A bit like the idiom
of ''.join(some_list).

But I can't think of any meaningful, non-contrived example where I might
want an actual mutable list of values as a dict key.


--
Steven

[D'Arcy J.M. Cain]

On 18 Nov 2012 16:50:52 GMT

Steven D'Aprano <steve+comp....@pearwood.info> wrote:
> On Sun, 18 Nov 2012 08:53:25 -0500, Roy Smith wrote:
>> > Use a list when you need an ordered collection which is mutable
> > (i.e. can be altered after being created). Use a tuple when you
> > need an immutable list (such as for a dictionary key).
>
> I keep hearing about this last one, but I wonder... who *actually*
> does this? I've created many, many lists over the years -- lists of
> names, lists of phone numbers, lists of directory search paths, all
> sorts of things. I've never needed to use one as a dictionary key.

Well, as long as *you* never needed it then...

CellBlock = 9 # There's a riot going on...
Cell = 17
Bunk = "top"

Prisoner = {(CellBlock, Cell, Bunk): "Bernie Madoff"}

--
D'Arcy J.M. Cain <da...@druid.net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.
IM: da...@Vex.Net

[Roy Smith]

In article <50a911ec$0$29978$c3e8da3$5496...@news.astraweb.com>,

Steven D'Aprano <steve+comp....@pearwood.info> wrote:

> Oh I'm sorry, did something I say suggest that the couple of examples I
> gave are the *only* acceptable uses? My apologies for not giving an
> exhaustive list of every possible use of lists and tuples, I'll be sure
> to punish myself severely for the lapse.

Hmmm. I didn't mean any offense. I was just pointing out that what's
true in theory and what's true in practice aren't always the same.

> Under what sort of circumstances would somebody want to take a mutable
> list of data, say a list of email addresses, freeze it into a known
> state, and use that frozen state as a key in a dict?

I've got a script which trolls our log files looking for python stack
dumps. For each dump it finds, it computes a signature (basically, a
call sequence which led to the exception) and uses this signature as a
dictionary key. Here's the relevant code (abstracted slightly for
readability):

def main(args):
crashes = {}
[...]
for line in open(log_file):
if does_not_look_like_a_stack_dump(line):
continue
lines = traceback_helper.unfold(line)
header, stack = traceback_helper.extract_stack(lines)
signature = tuple(stack)
if signature in crashes:
count, header = crashes[signature]
crashes[signature] = (count + 1, header)
else:
crashes[signature] = (1, header)

You can find traceback_helper at
https://bitbucket.org/roysmith/python-tools/src/4f8118d175ed/logs/traceba
ck_helper.py

The stack that's returned is a list. It's inherently a list, per the
classic definition:

* It's variable length. Different stacks have different depths.

* It's homogeneous. There's nothing particularly significant about each
entry other than it's the next one in the stack.

* It's mutable. I can build it up one item at a time as I discover them.

* It's ordered. f1(f2()) is not the same as f2(f1()).

But, to use it as a dictionary key, I need to make it into a tuple,
since keys have to be immutable.

[Chris Angelico]

On Mon, Nov 19, 2012 at 4:16 AM, D'Arcy J.M. Cain <da...@druid.net> wrote:
> On 18 Nov 2012 16:50:52 GMT

> Steven D'Aprano <steve+comp....@pearwood.info> wrote:
>> On Sun, 18 Nov 2012 08:53:25 -0500, Roy Smith wrote:
>>> > Use a list when you need an ordered collection which is mutable
>> > (i.e. can be altered after being created). Use a tuple when you
>> > need an immutable list (such as for a dictionary key).
>>
>> I keep hearing about this last one, but I wonder... who *actually*
>> does this? I've created many, many lists over the years -- lists of
>> names, lists of phone numbers, lists of directory search paths, all
>> sorts of things. I've never needed to use one as a dictionary key.
>

> Well, as long as *you* never needed it then...
>
> CellBlock = 9 # There's a riot going on...
> Cell = 17
> Bunk = "top"
>
> Prisoner = {(CellBlock, Cell, Bunk): "Bernie Madoff"}

That's a structure, not a list. Every key will consist of precisely
three values: two integers and one keyword string. Already covered by
a previous example.

ChrisA

[Steven D'Aprano]

Er, no, it's inherently a blob of multiple text lines. Sure, you've built
it a line at a time by using a list, but I've already covered that case.
Once you've identified a stack, you never append to it, sort it, delete
lines in the middle of it... none of these list operations are meaningful
for a Python stack trace. The stack becomes a fixed string, and not just
because you use it as a dict key, but because inherently it counts as a
single, immutable blob of lines.

A tuple of individual lines is one reasonable data structure for a blob
of lines. Another would be a single string:

signature = '\n'.join(stack)

Depending on what you plan to do with the signatures, one or the other
implementation might be better. I'm sure that there are other data
structures as well.

> * It's variable length. Different stacks have different depths.

Once complete, the stack trace is fixed length, but that fixed length is
different from one stack to the next. Deleting a line would make it
incomplete, and adding a line would make it invalid.

> * It's homogeneous. There's nothing particularly significant about each
> entry other than it's the next one in the stack.
>
> * It's mutable. I can build it up one item at a time as I discover
> them.

The complete stack trace is inhomogeneous and immutable. I've already
covered immutability above: removing, adding or moving lines will
invalidate the stack trace. Inhomogeneity comes from the structure of a
stack trace. The mere fact that each line is a string does not mean that
any two lines are equivalent. Different lines represent different things.

Traceback (most recent call last):
File "./prattle.py", line 873, in select
selection = self.do_callback(cb, response)
File "./prattle.py", line 787, in do_callback
raise callback
ValueError: what do you mean?

is a valid stack. But:

Traceback (most recent call last):
raise callback
selection = self.do_callback(cb, response)
File "./prattle.py", line 787, in do_callback
ValueError: what do you mean?
File "./prattle.py", line 873, in select

is not. A stack trace has structure. The equivalent here is the
difference between:

ages = [23, 42, 19, 67, # age, age, age, age
17, 94, 32, 51, # ...
]

values = [23, 1972, 1, 34500, # age, year, number of children, income
35, 1985, 0, 67900, # age, year, number of children, income
]

A stack trace is closer to the second example than the first: each item
may be the same type, but the items don't represent the same *kind of
thing*.


You could make a stack trace homogeneous with a little work:

- drop the Traceback line and the final exception line;
- parse the File lines to extract the useful fields;
- combine them with the source code.

Now you have a blob of homogeneous records, here shown as lines of text
with ! as field separator:

./prattle.py ! 873 ! select ! selection = self.do_callback(cb, response)
./prattle.py ! 787 ! do_callback ! raise callback

But there's really nothing you can do about the immutability. There isn't
any meaningful reason why you might want to take a complete stack trace
and add or delete lines from it.


--
Steven

[Roy Smith]

In article <50a97de0$0$29983$c3e8da3$5496...@news.astraweb.com>,

Steven D'Aprano <steve+comp....@pearwood.info> wrote:

> > The stack that's returned is a list. It's inherently a list, per the
> > classic definition:
>
> Er, no, it's inherently a blob of multiple text lines.

No, it's a list that looks like (taken from the doc string of the code I
referenced):

[('/usr/lib/.../base.py', 'get_response'),
('/home/songza/.../views.py', 'song_info'),
('/home/songza.../api.py', 'get_song'),
('/home/songza/.../api.py', 'api')]

[it doesn't really have ...'s in the paths; I just elided some text to
make it easier to read]

> > * It's homogeneous. There's nothing particularly significant about each
> > entry other than it's the next one in the stack.
>

> The complete stack trace is inhomogeneous and immutable. I've already
> covered immutability above: removing, adding or moving lines will
> invalidate the stack trace. Inhomogeneity comes from the structure of a
> stack trace. The mere fact that each line is a string does not mean that
> any two lines are equivalent. Different lines represent different things.

No. Each entry in the list represents a source file and a function
name. They're all the same "shape". You could remove one or add
another one, or shuffle the order, and you would have something which
was syntactically correct and semantically meaningful (even if it didn't
represent an actual code path.

> - drop the Traceback line and the final exception line;
> - parse the File lines to extract the useful fields;
> - combine them with the source code.

You mean, kind of like the code I cited does? :-)

I think we're going to have to let this be. You obviously have your
concept of what a tuple is and what a list is. I disagree. I don't
think either of us is right or wrong, we just have different ways of
thinking about things.

You come at it from a theoretical point of view. You think of each type
as an embodiment of certain concepts ("it represents a fixed-length
heterogenous sequence"). Your thinking is driven by what each type was
intended to be used for.

I come at it from a practical point of view. To me, each type is a
collection of methods. I have certain operations I need to perform. I
pick the type which offers those operations. If the set of operations I
need to perform (in this case, {append, hash}) don't exist in a single
type, I'm forced to use both types and convert from one to the other as
needed.

The theorist understands that a chisel and a screwdriver were intended
for different purposes, but the pragmatist gets the paint can open.

[Chris Angelico]

On Mon, Nov 19, 2012 at 1:09 PM, Roy Smith <r...@panix.com> wrote:
> The theorist understands that a chisel and a screwdriver were intended
> for different purposes, but the pragmatist gets the paint can open.

A good tool can always be used in ways its inventor never intended -
and it will function as its user expects.

$ some_program | egrep --color=always '(ERROR|^)'

will highlight the word ERROR in red anywhere it appears in the
program's output, while maintaining all other lines without color. Not
normal use of grep, to be sure, but quite functional.

A tuple may have been intended to be a record, a struct, whatever, but
it is what it is, and I'll use one any time it's the best tool for the
job. Maybe its immutability is critical; or maybe it's just the most
convenient syntax and all I care about is that it be iterable.

But when I'm explaining grep to someone, I'll describe it as a filter
that keeps only some lines from the original, and when I describe a
tuple, I'll point out that it's immutable and (potentially) hashable.
The obvious first, the unobvious later.

ChrisA

[Mark Lawrence]

On 19/11/2012 02:09, Roy Smith wrote:
>
> The theorist understands that a chisel and a screwdriver were intended
> for different purposes, but the pragmatist gets the paint can open.
>

To throw a chiseldriver into the works, IIRC a tuple is way faster to
create but accessing a list is much faster. The obvious snag is that
may have been Python 2.7 whereas 3.3 is completely different. Sorry but
I'm currently wearing my XXXL size Lazy Bone Idle Hat so have no figures
to back my probably incorrect memory up, anyone know anything about this?

--
Cheers.

Mark Lawrence.

[Ian Kelly]

On Sun, Nov 18, 2012 at 7:42 PM, Mark Lawrence <bream...@yahoo.co.uk> wrote:
> To throw a chiseldriver into the works, IIRC a tuple is way faster to create
> but accessing a list is much faster. The obvious snag is that may have been
> Python 2.7 whereas 3.3 is completely different. Sorry but I'm currently
> wearing my XXXL size Lazy Bone Idle Hat so have no figures to back my
> probably incorrect memory up, anyone know anything about this?

It's not been my experience with Python 2.7 that list access is faster
than tuple access. Tuples are as fast as or faster than lists, pretty
much universally. They seem to have closed the gap a bit in
Python 3.3, though, as the following timings show. For one-shot
construction, tuples seem to be more efficient for short sequences,
but then lists win for longer sequences, although not by much. Of
course, lists are always going to be much slower if you build them up
with appends and extends.

C:\>python -m timeit -s "x = range(10)" "tuple(x)"
1000000 loops, best of 3: 0.773 usec per loop

C:\>python -m timeit -s "x = range(10)" "list(x)"
1000000 loops, best of 3: 0.879 usec per loop

C:\>python -m timeit -s "x = range(100)" "tuple(x)"
100000 loops, best of 3: 2.88 usec per loop

C:\>python -m timeit -s "x = range(100)" "list(x)"
100000 loops, best of 3: 2.63 usec per loop

C:\>python -m timeit -s "x = range(1000)" "tuple(x)"
10000 loops, best of 3: 37.4 usec per loop

C:\>python -m timeit -s "x = range(1000)" "list(x)"
10000 loops, best of 3: 36.2 usec per loop

C:\>python -m timeit -s "x = range(10000)" "tuple(x)"
1000 loops, best of 3: 418 usec per loop

C:\>python -m timeit -s "x = range(10000)" "list(x)"
1000 loops, best of 3: 410 usec per loop


For iteration, tuples are consistently 7-8% faster.


C:\>python -m timeit -s "x = tuple(range(10))" "for i in x: pass"
1000000 loops, best of 3: 0.467 usec per loop

C:\>python -m timeit -s "x = list(range(10))" "for i in x: pass"
1000000 loops, best of 3: 0.498 usec per loop

C:\>python -m timeit -s "x = tuple(range(100))" "for i in x: pass"
100000 loops, best of 3: 3.31 usec per loop

C:\>python -m timeit -s "x = list(range(100))" "for i in x: pass"
100000 loops, best of 3: 3.56 usec per loop

C:\>python -m timeit -s "x = tuple(range(1000))" "for i in x: pass"
10000 loops, best of 3: 31.6 usec per loop

C:\>python -m timeit -s "x = list(range(1000))" "for i in x: pass"
10000 loops, best of 3: 34.3 usec per loop

C:\>python -m timeit -s "x = tuple(range(10000))" "for i in x: pass"
1000 loops, best of 3: 318 usec per loop

C:\>python -m timeit -s "x = list(range(10000))" "for i in x: pass"
1000 loops, best of 3: 341 usec per loop


For direct item access, tuples seem to be about 2-3% faster.


C:\>python -m timeit -s "import operator as o; x = tuple(range(10)); g
= o.itemgetter(*range(len(x)))" "g(x)"
1000000 loops, best of 3: 0.67 usec per loop

C:\>python -m timeit -s "import operator as o; x = list(range(10)); g
= o.itemgetter(*range(len(x)))" "g(x)"
1000000 loops, best of 3: 0.674 usec per loop

C:\>python -m timeit -s "import operator as o; x = tuple(range(100));
g = o.itemgetter(*range(len(x)))" "g(x)"
100000 loops, best of 3: 4.52 usec per loop

C:\>python -m timeit -s "import operator as o; x = list(range(100)); g
= o.itemgetter(*range(len(x)))" "g(x)"
100000 loops, best of 3: 4.65 usec per loop

C:\>python -m timeit -s "import operator as o; x = tuple(range(1000));
g = o.itemgetter(*range(len(x)))" "g(x)"
10000 loops, best of 3: 43.2 usec per loop

C:\>python -m timeit -s "import operator as o; x = list(range(1000));
g = o.itemgetter(*range(len(x)))" "g(x)"
10000 loops, best of 3: 43.7 usec per loop

C:\>python -m timeit -s "import operator as o; x =
tuple(range(10000)); g = o.itemgetter(*range(len(x)))" "g(x)"
1000 loops, best of 3: 422 usec per loop

C:\>python -m timeit -s "import operator as o; x = list(range(10000));
g = o.itemgetter(*range(len(x)))" "g(x)"
1000 loops, best of 3: 447 usec per loop

[Steven D'Aprano]

On Sun, 18 Nov 2012 21:09:36 -0500, Roy Smith wrote:

> In article <50a97de0$0$29983$c3e8da3$5496...@news.astraweb.com>,
> Steven D'Aprano <steve+comp....@pearwood.info> wrote:
>
>
>> > The stack that's returned is a list. It's inherently a list, per the
>> > classic definition:
>>
>> Er, no, it's inherently a blob of multiple text lines.
>
> No, it's a list that looks like (taken from the doc string of the code I
> referenced):
>
> [('/usr/lib/.../base.py', 'get_response'),
> ('/home/songza/.../views.py', 'song_info'),
> ('/home/songza.../api.py', 'get_song'),
> ('/home/songza/.../api.py', 'api')]
>
> [it doesn't really have ...'s in the paths; I just elided some text to
> make it easier to read]

I see. It wasn't clear from your earlier description that the items had
been post-processed from collections of raw log lines to fixed records.
But it doesn't actually change my analysis any. See below.

By the way, based on the sample data you show, your script is possibly
broken. You don't record either the line number that raises, or the
exception raised, so your script doesn't differentiate between different
errors that happen to occur with similar stack traces. (I say "possibly"
broken because I don't know what your requirements are. Maybe your
requirements are sufficiently wide that you don't care that distinct
failures are counted together.)

E.g. these three stack traces will probably generate the same fixed
record, even though the errors are distinct:

#1

Traceback (most recent call last):

File "./spam.py", line 20, in select
selection = func(a, b)
File "./spam.py", line 60, in func
return 1/x
ZeroDivisionError: float division


#2

Traceback (most recent call last):

File "./spam.py", line 20, in select
selection = func(a, b)
File "./spam.py", line 60, in func
return 1/x
TypeError: unsupported operand type(s) for /: 'int' and 'NoneType'


#3

Traceback (most recent call last):

File "./spam.py", line 20, in select
selection = func(a, b)
File "./spam.py", line 55, in func
y = 1/(a + b)
ZeroDivisionError: float division


Maybe that's okay for your application, but it strikes me as odd that you
do distinguish *some* distinct errors in the same function, but not
others.

>> > * It's homogeneous. There's nothing particularly significant about
>> > each entry other than it's the next one in the stack.
>>
>> The complete stack trace is inhomogeneous and immutable. I've already
>> covered immutability above: removing, adding or moving lines will
>> invalidate the stack trace. Inhomogeneity comes from the structure of a
>> stack trace. The mere fact that each line is a string does not mean
>> that any two lines are equivalent. Different lines represent different
>> things.
>
> No. Each entry in the list represents a source file and a function
> name. They're all the same "shape". You could remove one or add
> another one, or shuffle the order, and you would have something which
> was syntactically correct and semantically meaningful (even if it didn't
> represent an actual code path.

If you remove/add/shuffle lines in the stack, you no longer have the same
stack. Take the example you gave before:

stack1 = [('/usr/lib/.../base.py', 'get_response'),

('/home/songza/.../views.py', 'song_info'),
('/home/songza.../api.py', 'get_song'),
('/home/songza/.../api.py', 'api')
]

Here's a different stack trace, representing a different code path, which
as you say is syntactically correct and semantically meaningful:

stack2 = [('/home/songza/.../api.py', 'api'),

('/home/songza.../api.py', 'get_song'),
('/home/songza/.../views.py', 'song_info'),

('/usr/lib/.../base.py', 'get_response')
]

Since they are different stacks, they are treated as different keys:

data = {stack1: 11, stack2: 22}

Do you agree that this is what your application expects? Different stack
traces are different keys, associated with different values.

I claim this only makes sense if you treat the stacks as inherently
immutable. Never mind Python's limitation. Let's pretend we were running
this code under some other language, NeoPython, which allowed mutable
keys.

You claim that stacks are *inherently mutable*. So I should be able to do
this:

stack1.sort() # it's the *same stack*, all I've done is mutate it
print data[stack1]

and expect to see "11" printed. I am looking at the same key, right? So I
certainly don't expect to see the value associated with a completely
different key.

But wait a minute... after sorting, stack1 and stack2 now are equal. I
could just as easily expect to see "22" printed.

I thought we had just agreed that stack1 and stack2 are *different* keys.
Of course they are different. They represent different code paths. But
after sorting stack1, it looks exactly like stack2. It looks like a
different code path. It *lies* -- it no longer represents the code path
that it actually represents, instead it looks like a *different* code
path.

I then generate another stack:

stack3 = [('/home/songza/.../api.py', 'api'),

('/home/songza.../api.py', 'get_song'),
('/home/songza/.../views.py', 'song_info'),

('/usr/lib/.../base.py', 'get_response')
]

should data[stack3] return 11 (it has the same value as stack1) or 22 (it
has the same value as stack2)? Or possibly 33? Or raise KeyError?

Treating stacks in this context as mutable is *incoherent*. It is nice
and convenient to be able to build up a stack trace using a mutable list,
you won't get an argument from me about that, but that can only be
considered a temporary data structure used to build the data structure
you actually care about, which is fixed.

That brings it back to my question: your application is not a counter-
example to my question about using lists as keys, because your data is
not inherently list-like. It is inherently tuple-like, you just build it
using a temporary list. That's perfectly fine, by the way, I do the same
thing.

As you say, the order of the lines in the stack trace is significant. You
cannot expect to mutate the stack and move lines around and treat it as
the same stack. If you move the lines about, it represents a different
stack. That is fundamentally different from the normal use of a list,
where you do expect to be able to move lines about and still have it
count as "the same list".

> I think we're going to have to let this be. You obviously have your
> concept of what a tuple is and what a list is. I disagree.

I think a tuple is an immutable sequence of items, and a list is a
mutable sequence of items.

> I don't
> think either of us is right or wrong, we just have different ways of
> thinking about things.
>
> You come at it from a theoretical point of view.

I certainly do not. My position here is imminently practical. The
alternative, the mutability of keys, is simply incoherent.

> You think of each type
> as an embodiment of certain concepts ("it represents a fixed-length
> heterogenous sequence"). Your thinking is driven by what each type was
> intended to be used for.

Not even close. My thinking is driven by the things each data structure
needs to do. See below.

> I come at it from a practical point of view. To me, each type is a
> collection of methods. I have certain operations I need to perform. I
> pick the type which offers those operations. If the set of operations I
> need to perform (in this case, {append, hash}) don't exist in a single
> type, I'm forced to use both types and convert from one to the other as
> needed.

I don't see that as a problem. Converting from one type to another is
exactly the sort of thing I described in my earlier question.

In your application, you build up a collection of code lines that
represent a stack trace. Here's that example from your own documentation
again:

[('/usr/lib/.../base.py', 'get_response'),
('/home/songza/.../views.py', 'song_info'),
('/home/songza.../api.py', 'get_song'),
('/home/songza/.../api.py', 'api')]

What are the sorts of things I might meaningfully want to do with this
*complete* stack trace?

Add extra lines to it? No. If I needed to add extra lines, it wouldn't be
complete.

Delete lines? Certainly not, that would change the code path it claims to
represent to a code path it doesn't represent.

Sort the list? Reverse it? Heavens no.

If you look at the available list methods, *not one* of the mutating
methods is appropriate to a completed stack trace object. *None* of the
mutator list methods are appropriate once the stack trace object is
complete, and using them would be counter-productive.

If you believe different, then please tell me what mutations your code
actually performs after the stack trace object is completed. In the code
you showed, you throw the list away after turning it into a tuple.

If the object represents a "list of code lines", in the sense of a
mutable Python list rather than a mere sequence, then why don't you use
any list methods on it?

The append method is useful during construction, but that is all. After
the stack is complete, use of any mutator method would be a bug. In other
words, it ought to be immutable, and the use of a list ought to be buried
in the appropriate function as an internal implementation detail. The
public interface ought to be that of an immutable tuple of immutable
strings, because once you have finished building the object, it should
not be possible to mutate it.

This is hardly a theoretical viewpoint. The idea of treating data that
ought not be changed as immutable is borne out of bitter experience of
millions of man-hours tracking down hundreds of thousands of bugs.

(Admittedly not all of those bugs were *my* bugs. I'm talking the
collective experience of programmers over fifty years of coding.)



--
Steven

[Terry Reedy]

On 11/19/2012 1:01 AM, Ian Kelly wrote:

> than tuple access. Tuples are as fast as or faster than lists, pretty
> much universally. They seem to have closed the gap a bit in
> Python 3.3, though, as the following timings show. For one-shot
> construction, tuples seem to be more efficient for short sequences,
> but then lists win for longer sequences, although not by much. Of
> course, lists are always going to be much slower if you build them up
> with appends and extends.

Interesting results. But what system (hardware, os). These sorts of
times tend to vary with the system.

--
Terry Jan Reedy

[Roy Smith]

In article <50a9e5cf$0$21863$c3e8da3$7649...@news.astraweb.com>,

Steven D'Aprano <steve+comp....@pearwood.info> wrote:

> I see. It wasn't clear from your earlier description that the items had
> been post-processed from collections of raw log lines to fixed records.

Well, I did provide the code that does this.

> But it doesn't actually change my analysis any. See below.
>
> By the way, based on the sample data you show, your script is possibly
> broken. You don't record either the line number that raises, or the
> exception raised, so your script doesn't differentiate between different
> errors that happen to occur with similar stack traces.

You really might want to read the code I provided. Here's the reference
again:

https://bitbucket.org/roysmith/python-tools/src/4f8118d175ed/logs/traceba
ck_helper.py

The "header" referred to does indeed contain the exception raised. And
the line numbers are included. Here's a typical output stanza:

2012-11-19T00:00:15+00:00 web5 ˇ˛2012-11-19 00:00:15,831 [2712]:
songza-api IGPhwNU2SJ691cx8 4C0ABFA9-50A974E7-384995 W6D-HSO
173.145.137.54 songza.django.middleware ERROR process_exception() Path =
u'/api/1/station/1459775/next', Exception =
ValueError(u"<SequentialSongPicker: <Station 1459775: u'Old School
105.3'>>: no song ids for mp3",)
/home/songza/env/python/local/lib/python2.7/site-packages/django/core/han
dlers/base.py:111:get_response()
/home/songza/deploy/current/pyza/djapi/decorators.py:11:_wrapped_view_fun
c()
/home/songza/env/python/local/lib/python2.7/site-packages/django/views/de
corators/http.py:45:inner()
/home/songza/deploy/current/pyza/djapi/views.py:1659:station_next()
/home/songza/deploy/current/pyza/models/station.py:660:next_song()
/home/songza/deploy/current/pyza/lib/song_picker.py:327:pick()

> I say "possibly" broken because I don't know what your requirements are.

Our requirements are to scan the logs of a production site and filter
down the gobs and gobs of output (we produced 70 GB of log files
yesterday) into something small enough that a human can see what the
most common failures were. The tool I wrote does that.

The rest of this conversation is just silly. It's turning into getting
hit on the head lessons.

[Roy Smith]

OK, I've just read back over the whole thread. I'm really struggling to
understand what point you're trying to make. I started out by saying:

> Use a list when you need an ordered collection which is mutable (i.e.
> can be altered after being created). Use a tuple when you need an
> immutable list (such as for a dictionary key).

To which you obviously objected. So now you write:

> I think a tuple is an immutable sequence of items, and a list is a
> mutable sequence of items.

So how is that different from what I said? Is this whole argument
boiling down to your use of "immutable sequence" vs. my use of
"immutable list"?

[Ian Kelly]

On Mon, Nov 19, 2012 at 7:30 AM, Roy Smith <r...@panix.com> wrote:
> In article <50a9e5cf$0$21863$c3e8da3$7649...@news.astraweb.com>,
> Steven D'Aprano <steve+comp....@pearwood.info> wrote:
>>
>> By the way, based on the sample data you show, your script is possibly
>> broken. You don't record either the line number that raises, or the
>> exception raised, so your script doesn't differentiate between different
>> errors that happen to occur with similar stack traces.
>
> You really might want to read the code I provided. Here's the reference
> again:
>
> https://bitbucket.org/roysmith/python-tools/src/4f8118d175ed/logs/traceba
> ck_helper.py
>
> The "header" referred to does indeed contain the exception raised. And
> the line numbers are included. Here's a typical output stanza:

Yes, but the dict is still keyed on the traceback alone, and only the
first header for a particular traceback is stored. If two different
exceptions occur at the same line of code and sharing the same
traceback, the second exception would be counted as a second
occurrence of the first, effectively squashing any reporting of the
second exception.

[Terry Reedy]

On 11/19/2012 9:30 AM, Roy Smith wrote:

> Our requirements are to scan the logs of a production site and filter
> down the gobs and gobs of output (we produced 70 GB of log files
> yesterday) into something small enough that a human can see what the
> most common failures were. The tool I wrote does that.
>
> The rest of this conversation is just silly. It's turning into getting
> hit on the head lessons.

I agree. In early Python, tuples were more different from lists than
they are today. They did not have any (public) methods. Today, they have
.index and .count methods, which make little sense from the 'tuple is a
record' viewpoint. The addition of those methods redefined tuples as
read-only (and therefore hashable) sequences.

From the collections.abc doc
'''
Sequence | Sized, Iterable, Container |
__getitem__ __contains__, __iter__, __reversed__, index, and count
...
class collections.abc.Sequence
class collections.abc.MutableSequence
ABCs for read-only and mutable sequences.
'''
>>> from collections.abc import Sequence
>>> issubclass(tuple, Sequence)
True

--
Terry Jan Reedy

[Steven D'Aprano]

On Mon, 19 Nov 2012 09:30:54 -0500, Roy Smith wrote:

> In article <50a9e5cf$0$21863$c3e8da3$7649...@news.astraweb.com>,
> Steven D'Aprano <steve+comp....@pearwood.info> wrote:
>
>> I see. It wasn't clear from your earlier description that the items had
>> been post-processed from collections of raw log lines to fixed records.
>
> Well, I did provide the code that does this.

You did? When? [goes back and looks]

Oh, so you did. Oops.

By the way, your news client seems to be mangling long URLs, by splitting
them when they exceed the maximum line length. I didn't follow the link
you gave because it was mangled, and then forgot it even existed. Sorry
about that.


[...]

> You really might want to read the code I provided. Here's the reference
> again:
>
> https://bitbucket.org/roysmith/python-tools/src/4f8118d175ed/logs/
traceba
> ck_helper.py

And mangled again :)

> The "header" referred to does indeed contain the exception raised. And
> the line numbers are included. Here's a typical output stanza:

[snip]

Ian Kelly has picked up on what I'm trying to say. You might collect the
traceback in the "header", but it doesn't get used in the key, and each
time you find a repeated stack trace, you toss away whatever header you
just saw and keep the header you saw the first time.

[quote]

header, stack = traceback_helper.extract_stack(lines)
signature = tuple(stack)
if signature in crashes:
count, header = crashes[signature]
crashes[signature] = (count + 1, header)
else:
crashes[signature] = (1, header)

[end quote]


In general, it is an unsafe assumption that the actual exception raised
will be the same just because the stack trace is the same. So as I said,
if you have two *distinct* failures occurring in the same function (not
even necessarily on the same line), your code appears to treat them as
the same error. That seems odd to me, but if you have a good reason for
doing it that way, so be it.



--
Steven

[Steven D'Aprano]

Sheesh, of course not. Give me some credit.

I gave some examples of when somebody might use lists, tuples, sets and
dicts. Apparently I forgot a couple, and you responded with a sarcastic
comment about the "One True Church Of Pythonic Orthodoxy And Theoretical
Correctness" and gave a couple of additional examples.

Although I didn't come out and *explicitly* say "I agree" to your
examples, I actually did, with one proviso: your example of using an
"immutable list" as dict key. So I asked a question about that *specific*
use-case:

[quote]

Under what sort of circumstances would somebody want to take a mutable
list of data, say a list of email addresses, freeze it into a known state,

and use that frozen state as a key in a dict? What would be the point?
Even if there was some meaningful reason to look up "this list of 12000
email addresses" as a single key, it is going to get out of sync with the
actual mutable list.

[end quote]

Your reply was to give your stack trace script as an example. That's a
fine example as a use-case for a temporary list, and I've done similar
things dozens, hundreds of times myself. As I said:

[quote]

Sure, I have built a collection of items as a list, because lists are
mutable, then frozen it into a tuple, and *thrown the list away*, then
used the tuple as a key. But that's not the same thing, the intent is
different. In my case, the data was never intended to be a list, it was
always intended to be a fixed record-like collection, the use of list was
as a temporary data structure used for construction. A bit like the idiom
of ''.join(some_list).

[end quote]

To me, this sounds *exactly* like your use-case: your data, stack traces,
represent a little chunk of immutable data that you build up a line at a
time using a temporary list first, just like I wrote. And I said so.
There's no sign in either your code or your description that the stack
traces get treated as mutable objects in any way once you have finished
building them a line at a time. So your real world, practical, "in the
trenches" example matches my experience: you build a *fixed data record*
using a *temporary list*, throw the list away, and then never mutate that
data record again.

So why are we disagreeing? Like many such discussions on the Internet,
this one has rambled a bit, and I've misunderstood some of your code
(sorry), and you seem to have misunderstood the question I am asking.
Maybe my explanation was not clear enough, in which case, sorry again.

I'm asking about the case where one might want the key to remain mutable
even after it is used as a key, but can't because Python won't let you.
There's no sign that your stack trace example is such an example.

As I earlier said:

[quote]

But I can't think of any meaningful, non-contrived example where I might
want an actual mutable list of values as a dict key.

[end quote]


and I still can't.



--
Steven