comparing dialects of csv-module

5 views
Skip to first unread message

Malte Dik

unread,
Dec 19, 2009, 6:08:19 AM12/19/09
to
Hi out there!

I want to put some intelligence into a csv reading script and in order to do
so I want to compare possible different dialects I collect with some random

d = csv.Sniffer().sniff("1,2,3,4"),

because the csv is kinda dirty.

Now sniff() returns a class object and those aren't comparable in the "if
dialect_1 == dialect_2: count something" sense.

Has anyone around here already dealt with this kind of problem and maybe
even a solution I could utilize? That would be great.

If not - I guess I would just write a quick function comparing the
attributes of those dialects ... - but just out of educational curiosity:
Would it be the right way to implement an __eq__(...) function into the
Dialect class or how would someone who would want to make it right do it?


Sincerest greetings,

Malte

Peter Otten

unread,
Dec 19, 2009, 9:06:58 AM12/19/09
to
Malte Dik wrote:

> Hi out there!
>
> I want to put some intelligence into a csv reading script and in order to
> do so I want to compare possible different dialects I collect with some
> random
>
> d = csv.Sniffer().sniff("1,2,3,4"),
>
> because the csv is kinda dirty.
>
> Now sniff() returns a class object and those aren't comparable in the "if
> dialect_1 == dialect_2: count something" sense.
>
> Has anyone around here already dealt with this kind of problem and maybe
> even a solution I could utilize? That would be great.

An implementation for the lazy

>>> import csv
>>> d = csv.Sniffer().sniff("1,2,3")
>>> def eq(a, b, attributes=[name for name in dir(d) if not
name.startswith("_")]):
... return all(getattr(a, n, None) == getattr(b, n, None) for n in
attributes)
...
>>> eq(d, csv.Sniffer().sniff("3,4,5"))
True
>>> eq(d, csv.Sniffer().sniff("'3','4','5'"))
False
>>> eq(d, csv.Sniffer().sniff("3;4;5"))
False
>>> eq(d, csv.Sniffer().sniff("3,4,' 5'"))
True


> If not - I guess I would just write a quick function comparing the
> attributes of those dialects ... - but just out of educational curiosity:
> Would it be the right way to implement an __eq__(...) function into the
> Dialect class or how would someone who would want to make it right do it?

I don't know if you can do it for classic classes; for newstyle classes
you'd have to reimplement comparison in the metaclass:

>>> class Dialect:
... class __metaclass__(type):
... def __eq__(self, other):
... return self._key() == other._key()
... def __ne__(self, other):
... return self._key() != other._key()
... def _key(self):
... return self.quotechar, self.delimiter #,...
...
>>> class A(Dialect):
... quotechar = "'"
... delimiter = ":"
...
>>> class B(Dialect):
... quotechar = "'"
... delimiter = ","
...
>>> A == B
False
>>> B.delimiter = ":"
>>> A == B
True

On a side note, I think it's a C++ism that dialects are classes rather than
class instances. That's a superfluous complication since in Python no work
will be moved from compile time to runtime anyway.

Peter

Malte Dik

unread,
Dec 19, 2009, 10:12:22 AM12/19/09
to
quote:

> An implementation for the lazy
>
>>>> import csv
>>>> d = csv.Sniffer().sniff("1,2,3")
>>>> def eq(a, b, attributes=[name for name in dir(d) if not
> name.startswith("_")]):
> ... return all(getattr(a, n, None) == getattr(b, n, None) for n in
> attributes)
> ...

Wow, this is awesome. I'd never come up with something like this.


Will digg into it deeper as I implement it (code snippets like this need to
melt in order to effuse all their flavor ;), but want to thank you very much
in the first place! :)

Have a nice day everyone,

Malte

Malte Dik

unread,
Dec 20, 2009, 10:22:36 AM12/20/09
to
quote:

>>>> d = csv.Sniffer().sniff("1,2,3")
>>>> def eq(a, b, attributes=[name for name in dir(d) if not
> name.startswith("_")]):
> ... return all(getattr(a, n, None) == getattr(b, n, None) for n in
> attributes)

Only change I made is substituting "dir(csv.excel)" or "dir(csv.Dialect)"
for "dir(d)", because I can't be always sure, that there'd be a nicely
defined "d".


Salute,

Malte

Reply all
Reply to author
Forward
0 new messages