                                                                                                                                                                                                                                                                
Path: l23ni58712bkv.0!nntp.google.com!news1.google.com!goblin3!goblin2!goblin.stu.neva.ru!weretis.net!feeder4.news.weretis.net!feeder1.news.weretis.net!news.solani.org!.POSTED!not-for-mail
From: Deadly Dirk <d...@pfln.invalid>
Newsgroups: comp.databases.oracle.misc
Subject: Re: data cleansing: externally or internally?
Date: Fri, 4 Nov 2011 17:51:02 +0000 (UTC)
Organization: solani.org
Lines: 22
Message-ID: <pan.2011.11.04.17.50.19@pfln.invalid>
References: <j9022g$4b5$1@news.task.gda.pl>
Mime-Version: 1.0
NNTP-Posting-Date: Fri, 4 Nov 2011 17:51:02 +0000 (UTC)
User-Agent: Pan/0.133 (House of Butterflies)
Cancel-Lock: sha1:HVd/Fyap7/YhpqqPYxSB2UDTh5Q=
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On Fri, 04 Nov 2011 07:51:49 +0100, geos wrote:

> there is a big text file with dirty data. 

How big is "big"?

> a company wants it to be
> clean. there are some known patterns expressed as like or regexp. I
> first thought about two approaches:
> 1) do this on the system level
> 2) or in a database

Database is not well suited for things like that. Personally, I would use 
Perl. Perl is my favorite tool because it's extremely versatile and fast 
but any scripting language with regex support will probably do.





-- 
I don't think, therefore I am not.