grep for words not in file

[Guillaume Dargaud]

Hello all,
I must have rusted during my vacation because I'm sure I've done this
before...
I have a list of words and a file that contains some of those words (and
plenty more). I want the list of words which aren't in the file... I've been
playing around with grep for 5 minutes without success...
--
Guillaume Dargaud
http://www.gdargaud.net/

[Janis Papanagnou]

Am 23.05.2012 10:13, schrieb Guillaume Dargaud:
> Hello all,
> I must have rusted during my vacation because I'm sure I've done this
> before...
> I have a list of words and a file that contains some of those words (and
> plenty more). I want the list of words which aren't in the file... I've been
> playing around with grep for 5 minutes without success...

If you want using grep you may be looking for options -f and -v.

A solution with awk may look like

awk '
NR==FNR{w[$0];next}
NF && !($0 in w) {w[$0];print}
' wordsfile <( tr ' ' '/012' otherfile )

Solutions are depending on the format of the data file - all words
on a line of their own? -, whether you want interpunctation to be
excluded in comparisons, etc.

Janis

[Dave Gibson]

Guillaume Dargaud <use_the_co...@www.gdargaud.net> wrote:
> Hello all,
> I must have rusted during my vacation because I'm sure I've done this
> before...
> I have a list of words and a file that contains some of those words (and
> plenty more). I want the list of words which aren't in the file... I've
> been playing around with grep for 5 minutes without success...

If both files have one word per line and are sorted:

comm -23 wordlist otherfile

[Thomas 'PointedEars' Lahn]

Guillaume Dargaud wrote:

> I have a list of words and a file that contains some of those words (and
> plenty more). I want the list of words which aren't in the file... I've
> been playing around with grep for 5 minutes without success...

Show your moves, then.

--
PointedEars

Please do not Cc: me. / Bitte keine Kopien per E-Mail.

[Guillaume Dargaud]

Thanks for the suggestions,
it's certainly not the fastest way, but I used the following:

for i in $(cat WordList); do grep -q $i index.html || echo $i; done

[Ed Morton]

On 5/24/2012 3:53 AM, Guillaume Dargaud wrote:
> Thanks for the suggestions,
> it's certainly not the fastest way, but I used the following:
>
> for i in $(cat WordList); do grep -q $i index.html || echo $i; done

That is SO far from a working solution....

You really should have tried the suggested approaches and followed up on
specifically what it was about them that didn't work for you, including some
sample input and expected output. Any of the suggestions you got is a much
better starting point than what you show above.

Ed.

[Eric Pement]

On Wednesday, May 23, 2012 3:13:05 AM UTC-5, Guillaume Dargaud wrote:

> I have a list of words and a file that contains some of those words (and
> plenty more). I want the list of words which aren't in the file... I've been
> playing around with grep for 5 minutes without success...

Here is one way, using 2 steps:

1. Convert the textfile to a glossary of each word found in the text file. A very rough way to do it is like this (this may "wrap", but it should be on one line):

perl -pe 's/[[:punct:]]*\s+[[:punct:]]*/\n/g' textfile | sort -u >glossary

2a. To find words in the wordlist that are NOT in the glossary:

fgrep -of wordlist glossary | sort -u | fgrep -vf - wordlist

2b. To find words in the wordlist that ARE in the glossary:

fgrep -of wordlist glossary | sort -u | fgrep -f - wordlist

Hope this helps

[Stephane Chazelas]

2012-05-23 17:57:19 +0100, Dave Gibson:

And if there're not, with GNU tools:

mysort() {
grep -o '[[:alnum:]_-]' | sort -u
}

comm -23 <(mysort < wordlist) <(mysort < otherfile)

POSIXly, you could use named pipes and tr -sc instead of grep
-o.

--
Stephane