How to keep only one of duplicates records

[damien henrotin]

Hello all,

I try to exclude some duplicate result but I need to keep the unique result.

Here is a example (no tthe real but just an example):

I have many RECORDS (not rows) and I have a column with one number by record.

I would like to keep only one :

N1

N2

N3

N4

N5

N6

N7

N8

N9

N10

N11

N12

Can you help me ? Thanks

[damien henrotin]

[damien henrotin]

Here is the project

[Thad Guidry]

If I understand you correctly, then I think you are looking for a way to filter or facet by duplicates?

Take a look at Facets -> Customized facets -> Duplicate facets  (then you might click on "false" to select only the rows that are not duplicated).

In order to eliminate duplicates, then that's another problem, which can be handled by flagging the rows, and then optionally deleting the rows, or simply blanking out the values after the 1st occurrence (blank down)

Thad

https://www.linkedin.com/in/thadguidry/

--
You received this message because you are subscribed to the Google Groups "OpenRefine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/79ef7664-b52b-4ce4-a848-949cbe434e42n%40googlegroups.com.

[damien henrotin]

Hello Thad,

Thanks for your reply.

I'm ok for Duplicate Facet. Problem is how to remove duplicate on my example. If I Push on True to see all the duplicates how can I remove the duplicates and keep only one of each?

[Thad Guidry]

If some ordering is important in your other record columns, then you might want to sort on those columns and reorder permanently.

Then try Blank Down on your Number column to see how it works.  You can always Undo as you experiment :-)

Thad

https://www.linkedin.com/in/thadguidry/

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/5f9e21c8-3680-4981-bde7-df4abeddae6fn%40googlegroups.com.

[damien henrotin]

Thad, I try but I did not get the good result. Can you show me on my example project please ?

Thank you

[Thad Guidry]

Here is an example of what I mean.

There are several guides and videos on the internet if you just search with "OpenRefine remove duplicates"  or "OpenRefine removing duplicates", etc.

https://guides.library.illinois.edu/openrefine/duplicates

Thad

https://www.linkedin.com/in/thadguidry/

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/6afd2ab7-6cf5-48bb-84be-c9f84667d95dn%40googlegroups.com.

[damien henrotin]

Nice Thank you thad. Just one thing again, you suggest me to transform my column to numeric value, it's better before apply a sort ?

[Thad Guidry]

Sort works on any of the value types that is listed in the Sort dialog.  Did you take a look? :-)

And you have control over ascending/descending order with the radio button option.

Click on the various radio buttons in the Sort dialog to see what and how it changes. (you can even position Blank values first if you want, or last, etc. with the other option in the Sort dialog)

Remember if you mess up on the Sort, you can always Undo and try again! :-)

Thad

https://www.linkedin.com/in/thadguidry/

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/6f8ed783-c90b-44eb-aadf-45cde49e656cn%40googlegroups.com.

[damien henrotin]

Ok NIce Thad, I think I'm on the right way

[Thad Guidry]

Great Damien, glad to hear it!

Facets are a great way to also double check things after you perform operations.  Use as many Facets as needed to accomplish inspection tasks you need to do.

Thad

https://www.linkedin.com/in/thadguidry/

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/065b146c-be97-4da9-810c-4e091b11a322n%40googlegroups.com.

[damien henrotin]

Indeed :-) 

As always, thank you for your advice Thad !