remove doublets

[Patrick Dupre]

Hello,

I wish to remove lines which appear twice in a file, actually
only when they appear more than one time.
How can I do so?

Thanks

===========================================================================
 Patrick DUPRÉ                                 | | email: [e-mail]
 Laboratoire interdisciplinaire Carnot de Bourgogne
 9 Avenue Alain Savary, BP 47870, 21078 DIJON Cedex FRANCE
 Tel: [phone]                    | | Room# D114A
===========================================================================

[George N. White III]

On Sun, 13 Jun 2021 at 06:51, Patrick Dupre  wrote:
> Hello,
>
> I wish to remove lines which appear twice in a file, actually
> only when they appear more than one time.
> How can I do so?

Do you need to keep the lines in the same order?   If not, then
"sort -u" should work.  You may also want to use "uniq".

% cat nonuniq.txt
dd
aa
bb
cc
bb
aa
 % sort -u nonuniq.txt
aa
bb
cc
dd

[Oğuz Ersen]

This preserves line order:

$ cat file.txt
dd
aa
bb
cc
bb
aa

$ awk '!line[$0]++' file.txt
dd
aa
bb
cc

--
Best Regards, Oğuz