https://stackoverflow.com/questions/39824733/awk-to-remove-duplicate-rows-totally-based-on-a-particular-column-value
https://unix.stackexchange.com/questions/171091/remove-lines-based-on-duplicates-within-one-column-without-sort
第三列重复的都删掉生成t2.txt
awk 'NR==FNR{c[$3]++;next} c[$3]<2' t.txt t.txt > t2.txt
找出重复的行(第一列打印出行号)
http://www.cocoachina.com/articles/113086
cat t.txt | awk 'n=x[$3]{print NR" "n;print NR" "$0;} {x[$3]=$0;}' > dup.txt