Using AWK how to remove these kind of Duplicates?

Btw, it is interesting, that I gave you a solution at unix.com/shell-programming-scripting/167... and you add something new in your file, then I added the tolower() function here. :D.

Kent$ cat a 0008.ASIA. NS AS2.DNS.ASIA.CN. 0008.ASIA.

NS AS2.DNS.ASIA.CN. Ns1.0008.asia. NS AS2.DNS.ASIA.CN.Www.0008.asia.

NS AS2.DNS.ASIA.CN. Anish. Asia NS AS2.DNS.ASIA.CN.

Ns2.anish. Asia NS AS2.DNS.ASIA. CN 0008.

ASIA0. NS AS2.DNS.ASIA.CN. Kent$ awk -F' NS' '{ gsub(/\.

$/,"",$1);split($1,a,".")} length(a)==2{btolower($1)++;}END{for (x in b)print x}' a anish. Asia 0008. Asia btw, it is interesting, that I gave you a solution at 0008.

ASIA1, and you add something new in your file, then I added the tolower() function here. :D.

1 for tolower() – glenn jackman Sep 20 at 13:41 +1 for 'I gave you a solution at ...' – shellter Sep 20 at 15:38.

By putting your AWK script into a separate file, you can tell what's really going on. Here's a simple approach to your "filter out the duplicates" problem: # For each line in the file { # Decide on a unique key (eg. Case insensitive without trailing period) unique_key = tolower($1) sub(/\.

$/, "", unique_key) # If this line isn't a duplicate (it hasn't been found yet) if (!(unique_key in already_found)) { # Mark this unique key as found already_foundunique_key = "found" # Print out the relevant data print($1) } } You can run AWK files by passing the -f option to awk. If the above script isn't recognizable as an AWK script, here it is in inline form: awk '{ key = tolower($1); sub(/\. $/, "", key); if (!(key in found)) { foundkey = 1; print($1) } }.

– Dan Cecile Sep 20 at 3:56 Thanks alot man but it not get the uniq dataa 0008. ASIA ns1.0008.Asia 0008.asia anish. Asia ns2.anish.

Asia But I want the output only main domains. Only 0008. ASIA anish.

Asia – Anish Kumar Sep 20 at 4:05 I will not write the whole script for you. Try to modify what Kent or I has answered with. Then edit your question if you're still stuck.

If you need to learn more AWK, read the Wikibook. – Dan Cecile Sep 20 at 10:12.

Or, just use the shell: echo ' 0008.ASIA. NS AS2.DNS.ASIA.CN. 0008.ASIA.

NS AS2.DNS.ASIA.CN. Ns1.0008.asia. NS AS2.DNS.ASIA.CN.Www.0008.asia.

NS AS2.DNS.ASIA.CN. Anish. Asia NS AS2.DNS.ASIA.CN.

Ns2.anish. Asia NS AS2.DNS.ASIA. CN 0008.

ASIA0. NS AS2.DNS.ASIA.CN. ' | while read domain rest; do domain=${domain%.

} case "$domain" in (*. *. *) : ;; (*.

AAsSiIaA) echo "$domain" ;; esac done | sort -fu produces 0008.ASIA anish.asia.

S sample output indicates he has stripped sub-domains like ns1, ns2. +1 for Nice use of bash while read ... case ... pipline! – shellter Sep 20 at 15:42 Indeed.

Updated to exclude domains with 2 (or more) dots. – glenn jackman Sep 20 at 15:51.

Don't use AWK. Use Python import readlines result= set() for line in readlines: words = lines.split() if "asia" in words0.lower(): result. Add( words0.lower() ) for name in result: print name That might be easier to work with than AWK.Yes.It's longer.

But it may be easier to understand.

It's just like AWK: python yourscript.py.Also. That question has already been asked. And that question is trivially answered via man python at the command prompt.

And that question is trivially answered by reading documentation at python.Org – S. Lott Sep 20 at 9:50.

Here is an alternative solution. Let sort create your cased-folded and uniq list (and it will be sorted! ) { cat - 2) targ=tmpArrn-1 "." tmpArrn print targ }' \ | sort -f -u output 0008.

ASIA anish. Asia Edit: fixed sort -i -u to sort -f -u. Many other unix utilties use '-i' to indcate 'ignorecase'.

My test showed me I need to fix it, and I forgot to fix the final posting.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.