WGET 401 Unauthorized?

Here is the manual for wget that includes some authentication details.

Up vote 0 down vote favorite share g+ share fb share tw.

I'm trying to use a batch file with WGET to download the public FCC file from here wireless.fcc.gov/uls/data/complete/l_micro.zip" rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic..." rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic..." rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic... When I intially run the batch file with parameters wget --server-response -owget. Log wireless.fcc.gov/uls/data/complete/l_micro.zip" rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic..." rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic..." rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic... It fails with an HTTP 401 unauthorized error. I can retry at this point and it keeps failing.

However I noticed if I open up IE, start a download and cancel when prompted to save, I can rerun the batch file and it executes perfectly! Here is my detailed server response from the log --2012-02-06 14:32:24-- wireless.fcc.gov/uls/data/complete/l_micro.zip" rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic..." rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic..." rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic... Resolving wireless.fcc.gov (wireless.fcc.gov)... 192.104.54.158 Connecting to wireless.fcc.gov (wireless.fcc.gov)|192.104.54.158|:80... connected. HTTP request sent, awaiting response... HTTP/1.1 302 Found Location: REMOVED - appears to have my IP Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset=utf-8 Connection: close Content-Length: 513 Location: REMOVED following --2012-02-06 14:32:24-- REMOVED Resolving REMOVED... 19Fri, 27 Jan 2012 18:37:51 GMT3Fri, 27 Jan 2012 18:37:51 GMT3 Connecting to REMOVED|19Fri, 27 Jan 2012 18:37:51 GMT3Fri, 27 Jan 2012 18:37:51 GMT3|:80... connected.

HTTP request sent, awaiting response... HTTP/1.1 401 Unauthorized Cache-Control: no-cache Pragma: no-cache WWW-Authenticate: NTLM WWW-Authenticate: BASIC realm="AD_BCAAA" Content-Type: text/html; charset=utf-8 Proxy-Connection: close Set-Cookie: BCSI-CS-8ECFB6B4AA642EF0=2; Path=/ Connection: close Content-Length: 575 Authorization failed. Here is the log after doing my little IE procedure and getting it to work --2012-02-08 15:52:43-- wireless.fcc.gov/uls/data/complete/l_micro.zip" rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic..." rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic..." rel="nofollow">wireless.fcc.gov/uls/data/complete/l_mic... Resolving wireless.fcc.gov (wireless.fcc.gov)... 192.104.54.158 Connecting to wireless.fcc.gov (wireless.fcc.gov)|192.104.54.158|:80... connected. HTTP request sent, awaiting response... HTTP/1.1 200 OK Server: Sun-Java-System-Web-Server/7.0 Date: Fri, 27 Jan 2012 18:37:51 GMT Content-type: application/zip Last-modified: Sun, 22 Jan 2012 11:18:09 GMT Etag: "46fa95c-4f1bf071" Accept-ranges: bytes Content-length: 74426716 Connection: Keep-Alive Age: Fri, 27 Jan 2012 18:37:51 GMT9 Length: 74426716 (71M) application/zip Saving to: `l_micro.

Zip' Any help is appreciated! Http wget http-status-code-401 unauthorized link|improve this question asked 12-02-089 at 21:03user119253731.

Your command works flawlessly to me ... – LemonBeagle Feb 8 at 21:10 I attempted wget wireless.fcc.gov/uls/data/complete/l_mic... and got a file l_micro. Zip of 74693506 bytes with sha1sum of 1f8393bf6231bfa9689ec91cab6a660244f35605. – Dan D.

Feb 8 at 21:13 Do you have a transparent proxy on your network? Can you run wireshark or fiddler on your IE session to see what it's doing differently? – Edward Thomson Feb 9 at 17:56.

Here is the manual for wget that includes some authentication details: linux.about.com/od/commands/l/blcmdl1_wg....

Thanks for the link, I've tried a bunch of different commands with no luck – user1192537 Feb 9 at 15:21.

Try setting a user-agent string with wget - e.g. --user-agent=Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; . NET CLR 1.1.4322; . NET CLR 2.0.50727) it's entirely feasible for a site to reject requests from certain user agents, particularly if they look to be circumventing the "usual" routes to information (i.e.

Through webpages). Although this doesn't explain your problem, it's a good idea anyway. Perhaps the site implements a mechanism whereby when you browse with a "known" browser (e.g. IE) it then caches your IP as "safe" then allows any user agent from your IP to download anything :).

I tried your suggestion and also used my own UA from my browser and it did not help. It seems to be on and off, when I got on my computer this morning it worked fine regardless of the settigns and recently stopped working. Could this somehow be a firewall issue or is it likely something else?

– user1192537 Feb 9 at 13:48.

You have a Blue Coat secure web gateway on your network, as evidenced by the line in the response: Set-Cookie: BCSI-CS-8ECFB6B4AA642EF0=2; Path=/ It looks like it wants you to authenticate, presumably with your domain credentials. Try passing them with --http-user and --http-passwd.

You have a Blue Coat secure web gateway on your network, as evidenced by the line in the response.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.