[phpBB Debug] PHP Warning: in file [ROOT]/phpbb/session.php on line 561: sizeof(): Parameter must be an array or an object that implements Countable
[phpBB Debug] PHP Warning: in file [ROOT]/phpbb/session.php on line 617: sizeof(): Parameter must be an array or an object that implements Countable
[phpBB Debug] PHP Warning: in file [ROOT]/phpbb/session.php on line 1065: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3925)
[phpBB Debug] PHP Warning: in file [ROOT]/phpbb/session.php on line 1065: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3925)
[phpBB Debug] PHP Warning: in file [ROOT]/phpbb/session.php on line 1065: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3925)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 5312: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3925)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 5312: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3925)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions.php on line 5312: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3925)
KrazyForum • Wget and User-Agent Header
Page 1 of 1

Wget and User-Agent Header

Posted: Tue Sep 08, 2009 3:09 am
by Igor
As you may already know, Wget is a popular (particularly in the Unix world) command-line downloader and Web crawler application. You can read more about Wget in one of my earlier posts on the subject. One issue with Wget is that some sites block it from accessing their content. This is usually done by adding Wget to the robots.txt on the Web server and by configuring the server to reject requests with the user-agent header containing “wget”...

Read more: http://www.krazyworks.com/wget-and-user-agent-header/