[svn] Fix the broken URLs that pointed to info.webcrawler.com to point to

the new www.robotstxt.org site.

diff --git a/NEWS b/NEWS
index 878bc88f476e9859971c6d575d50bb9678b9efb1..58c2dcbba2495a61e89b603d1854b52c0310c713 100644 (file)
--- a/NEWS
+++ b/NEWS
@@ -56,7 +56,7 @@ addresses when accessing the first one fails.
 non-standard port.
 
 ** Wget now supports the robots.txt directives specified in
-<http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html>.
+<http://www.robotstxt.org/wc/norobots-rfc.txt>.
 
 ** URL parser has been fixed, especially the infamous overzealous
 quoting.  Wget no longer dequotes reserved characters, e.g. `%3F' is

diff --git a/doc/ChangeLog b/doc/ChangeLog
index cadc4e4d1f40339be16dee72517c76e8bb09bc53..c346427107828b321a814d419c071f01270be43a 100644 (file)
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,8 @@
+2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
+
+       * wget.texi (Robots): Fix broken URLs that point to the webcrawler
+       web site.
+
 2001-12-11  Hrvoje Niksic  <hniksic@arsdigita.com>
 
        * wget.texi (HTTP Options): Explain how to make IE produce a

diff --git a/doc/wget.texi b/doc/wget.texi
index e498d541c8ba9f7b8390174e4f93898bbe3c84d8..167a4e2fbba5977ce0cfae4f6b051857dc493aeb 100644 (file)
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -2743,12 +2743,12 @@ server.
 
 Until version 1.8, Wget supported the first version of the standard,
 written by Martijn Koster in 1994 and available at
-@url{http://info.webcrawler.com/mak/projects/robots/norobots.html}.  As
-of version 1.8, Wget has supported the additional directives specified
-in the internet draft @samp{<draft-koster-robots-00.txt>} titled ``A
-Method for Web Robots Control''.  The draft, which has as far as I know
-never made to an @sc{rfc}, is available at
-@url{http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html}.
+@url{http://www.robotstxt.org/wc/norobots.html}.  As of version 1.8,
+Wget has supported the additional directives specified in the internet
+draft @samp{<draft-koster-robots-00.txt>} titled ``A Method for Web
+Robots Control''.  The draft, which has as far as I know never made to
+an @sc{rfc}, is available at
+@url{http://www.robotstxt.org/wc/norobots-rfc.txt}.
 
 This manual no longer includes the text of the Robot Exclusion Standard.
 
@@ -2762,9 +2762,9 @@ this:
 @end example
 
 This is explained in some detail at
-@url{http://info.webcrawler.com/mak/projects/robots/meta-user.html}.
-Wget supports this method of robot exclusion in addition to the usual
-@file{/robots.txt} exclusion.
+@url{http://www.robotstxt.org/wc/meta-user.html}.  Wget supports this
+method of robot exclusion in addition to the usual @file{/robots.txt}
+exclusion.
 
 @node Security Considerations, Contributors, Robots, Appendices
 @section Security Considerations

diff --git a/src/ChangeLog b/src/ChangeLog
index af8fb0cadd44fbd62095b4927940738219ddfe3d..bba3f64d79bfb443c943615c15ba394cccdf9e73 100644 (file)
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,3 +1,7 @@
+2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
+
+       * res.c (matches): Fix broken URL in the docstring.
+
 2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
 
        * html-url.c (tag_url_attributes): Mark <embed href=...> as

diff --git a/src/res.c b/src/res.c
index 38ed7f96a50eedd6289951c3e68dfa417fa1ca34..4aac318e829de75ba03c325f2e872c5243460dc4 100644 (file)
--- a/src/res.c
+++ b/src/res.c
@@ -422,8 +422,7 @@ free_specs (struct robot_specs *specs)
 
 /* The inner matching engine: return non-zero if RECORD_PATH matches
    URL_PATH.  The rules for matching are described at
-   <http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html>,
-   section 3.2.2.  */
+   <http://www.robotstxt.org/wc/norobots-rfc.txt>, section 3.2.2.  */
 
 static int
 matches (const char *record_path, const char *url_path)