From: hniksic <devnull@localhost>
Date: Thu, 13 Dec 2001 07:29:05 +0000 (-0800)
Subject: [svn] Fix the broken URLs that pointed to info.webcrawler.com to point to
X-Git-Tag: v1.13~1888
X-Git-Url: http://sjero.net/git/?p=wget;a=commitdiff_plain;h=3b44ca73abb1c9b52ea59175097b06065a12d6e2

[svn] Fix the broken URLs that pointed to info.webcrawler.com to point to
the new www.robotstxt.org site.
---

diff --git a/NEWS b/NEWS
index 878bc88f..58c2dcbb 100644
--- a/NEWS
+++ b/NEWS
@@ -56,7 +56,7 @@ addresses when accessing the first one fails.
 non-standard port.
 
 ** Wget now supports the robots.txt directives specified in
-<http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html>.
+<http://www.robotstxt.org/wc/norobots-rfc.txt>.
 
 ** URL parser has been fixed, especially the infamous overzealous
 quoting.  Wget no longer dequotes reserved characters, e.g. `%3F' is
diff --git a/doc/ChangeLog b/doc/ChangeLog
index cadc4e4d..c3464271 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,8 @@
+2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
+
+	* wget.texi (Robots): Fix broken URLs that point to the webcrawler
+	web site.
+
 2001-12-11  Hrvoje Niksic  <hniksic@arsdigita.com>
 
 	* wget.texi (HTTP Options): Explain how to make IE produce a
diff --git a/doc/wget.texi b/doc/wget.texi
index e498d541..167a4e2f 100644
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -2743,12 +2743,12 @@ server.
 
 Until version 1.8, Wget supported the first version of the standard,
 written by Martijn Koster in 1994 and available at
-@url{http://info.webcrawler.com/mak/projects/robots/norobots.html}.  As
-of version 1.8, Wget has supported the additional directives specified
-in the internet draft @samp{<draft-koster-robots-00.txt>} titled ``A
-Method for Web Robots Control''.  The draft, which has as far as I know
-never made to an @sc{rfc}, is available at
-@url{http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html}.
+@url{http://www.robotstxt.org/wc/norobots.html}.  As of version 1.8,
+Wget has supported the additional directives specified in the internet
+draft @samp{<draft-koster-robots-00.txt>} titled ``A Method for Web
+Robots Control''.  The draft, which has as far as I know never made to
+an @sc{rfc}, is available at
+@url{http://www.robotstxt.org/wc/norobots-rfc.txt}.
 
 This manual no longer includes the text of the Robot Exclusion Standard.
 
@@ -2762,9 +2762,9 @@ this:
 @end example
 
 This is explained in some detail at
-@url{http://info.webcrawler.com/mak/projects/robots/meta-user.html}.
-Wget supports this method of robot exclusion in addition to the usual
-@file{/robots.txt} exclusion.
+@url{http://www.robotstxt.org/wc/meta-user.html}.  Wget supports this
+method of robot exclusion in addition to the usual @file{/robots.txt}
+exclusion.
 
 @node Security Considerations, Contributors, Robots, Appendices
 @section Security Considerations
diff --git a/src/ChangeLog b/src/ChangeLog
index af8fb0ca..bba3f64d 100644
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,3 +1,7 @@
+2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
+
+	* res.c (matches): Fix broken URL in the docstring.
+
 2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
 
 	* html-url.c (tag_url_attributes): Mark <embed href=...> as
diff --git a/src/res.c b/src/res.c
index 38ed7f96..4aac318e 100644
--- a/src/res.c
+++ b/src/res.c
@@ -422,8 +422,7 @@ free_specs (struct robot_specs *specs)
 
 /* The inner matching engine: return non-zero if RECORD_PATH matches
    URL_PATH.  The rules for matching are described at
-   <http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html>,
-   section 3.2.2.  */
+   <http://www.robotstxt.org/wc/norobots-rfc.txt>, section 3.2.2.  */
 
 static int
 matches (const char *record_path, const char *url_path)