[svn] Fix the broken URLs that pointed to info.webcrawler.com to point to

the new www.robotstxt.org site.

diff --git a/NEWS b/NEWS
index 878bc88f476e9859971c6d575d50bb9678b9efb1..58c2dcbba2495a61e89b603d1854b52c0310c713 100644 (file)
--- a/NEWS
+++ b/NEWS
@@ -56,7 +56,7 @@ addresses when accessing the first one fails.
 non-standard port.
 
 ** Wget now supports the robots.txt directives specified in
 non-standard port.
 
 ** Wget now supports the robots.txt directives specified in

-<http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html>.
+<http://www.robotstxt.org/wc/norobots-rfc.txt>.

 
 ** URL parser has been fixed, especially the infamous overzealous
 quoting.  Wget no longer dequotes reserved characters, e.g. `%3F' is
 
 ** URL parser has been fixed, especially the infamous overzealous
 quoting.  Wget no longer dequotes reserved characters, e.g. `%3F' is

diff --git a/doc/ChangeLog b/doc/ChangeLog
index cadc4e4d1f40339be16dee72517c76e8bb09bc53..c346427107828b321a814d419c071f01270be43a 100644 (file)
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,8 @@
+2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
+
+       * wget.texi (Robots): Fix broken URLs that point to the webcrawler
+       web site.
+

 2001-12-11  Hrvoje Niksic  <hniksic@arsdigita.com>
 
        * wget.texi (HTTP Options): Explain how to make IE produce a
 2001-12-11  Hrvoje Niksic  <hniksic@arsdigita.com>
 
        * wget.texi (HTTP Options): Explain how to make IE produce a

diff --git a/doc/wget.texi b/doc/wget.texi
index e498d541c8ba9f7b8390174e4f93898bbe3c84d8..167a4e2fbba5977ce0cfae4f6b051857dc493aeb 100644 (file)
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -2743,12 +2743,12 @@ server.
 
 Until version 1.8, Wget supported the first version of the standard,
 written by Martijn Koster in 1994 and available at
 
 Until version 1.8, Wget supported the first version of the standard,
 written by Martijn Koster in 1994 and available at

-@url{http://info.webcrawler.com/mak/projects/robots/norobots.html}.  As
-of version 1.8, Wget has supported the additional directives specified
-in the internet draft @samp{<draft-koster-robots-00.txt>} titled ``A
-Method for Web Robots Control''.  The draft, which has as far as I know
-never made to an @sc{rfc}, is available at
-@url{http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html}.
+@url{http://www.robotstxt.org/wc/norobots.html}.  As of version 1.8,
+Wget has supported the additional directives specified in the internet
+draft @samp{<draft-koster-robots-00.txt>} titled ``A Method for Web
+Robots Control''.  The draft, which has as far as I know never made to
+an @sc{rfc}, is available at
+@url{http://www.robotstxt.org/wc/norobots-rfc.txt}.

 
 This manual no longer includes the text of the Robot Exclusion Standard.
 
 
 This manual no longer includes the text of the Robot Exclusion Standard.
 


@@ -2762,9 +2762,9 @@ this:
 @end example
 
 This is explained in some detail at
 @end example
 
 This is explained in some detail at

-@url{http://info.webcrawler.com/mak/projects/robots/meta-user.html}.
-Wget supports this method of robot exclusion in addition to the usual
-@file{/robots.txt} exclusion.
+@url{http://www.robotstxt.org/wc/meta-user.html}.  Wget supports this
+method of robot exclusion in addition to the usual @file{/robots.txt}
+exclusion.

 
 @node Security Considerations, Contributors, Robots, Appendices
 @section Security Considerations
 
 @node Security Considerations, Contributors, Robots, Appendices
 @section Security Considerations

diff --git a/src/ChangeLog b/src/ChangeLog
index af8fb0cadd44fbd62095b4927940738219ddfe3d..bba3f64d79bfb443c943615c15ba394cccdf9e73 100644 (file)
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,3 +1,7 @@
+2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
+
+       * res.c (matches): Fix broken URL in the docstring.
+

 2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
 
        * html-url.c (tag_url_attributes): Mark <embed href=...> as
 2001-12-13  Hrvoje Niksic  <hniksic@arsdigita.com>
 
        * html-url.c (tag_url_attributes): Mark <embed href=...> as

diff --git a/src/res.c b/src/res.c
index 38ed7f96a50eedd6289951c3e68dfa417fa1ca34..4aac318e829de75ba03c325f2e872c5243460dc4 100644 (file)
--- a/src/res.c
+++ b/src/res.c
@@ -422,8 +422,7 @@ free_specs (struct robot_specs *specs)
 
 /* The inner matching engine: return non-zero if RECORD_PATH matches
    URL_PATH.  The rules for matching are described at
 
 /* The inner matching engine: return non-zero if RECORD_PATH matches
    URL_PATH.  The rules for matching are described at

-   <http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html>,
-   section 3.2.2.  */
+   <http://www.robotstxt.org/wc/norobots-rfc.txt>, section 3.2.2.  */

 
 static int
 matches (const char *record_path, const char *url_path)
 
 static int
 matches (const char *record_path, const char *url_path)