-portions of the site they wish to protect from the robots.
-
-The most popular mechanism, and the de facto standard supported by all
-the major robots, is the ``Robots Exclusion Standard'' (RES) written by
-Martijn Koster et al. in 1994. It specifies the format of a text file
-containing directives that instruct the robots which URL paths to avoid.
-To be found by the robots, the specifications must be placed in
-@file{/robots.txt} in the server root, which the robots are supposed to
+portions of the site they wish to protect from robots and those
+they will permit access.
+
+The most popular mechanism, and the @i{de facto} standard supported by
+all the major robots, is the ``Robots Exclusion Standard'' (RES) written
+by Martijn Koster et al. in 1994. It specifies the format of a text
+file containing directives that instruct the robots which URL paths to
+avoid. To be found by the robots, the specifications must be placed in
+@file{/robots.txt} in the server root, which the robots are expected to