Yes. Your site or webmaster may be generating the increasingly popular XML sitemap protocol described at www.sitemaps.org to help your pages get indexed on web search engines. If so, PicoSearch can also read these special files to search all of your site. All you need to do is add the URL as another Entry Point for the PicoSearch indexer to follow the links.
To clarify, the sitemap protocol refers to an XML file of a certain format; it is not the general concept of HTML pages on your website that list some or all the links on your site. General HTML maps of your site have always been useful to PicoSearch, and can make good Entry Points for ensuring the finding of all pages. But a sitemap protocol file requires special detection by PicoSearch, and this will happen when you follow the spec that requires one of the following:
For listing URLs:
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
For listing more sitemaps:
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
In these examples, 0.9 is just the latest version number, which of course PicoSearch is flexible about. But if PicoSearch sees the urlset or sitemap index tag in the xml, then it will look for URLs between the tags <loc> and </loc> If they are files on your site, they will definitely be indexed. If they are more sitemaps, those will in turn be pulled in and analyzed. PicoSearch will also uncompress any .xml.gz compressed sitemaps, since that is part of the spec.
Besides the loc tag, PicoSearch is not guaranteed to make use of other sitemap information. This is because PicoSearch has its own enhanced controls for details such as: last modification date (taken from your server's headers, used for faster incremental reindexing in paid accounts), change frequency (paid accounts can directly set the automatic reindexing scheduler in PicoSearch), and priority (ranking of documents is detemined by methods under your control in your account manager, see FAQ for details).
|
