PassciteBot

1. What it reads

Someone asked Passcite for a report on your website — usually the agency or consultant looking after it, sometimes the owner. To build that report PassciteBot reads the public pages a buyer could open without signing in: the homepage, robots.txt, the sitemap if there is one, and up to about two dozen pages the sitemap and navigation point at — services, pricing, locations, FAQs, reviews — plus up to four more whose addresses match the questions the report asks. It reads the text and the schema.org markup. It does not submit forms, follow links off your site, run JavaScript, or download images or files.

It also reads a page when a person asks it to: a free tool on passcite.com reads the address typed into it (the sitemap and llms.txt tools up to fifty pages of that site); a site health check reads robots.txt, llms.txt, the sitemap and the one page being checked — a client's page, or a competitor's homepage beside it — and asks Google PageSpeed Insights to load that page; a research reads the page it is given for the questions it answers; and writing an article reads the site again for what it already says, as approving it does once more.

When an AI assistant cites one of your pages in an answer an agency tracks, PassciteBot reads that page for its title, who wrote it, and which businesses it names. For a YouTube video it asks YouTube's public oEmbed endpoint for the channel instead.

When an agency opens Indexing for your site, PassciteBot reads your robots.txt and your sitemap — up to 50 of the files a sitemap index lists, a few at a time — for the pages to ask Google about. Whether Google has each page comes from Google's Search Console, not from your site.

If an agency wants its client on that page, PassciteBot reads your homepage — and your contact page, when the homepage shows no email address — for the address or contact page you publish, so it knows whom to write to. When the agency asks, it reads the cited page again to see whether its client is on it now. Any email comes from the agency's own mailbox, not from us.

And when an agency is looking for new clients: if your business is on its list, PassciteBot reads your homepage — and your contact page, when the homepage shows no email address — for the contact details you publish there and what the site says it is. Before the agency pitches you, it reads the site as it would for a report.

If your agency is in the agency directory, PassciteBot reads your robots.txt when your entry is added or checked again, about twice a year, and if it may read your homepage, asks Google PageSpeed Insights to load it for the small picture on your entry. If your agency asks to be listed there, it reads the page you sent, your homepage and the contact and about pages your homepage links, once, to check them against the directory's rules.

It does not write anything to your site. Passcite publishes only to a WordPress site connected with an application password, or a Webflow site connected with an API token, made for the purpose — through that connection, never through this crawler.

2. How often

Once per report, and once more after it to see what the site already answers. Pages are fetched a few at a time, one batch after another. A prospect's homepage, or the homepage of a site an agency wants to write to, is read once in thirty days — once a day if it did not answer — unless the agency asks for a fresh look. A page an answer cites is read once in 30 days for each brand whose answers cite it, and sooner only when that brand's list of competitors changes. Your sitemap, for Indexing, at most once a day.

One page more, after a page goes up. When Passcite publishes a page into a site connected to it, or someone marks a page as put up by hand — the agency, or the site owner it sent the page to — PassciteBot opens the page it went on to see whether it is there: right away, then up to eight times over the next two weeks until it shows up, and whenever they ask it to look again. The day it first appears is the day the change counts from.

The reads a person asks for happen when they ask: a page at a time for a check, a few pages at once when an article is written or approved.

3. How to refuse it

The two lines at the top of this page, added to your robots.txt, stop PassciteBot before it reads a second page. A blanket User-agent: * with Disallow: / is obeyed the same way, and so is a Disallow on a single path. A redirect is asked about again: a page that redirects to a refused path, or to a site whose own robots.txt refuses PassciteBot, is not read. A robots.txt refusal is final for every read above: for a report, nothing on your site is read — the assistants are still asked — an article for your site is refused and its credits given back, a pitch to your business is not made, and a tool or a check says it was refused instead of reading the page. The only file still read is robots.txt itself, because that is where the refusal is written. To check that the lines you added form a group of their own, and did not join one meant for another crawler, paste the file into the robots.txt validator.

4. Why it may retry as a browser

Some firewalls answer 401 or 403 to any User-Agent containing the word “Bot”, including ours, without the site owner having decided anything about Passcite. When the declared request is refused that way, PassciteBot retries the same URL once with an ordinary browser User-Agent.

That retry never overrides robots.txt. If your robots.txt names PassciteBot or all agents and disallows them, we stop regardless of what the firewall does.

5. Contact

Questions, or a request to delete what was read: hello@passcite.com. What a report stores is described on the privacy page.