1·
6 months agoYou should put these entries into your robots.txt file.
To block the Google search crawler use for all of your site:
User-agent: Googlebot
Disallow: /
To block the Google AI crawler use:
User-agent: Google-Advanced
Disallow: /
CEO @ Mojeek No-tracking indepdendent search engine
- 0 Posts
- 2 Comments
Joined 6 months ago
Cake day: August 19th, 2024
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
We have submitted a PR to the Open WebUI repo, which would enable Mojeek: https://github.com/open-webui/open-webui/discussions/6588