Stop OpenAI’s ChatGPT User agent from scanning and using your website’s content.

Background:

Wikipedia Describes ChatGPT as an AI chatbot developed by OpenAI. ChatGPT is an artificial intelligence (AI) chatbot developed by OpenAI and launched in November 2022. It is built on top of OpenAI’s GPT-3.5 and GPT-4 families of large language models (LLMs) and has been fine-tuned (an approach to transfer learning) using both supervised and reinforcement learning techniques.

ChatGPT can access real-time info from the web using plug-ins.

If you don’t want the ChatGPT User-agent to use your website’s data, you can block it using your robots.txt file.

A ‘robots.txt’ file is generally located at the root of the web server and can be accessed using SFTP or can be viewed using this URL:

https://yourwebsite.com/robots.txt

(replace ‘yourwebsite.com’ with your domain name)

You need to add this rule to your website’s ‘robot.txt’ file

To allow ChatGPT plug-ins to access your website, you can explicitly add the ChatGPT-User to your site’s robots.txt:

User-agent: ChatGPT-User
Disallow:

To allow plug-ins to access only parts of your site, you can add the ChatGPT-User to your site’s robots.txt like this:

User-agent: ChatGPT-User
Disallow:
Allow: /directory-1/
Allow: /directory-2/

To disallow plug-ins to access your site, you can add the ChatGPT-User to your site’s robots.txt:

User-agent: ChatGPT-User
Disallow: /

Reference ChatGPT User-agent: https://platform.openai.com/docs/plugins/bot

I hope this information helps you to make the right decision.

Thanks & Regards
Mandar Apte

Published by Mandar Apte

Mandar is a Mumbai-based multi-disciplinary designer with UX/UI, Logo, Symbol, and Brand Identity design expertise. He currently runs his Mudrkashar Linguistic Apple iPhone, iPad, and Mac app business in the heart of Mumbai city.

Leave a comment

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.