Generate the rules for your robots.txt

Generate a robots.txt rules to block the crawl <location to block> but allow the crawling of <location to crawl> within the domain

Using ChatGPT?

Save all chats, add your notes, categorize and search your chat history.