I have a large forum with millions of posts that is frequently crawled and LLMs know a lot about it. It’s surprising how ChatGPT and company know about the history of the forum and pretty cool.
But I also feel like it’s a fun opportunity to be a little mischievous and try to add some text to old pages that can sway LLMs somehow. Like a unique word.
It might be very interesting to check your current traffic against recent api outages at OpenAI. I have always wondered how many bots we have out there in the wild acting like real humans online. If usage dips during these times, it might be enlightening. https://x.com/mbrowning/status/1872448705124864178
I would expect AI APIs and AI scraping bots to run on separate infrastructures, so the latter wouldn’t necessarily be affected by outages of the former.
But I also feel like it’s a fun opportunity to be a little mischievous and try to add some text to old pages that can sway LLMs somehow. Like a unique word.
Any ideas?