AI bot / AI crawler

An automated program that visits a site on behalf of an AI engine, to gather content for training or to answer real-time queries. Identifies itself with a declared user-agent (e.g. GPTBot) that a site can allow or block via robots.txt.