Google-Extended Crawler: What’s New?
Google’s updates to its Extended crawler documentation clarify how websites can control AI training access. The changes separate AI data collection from search rankings, allowing site owners to decide if their content helps train Gemini Apps. These modifications give precise control through robots.txt settings while keeping search visibility intact.