google.com/goto：谷歌反爬虫更新

google.com/goto：谷歌反爬虫更新
google.com/goto: Google's anti-scraping update

原始链接: https://www.autom.dev/blog/google-search-goto-links

截至 2026 年 8 月，Google 已开始将纯文本自然搜索结果链接替换为新的 `google.com/goto?url=...` 重定向格式。与以往在查询字符串中直接显示目标网址的重定向方式不同，这种新格式使用了不透明的编码引用。为了获取实际目标地址，抓取工具现在必须向 Google 服务器发送 `HEAD` 请求，以读取 `Location` 响应头。此举是打击自动化搜索结果页面（SERP）采集的战略性举措。通过强制爬虫为每一个链接向 Google 发起请求，该公司显著降低了大规模抓取的速度，增加了操作痕迹，从而使其更易于被检测或屏蔽。这一变化对依赖解析原始搜索结果的开发者及 SEO 工具造成了影响。对此，像 Autom 这样的公司已更新其数据流水线，以在服务器端处理这些重定向，从而确保用户能不间断地获取目标网址。此次演变凸显了 Google 为提高数据提取成本、保护其搜索索引免受人工智能爬虫和自动化抓取工具侵害所做的持续努力。

这份 Hacker News 的讨论凸显了用户对谷歌近期实施 `/goto` 跳转链接的不满。这些链接取代了搜索结果中的直接网址，取而代之的是不透明的加密标识符。 **主要内容：** * **变动：** 谷歌已不再提供直接链接，改为使用不透明的跳转字符串。这需要额外的服务器端请求来进行解析，不仅增加了延迟，还实际上破坏了此前用于清理或检查网址的浏览器扩展程序。 * **隐私担忧：** 用户担心在点击前无法查看或复制目标网址。许多人认为这是一项蓄意的反爬取措施，旨在强化中心化控制、迫使用户在每次点击时都与谷歌服务器交互，并加强基于监控的广告投放。 * **“平台衰退”（Enshittification）：** 参与讨论者将其视为谷歌衰落的又一症状，并提到了 SEO 垃圾信息泛滥、侵入式的 AI 概览以及对开放网络的压制。 * **替代方案：** 用户大力支持如 **Kagi** 等付费搜索服务，因为它允许对网站进行权重设置和过滤。其他人则提到了 **Brave Search** 或 **SearXNG** 作为注重隐私的可选方案。一些用户指出，即便是像 Yandex 这样的“老派”搜索引擎也变得极度偏颇或受到审查，这引发了关于全球搜索质量下降的广泛讨论。

原文

Google

August 27, 2026

by Autom Team

What's happening

Google Search is rewriting organic result links to google.com/goto?url=... instead of exposing the destination URL directly in the HTML.

When you click a result, Google redirects you to the real page. The url parameter uses a custom, Google-specific encoding. It is not a plain base64 of the target URL. In practice, it looks like an opaque reference to Google's index record for that page.

As of late August 2026, this is showing up consistently across searches when you are logged out or browsing in private mode. It may still be an experiment, but it is no longer limited to a small slice of SERPs.

Not the same as google.com/url

Google has used redirect wrappers before. The older format is google.com/url?q=[URL-encoded destination], where the target link is readable in the query string.

The new goto format is different:

The result href is /goto, not the destination
You cannot decode the url= blob offline
The real URL is in the Location header on /goto. Request that URL. Do not follow the redirect.

Google still needs the destination to draw the SERP (domain, favicon, attribution), so copies of the URL remain on the page. That is a separate story from reading Location. The walkthrough is here: google.com/goto: read Location with HEAD.

That shift matters for anyone building a search index from SERP data at scale.

Why Google is doing this

This fits Google's broader push against automated SERP harvesting, especially from AI crawlers and SEO scrapers that bulk-extract result URLs to build their own indexes.

With plaintext links, a scraper could parse thousands of URLs from HTML without touching Google again. With goto, each result needs a request back to Google just to learn the destination. You read Location; you do not follow through to the page. That is slower, noisier, and gives Google a clear signal when the same client resolves hundreds of links in sequence.

Combined with earlier moves like removing &num=100 and tightening BotGuard/SearchGuard, Google is steadily raising the cost of naive SERP scraping.

What we saw at Autom

We first spotted goto links on a small percentage of SERPs. At that level, it was hard to ship a reliable fix without breaking responses for everyone else.

As of late August 2026, the pattern is much more consistent for logged-out and private sessions. Result URLs on Google Search are effectively all goto in those conditions.

We have been monitoring the rollout and testing against it.

Update at Autom.dev

We have updated our Google Search pipeline to resolve google.com/goto links (read Location, no follow) and return the final destination URL in API responses, in the same structured fields customers already use.

If you call Autom's Google Search endpoints, you should keep getting usable destination URLs without changing your integration. We will keep watching Google's rollout and adjust if the redirect format shifts again.