The Evolution of Link Building in Search Engine Optimization: From PageRank Foundations to AI-Driven Entity Authority (1990s–Present)

1. Abstract and Keywords

Abstract

Hyperlinks have been the structural foundation of web search since the mid-1990s. They are also the signal that the practice of search engine optimization (SEO) has most persistently tried to manufacture. This paper traces the co-evolution of link-based ranking algorithms and link-building practice from the first web-scale crawlers in 1994 to the generative, entity-centric search systems of the mid-2020s.

The study uses a historical-analytical methodology. It triangulates four kinds of evidence: foundational information-retrieval and bibliometric literature, patent filings and academic papers from search engine researchers, official search engine announcements and documentation, and contemporaneous practitioner discourse. Evidence from antitrust litigation and the 2024 disclosure of internal Google API documentation is treated cautiously, as partial and interpretive.

The paper divides the period into five eras: link quantity (1994–2003), industrialized manipulation and first-wave defenses (2004–2011), algorithmic reckoning and the quality shift (2012–2017), relevance, editorial integrity and digital public relations (2018–2022), and AI search, entity authority and linkless signals (2023–present).

Three findings structure the analysis.

1. The relationship between search engines and link builders is adversarial but co-adaptive. Each ranking innovation created a new optimization target, and each wave of manipulation prompted a corresponding defense. This is consistent with Goodhart’s Law and Campbell’s Law.
2. The unit of value has migrated. It moved from the raw hyperlink (a count), to the link’s source and context (a weighted, trust-qualified vote), to the entity and its corroborated reputation across the web, where a link is one evidence type among several.
3. In generative and answer-oriented search, brand mentions, citation by language models, and entity co-occurrence increasingly work as functional equivalents of links. This does not make hyperlinks obsolete. It changes what link acquisition is for: building verifiable, retrievable authority rather than accumulating votes.

The paper concludes that link building has not died but has been reclassified, from a technical manipulation of a graph into a component of reputation management and public relations, and that this shift has ethical, economic and infrastructural consequences.

Keywords

link building; search engine optimization; PageRank; link analysis; link spam; TrustRank; Google Penguin; SpamBrain; E-E-A-T; entity authority; digital public relations; generative engine optimization; AI Overviews; information retrieval; web spam

2. Introduction

2.1 Background: The Architecture of the Web and the Primacy of the Link

Any account of link building must begin with the architecture that made links meaningful. The World Wide Web was not the first hypertext system, but it was the first to scale globally, and the properties that made it scalable are the same ones that made it searchable and, eventually, gameable.

The conceptual lineage runs through Vannevar Bush’s 1945 essay “As We May Think,” which proposed the Memex, a device for building associative trails between documents. It continues through Ted Nelson’s coinage of “hypertext” in the 1960s and the development of Xanadu, which envisioned bidirectional, versioned and attributed links. Nelson’s design held that links should be two-way and that the system should track the provenance of every quotation. The Web adopted neither property.

Tim Berners-Lee’s 1989 proposal at CERN, “Information Management: A Proposal,” and the working prototype he built in 1990 and 1991, reduced hypertext to three deliberately minimal technologies: the Uniform Resource Locator (URL) as a global naming scheme, the HyperText Transfer Protocol (HTTP) for retrieval, and the HyperText Markup Language (HTML) for document structure and embedded references. The decision with the greatest downstream consequence was that links are unidirectional and unmediated. A page may point to any other page without that page’s knowledge, consent or notification. Such a design needs no central registry and no permission, and it lets the network grow organically. It also means the target of a link has no native way to know who is pointing at it, or why.

This asymmetry matters for search. A hyperlink on the Web is an act of unilateral reference whose cost to the author is close to zero and whose semantic content (endorsement, criticism, navigation, advertising, citation or mere convenience) is not encoded in the link itself. The <a href> element expresses only that a path exists. Whatever meaning the link carries must be inferred from its surroundings: the anchor text, the surrounding prose, the authority of the page it sits on, and the topical relationship between source and target.

The Web’s growth supplied the pressure that turned this inference problem into a commercial and technical one. The Mosaic browser (1993) and Netscape Navigator (1994) took the Web from research networks to the general public, and the number of pages grew explosively. Lawrence and Giles (1998) estimated the indexable Web at roughly 320 million pages in late 1997, and found that no single engine covered more than a fraction of it. By 2008, Google reported that its systems had encountered a trillion unique URLs. In an environment of that scale, manual curation, the model of early directories such as Yahoo!, could not keep pace, and automated retrieval became a necessity.

Structural studies of the Web graph showed why links were rich in information. Albert, Jeong and Barabási (1999) and Barabási and Albert (1999) showed that the Web’s link distribution follows a power law, with a small number of heavily linked hubs and a long tail of sparsely linked pages. They attributed it to preferential attachment, in which new nodes tend to link to already well-connected ones. Broder et al. (2000) mapped the graph’s macroscopic “bow-tie” structure, with a strongly connected core, upstream and downstream components, and disconnected islands. These findings implied that link structure is not random noise. It encodes collective, if imperfect, judgments about which documents matter. They also implied that the network’s inequality of attention is self-reinforcing, a property that later made link acquisition so economically valuable.

2.2 Why Hyperlinks Became the Core of Information Retrieval

Before the Web, information retrieval (IR) was a discipline of closed, curated collections: library catalogs, abstract databases and newswire archives. Its dominant paradigms were content-based. Gerard Salton’s SMART system and the vector space model (Salton, Wong and Yang, 1975) represented documents and queries as weighted term vectors. Spärck Jones (1972) introduced inverse document frequency, the principle that rare terms discriminate better than common ones. Robertson and Spärck Jones (1976) developed the probabilistic relevance framework, which culminated in the BM25 family of ranking functions (Robertson and Walker, 1994). These methods estimate topical relevance from the text of the document and the query.

Content-based methods worked well in controlled collections where documents were written by professionals, had roughly uniform quality and were not adversarial. The early Web violated all three assumptions. Authors were anonymous and heterogeneous, quality varied by orders of magnitude, and, critically, authors had a financial incentive to misrepresent their pages’ content to attract traffic. First-generation engines such as WebCrawler, Lycos, Excite and AltaVista relied heavily on on-page text, which meant that keyword repetition, hidden text and misleading meta tags could manipulate rankings directly. Content, being entirely under the author’s control, was a weak signal in an adversarial setting.

The answer came from an older tradition, bibliometrics, the quantitative study of scholarly citation. Eugene Garfield’s citation indexing (1955) proposed that references between papers carry evaluative information, and that a heavily cited paper is, on average, an influential one. The idea generated an extensive methodological family. Kessler (1963) described bibliographic coupling, in which papers are related if they cite the same sources. Small (1973) described co-citation, in which papers are related if they are cited together. Pinski and Narin (1976) introduced an influence measure for journals that weighted citations by the influence of the citing journal. This is a recursive notion of authority, in which a citation from an important source is worth more than one from an obscure source. Earlier work in sociometry and econometrics, including Katz (1953) and Hubbell (1965), had formalized recursive status measures on directed graphs.

The conceptual step taken by early web-search researchers was to treat hyperlinks as analogues of citations and to apply recursive authority measures to the Web graph. Several lines of work emerged almost simultaneously in the mid-1990s. Marchiori (1997) proposed that a page’s quality should depend on the quality of the pages it links to. Spertus (1997) explored using link structure for web-search ranking. Kleinberg’s Hyperlink-Induced Topic Search (HITS) algorithm, first presented in 1998 and published in 1999, distinguished “hubs” and “authorities” and computed them as mutually reinforcing eigenvector quantities on a query-specific subgraph. Bharat and Henzinger (1998) refined that approach to address problems of topic drift. Most consequentially, Brin and Page (1998) described PageRank, a query-independent measure of a page’s importance, computed as the stationary distribution of a random surfer who follows links and occasionally jumps to a random page. The accompanying technical report (Page, Brin, Motwani and Winograd, 1999) developed the mathematics, and the associated patent (US 6,285,999) was granted to Stanford University in 2001 and exclusively licensed to Google.

The reasons why this approach displaced content-only ranking are instructive and recur throughout this paper.

1. Links are an external signal. A page’s author controls its text, but, in principle, does not control who links to it. External signals are harder to manipulate than internal ones.
2. Links aggregate human judgment at scale. Each link represents an author’s decision, however minimal, to direct attention elsewhere. Summed over a graph, these decisions provide a crowd-sourced estimate of importance that no keyword-matching heuristic could replicate.
3. Link-derived authority is query-independent and can be precomputed, which makes it computationally efficient at web scale.
4. Link anchor text provides a description of the target page written by a third party. Brin and Page (1998) noted that anchor text often describes a page more accurately than the page describes itself, and that anchors permit retrieval of resources the crawler has not fully indexed, such as images or programs.

By the end of the 1990s, link analysis had moved from a research proposal to the organizing principle of the leading search engine. The Web’s hyperlink structure had become, in effect, an economic ledger of attention.

2.3 Problem Statement: Manipulation Versus Algorithmic Defense

The adoption of links as a primary ranking signal introduced a structural vulnerability. Once links determine visibility, and visibility determines revenue, links acquire a market value, and the incentive to produce them artificially follows. This dynamic is an instance of what Goodhart’s Law (in Strathern’s 1997 phrasing, “when a measure becomes a target, it ceases to be a good measure”) and Campbell’s (1979) analogous observation about social indicators describe: a metric that is used for decisions becomes subject to pressures that corrupt its informational value.

The authors of PageRank anticipated aspects of this risk. Brin and Page (1998) acknowledged that commercial pressure and mixed incentives threaten the integrity of search results, and the paper argued for transparency and academic scrutiny. Early in its history, however, Google’s ranking signals were visible to a public that could reverse-engineer them. By the early 2000s, a nascent profession of search engine optimizers had developed a working model of link-based ranking. The profession’s most reliable lever was the acquisition of links, by whatever means.

The problem this paper addresses can therefore be stated as a recurring cycle:

1. A search engine adopts a link-based signal because it correlates with quality.
2. Webmasters learn that the signal affects rankings and produce it artificially, which breaks the correlation.
3. The search engine develops countermeasures, in the form of algorithmic devaluation, manual penalties, new link attributes or fundamentally different signals.
4. Practitioners adapt, either by finding new methods of manipulation or by shifting to methods the engine tolerates or rewards.

Gyöngyi and Garcia-Molina (2005) provided an early systematic taxonomy of web spam, including link spam, link farms and honey pots. Gyöngyi, Garcia-Molina and Pedersen (2004) proposed TrustRank, which propagates trust from a manually vetted seed set. Henzinger, Motwani and Silverstein (2002) had already identified spam as one of the principal open challenges facing web search engines. These works show that the research community recognized adversarial manipulation as a first-order problem from the early 2000s.

What remains less well documented is the long-run trajectory of the cycle: how the unit of value moved over thirty years, and how the practice known as link building changed in response. Much of the existing literature is either technical (algorithmic descriptions of specific methods) or practitioner-oriented (tactical guides tied to a particular algorithm version). Few works treat the whole period as a coherent historical process, and fewer connect the classical link-analysis era to the present phase, in which large language models, generative answer interfaces and entity-based knowledge systems are changing how authority is assessed and how users reach information.

2.4 Objectives and Research Questions

This paper has four objectives.

1. To provide a rigorous historical account of link-based ranking and link building from 1994 to the present, grounded in primary technical sources and official search engine communications.
2. To analyze the major algorithmic and policy interventions in this history, including the Florida update, Jagger, the introduction of nofollow, Penguin, the Disavow Tool, the sponsored and ugc attributes, link spam updates and SpamBrain, and to explain the information-retrieval principles behind each.
3. To characterize the changing nature of authority signals, from raw link counts to trust-weighted, topically relevant links, and from links to entities and brand-level corroboration.
4. To evaluate the implications of generative and AI-mediated search for the future of link building, including the possibility that links may serve as training and retrieval inputs for language models rather than only as ranking votes.

These objectives are pursued through the following research questions:

• RQ1. How did the dominant link-building tactics of each era respond to the ranking signals and enforcement mechanisms of that era?
• RQ2. Which design principles in the information-retrieval literature (recursive authority, trust propagation, topical locality, anchor-text semantics) explain the successes and failures of successive link-based algorithms?
• RQ3. To what extent have non-link signals, including brand mentions, entity co-occurrence and AI citation, replaced or supplemented hyperlinks as indicators of authority?
• RQ4. What ethical, economic and infrastructural challenges (sponsored-content disclosure, link rot, and the shift of links from ranking votes to AI training weights) bear on the future of the discipline?

2.5 Methodology

This study uses a historical-analytical review, a qualitative method that reconstructs and interprets a technical and commercial development through systematic examination of documentary sources. It does not claim to be an empirical measurement of ranking behavior. The method has four components.

Source categories. The analysis draws on four evidentiary categories.

1. Foundational scholarly literature in information retrieval, bibliometrics, network science and web-spam research, including the works cited in this section, which establish the conceptual and mathematical basis of link analysis.
2. Patents and technical papers from search engine researchers, which reveal design intent and, with caveats, the architecture of production systems. Patents describe what an organization sought to protect, not necessarily what it deployed, so they are used as evidence of conceptual direction rather than proof of implementation.
3. Official communications and documentation from search engines, including announcements on Google’s Search Central blog, published webmaster guidelines, the Search Quality Rater Guidelines and public statements by Google representatives. These are the most reliable source on stated policy, though they are curated and often omit operational detail.
4. Practitioner and industry discourse, including archived forum discussions, conference presentations, industry studies and trade publications. These sources document how updates were experienced and how tactics evolved, but they are subject to survivorship bias, commercial interest and the correlation-versus-causation problems inherent in reverse-engineering a proprietary system.

Treatment of proprietary evidence. Search ranking systems are proprietary and change continuously. Two recent sources have enlarged the evidentiary record. The first is testimony and exhibits from antitrust proceedings in the United States concerning Google’s search business, which have described certain ranking components, including click-based systems. The second is the May 2024 disclosure of internal Google Content Warehouse API documentation, whose authenticity was subsequently acknowledged by Google, with a caution that it lacked context and might be outdated. Because these materials describe attributes and modules rather than weights or live behavior, this paper treats them as indicative of the existence of signal categories, never as proof of how those signals are weighted.

Periodization. The five-era framework is an analytical construct, not a claim about discrete breaks. Eras overlap, tactics persist after the mechanisms that devalued them, and several practices (for instance, directory submission or reciprocal linking) never fully disappeared. Boundary years are selected because they coincide with identifiable inflection points: the commercialization of Google and the Florida update (2003), the introduction of nofollow (2005) and the industrialization of spam that preceded it, the first Penguin release (2012), the maturing of editorial and relevance-based criteria through the late 2010s, and the rise of generative AI search from 2023.

Analytical approach. For each era the paper (a) describes the dominant ranking signals and their information-retrieval basis, (b) identifies the characteristic manipulation tactics, (c) analyzes the principal search engine countermeasures, and (d) assesses how practitioners adapted. A comparative synthesis (Section 8) then examines the shifts across eras along consistent dimensions: core signal, unit of value, dominant tactics, primary defenses and economic incentives.

2.6 Scope, Limitations and Terminology

The scope is deliberately focused. The paper concentrates on Google because its link-based algorithms define the history of the field, because it has published the most relevant research and documentation, and because it has held the dominant share of Western web search throughout the period. Other engines, including AltaVista, Yahoo!, Bing, Baidu and Yandex, are discussed where they illuminate the argument. Yandex’s brief 2014 removal of link factors from commercial queries in Moscow, and later reintroduction, is one such case. The treatment of non-Western ecosystems is necessarily partial.

Four limitations should be stated plainly. First, the opacity of ranking systems means that many claims about the effect of a particular update are inferential. Second, practitioner literature is not peer reviewed, and the paper flags where a claim rests on industry consensus rather than primary evidence. Third, the discussion of AI-driven search (Section 7) concerns a domain that is changing quickly, so conclusions there are provisional. Fourth, the paper analyzes link building as a social and technical phenomenon, not as a manual of tactics.

Terminology is used consistently. Link building denotes the deliberate acquisition of hyperlinks from external sites to a target page or domain, for the purpose of improving search visibility, referral traffic or reputation. Link spam denotes link practices that violate a search engine’s published policies against manipulating ranking. Entity authority denotes the degree to which a real-world person, organization or concept is recognized as a distinct, reputable and topically associated entity across a body of sources, a notion elaborated in Section 7. Generative engine optimization (GEO) denotes the practice of shaping content and reputation so that it is retrieved and cited by generative answer systems.

2.7 Structure of the Paper

The remainder of the paper proceeds chronologically before turning to synthesis. Section 3 examines Era I (1994–2003), covering pre-Google retrieval, the publication of PageRank, directories, reciprocal links and blogrolls. Section 4 examines Era II (2004–2011), the industrialization of link spam, the introduction of nofollow, the Jagger update and trust-based defenses. Section 5 covers Era III (2012–2017), including Penguin, the Disavow Tool, third-party link-quality metrics and the rise of content-led outreach. Section 6 covers Era IV (2018–2022), in which topical relevance, E-A-T, link attributes and digital PR matured. Section 7 covers Era V (2023–present), addressing SpamBrain, E-E-A-T, brand co-occurrence and link building for AI-mediated retrieval. Section 8 provides the comparative synthesis. Section 9 considers ethical questions, link decay and the future of the hyperlink as both a ranking signal and a machine-learning input. Section 10 concludes and provides the full reference list.

Taken together, these sections argue that the history of link building is best understood not as a sequence of tricks and penalties, but as a long negotiation over how a decentralized network can communicate trustworthy judgments about itself. That negotiation began with a single design choice, the unmediated, unidirectional hyperlink, and it continues in the age of generative AI.

3. Era I: The Dawn of Search and Link Quantity (1994–2003)

3.1 The Pre-Google Retrieval Landscape

The first Internet search tools predate the Web. Archie (1990) indexed file names on anonymous FTP servers, and Veronica and Jughead later performed similar functions for Gopher space. These systems matched queries against titles and file names. They had no concept of document content, let alone of link structure. The first web-specific crawlers appeared in 1993 and 1994. Matthew Gray’s World Wide Web Wanderer measured the Web’s growth, and Martijn Koster’s ALIWEB let site owners submit descriptions of their own pages, an early example of the self-declared metadata approach whose weaknesses would later be exposed. Koster also proposed the Robots Exclusion Protocol in 1994, the first widely adopted convention governing how automated agents interact with sites, and a precursor to the control mechanisms discussed in Section 9.

WebCrawler (1994), built by Brian Pinkerton, was among the first engines to index the full text of pages. Lycos, developed at Carnegie Mellon, appeared in the same year. Excite and Infoseek followed in 1995. AltaVista, launched by Digital Equipment Corporation in December 1995, demonstrated the technical feasibility of indexing a substantial fraction of the Web with fast query response, and for a time it set the standard for coverage. Inktomi, which powered HotBot from 1996, and Ask Jeeves (1996) completed a crowded field. Yahoo!, founded in 1994 by Jerry Yang and David Filo, took a different approach, a human-edited hierarchical directory. For several years Yahoo! was the principal gateway to the Web, and a listing in its directory was itself a valuable commercial asset.

The ranking logic of the full-text engines was almost entirely on-page. Typical signals included the frequency of query terms in the document, their position (title, headings, early body text), their proximity to one another, and the contents of the <meta name="keywords"> and <meta name="description"> tags. These are descendants of the vector-space and probabilistic models described in Section 2.2, applied to a collection that the models had never been designed for. The consequence was immediate. Because every input to the score was under the page author’s control, the signal was cheap to manipulate. Keyword stuffing, invisible text (white text on a white background), repeated meta keywords, and “doorway pages” built to rank for a term and redirect visitors elsewhere were widespread by 1996 and 1997. The term “search engine optimization” came into circulation in this period, although its precise origin is disputed.

Two features of this landscape are worth emphasizing. First, the manipulation of on-page signals required no cooperation from anyone else and no capital, only technical knowledge. Second, the engines’ response was largely reactive and heuristic: filters for repeated terms, penalties for hidden text, and, in some cases, manual removal. Link structure was not yet a ranking input in any major engine. It was a navigation device, and its value for retrieval remained a research proposal.

3.2 BackRub, PageRank and the Link as Vote

Sergey Brin and Larry Page began work on a link-based crawler at Stanford in 1996. The prototype, called BackRub after its ability to follow links backward, was motivated by the observation that the Web’s link structure contained information that existing engines ignored. The google.com domain was registered in 1997, and the company was incorporated in September 1998. Its founding paper, “The Anatomy of a Large-Scale Hypertextual Web Search Engine” (Brin and Page, 1998), presented the architecture of the system and the PageRank algorithm.

The PageRank formulation. The basic intuition is that a page is important if important pages link to it. In the 1998 paper, the PageRank of a page A with in-linking pages T₁ … Tₙ is expressed as PR(A) = (1 − d) + d · Σ [PR(Tᵢ) / C(Tᵢ)], where C(Tᵢ) is the number of outbound links on page Tᵢ and d is a damping factor, which the authors set to about 0.85. The recursion can be read as the stationary distribution of a “random surfer” who follows a randomly chosen outlink with probability d and jumps to a random page with probability 1 − d. Mathematically, it is the principal eigenvector of a modified Web adjacency matrix, computed efficiently by power iteration. Bianchini, Gori and Scarselli (2005) and Langville and Meyer (2006) later analyzed its convergence properties, its treatment of dangling nodes (pages without outlinks), and its sensitivity to link-structure changes.

Four properties of this design shaped everything that followed.

1. Link value is divided. A page distributes its rank among its outlinks, so a link from a page with few outlinks passes more value than one from a page with many. This is the origin of the intuition, central to later link-building practice, that a link from a prominent, selective page is worth more than many links from minor ones.
2. Value is recursive. A link’s worth depends on the rank of its source, which in turn depends on that source’s inbound links. Authority therefore concentrates where it already exists, which reflects the preferential-attachment structure of the Web graph (Section 2.1).
3. The score is query-independent. PageRank measures global importance, not relevance to a query. It was combined with text-matching signals, including anchor text, at query time.
4. The damping factor limits the effect of long link chains, making rank harder to inflate through remote manipulation, at least in principle.

Anchor text. The second innovation, arguably more consequential for link building, was the use of anchor text. Brin and Page (1998) observed that the text of a link usually describes its target concisely and accurately, often better than the target’s own content. Google therefore associated anchor text with the linked page as well as with the page on which it appeared. This allowed it to return pages for terms that did not appear on them, and it made a link a two-part signal: a vote of importance and a label of subject matter. The label function would become the principal target of manipulation over the following decade.

The ledger becomes visible. PageRank was initially invisible to webmasters. In December 2000, Google released its browser toolbar, which displayed a page’s PageRank on a logarithmic scale from 0 to 10. The intention was presumably to give users a quality indicator. The effect was to give the SEO community a numeric proxy for a ranking input, and thus a measurable target. “PR” became a currency. Sites advertised their toolbar PageRank, link exchanges were negotiated around it, and the market for links acquired a unit of account. Google stopped updating the displayed value in 2013 and removed it entirely in 2016 (Section 5), by which time it had spent more than a decade shaping behavior.

Context: contemporaneous link-analysis models. PageRank was not the only link-based model of the period. Kleinberg’s HITS (1999) computed hub and authority scores on a query-specific neighborhood graph, making it query-dependent and expensive but sensitive to topic. Bharat and Henzinger (1998) addressed HITS’s vulnerability to topic drift and to mutually reinforcing links among affiliated hosts. Haveliwala (2002) proposed topic-sensitive PageRank, which biases the random jump toward a set of pages on a given topic. These variants anticipated the central weakness of the original algorithm: it treated all links as equivalent regardless of topic, so that a high-PageRank page on one subject could confer authority on a page on an unrelated subject. The need to address that weakness runs through the history until the relevance-centered Era IV.

Competitive significance. By 2000 and 2001, Google’s results were widely regarded as superior to those of its competitors. The reasons are partly technical (better handling of navigational and popular queries, where link-based authority is most informative) and partly interface-related (a minimal page and fast response). Its adoption of the AdWords advertising program in October 2000 and, in 2003, of AdSense, which placed Google ads on third-party sites, created a financial architecture that rewarded traffic and therefore rewarded rankings. The consequences for the link economy are examined in Section 3.6.

3.3 Directories and the Economics of Editorial Gatekeeping

Directories were the most prominent link source of the 1990s, and they occupy a peculiar place in this history. They were editorial, in the sense that humans decided which sites to list and under which category. They were also, in effect, the first large-scale link-building market.

Yahoo! Directory. Yahoo! began as a hand-curated list and, by the late 1990s, charged a fee for expedited review of commercial listings. A listing carried commercial value in two ways: direct referral traffic and, after PageRank, a link from a high-authority domain. The fee, and the editorial review that came with it, constituted an early quality filter. The Yahoo! Directory was closed at the end of 2014.

The Open Directory Project (DMOZ). Founded in 1998 by Rich Skrenta and Bob Truel as a volunteer-edited alternative, and acquired by Netscape later that year, the Open Directory Project (ODP, widely known as DMOZ) distributed its data under an open license. Many search engines and portals, including Google through its own Google Directory, reused it, which meant that a single listing could propagate into dozens of downstream sites. For some years, inclusion in DMOZ was regarded as one of the most valuable links a site could obtain, and the volunteer editors became powerful gatekeepers. The system’s weaknesses became well known: backlogs that left submissions unreviewed for months or years, allegations of editor bias and corruption, and a structure that could not keep up with the Web’s growth. Google closed its own directory in 2011, and DMOZ shut down in March 2017.

Directory submission as a tactic. As the number of directories grew, directory submission became the first scalable, outsourceable link-building service. Lists of “SEO directories” proliferated, many accepting any submission for a fee or in exchange for a reciprocal link. The tactic exhibits a pattern that repeats across eras: a link source whose value derives from editorial selectivity attracts imitators that preserve the form (a categorized list of links) while discarding the selectivity. By the mid-2000s, most generic directories had lost any ranking value they once held, because search engines learned to discount sites whose primary function was to host outbound links with no meaningful screening.

The directory era makes a point that is easy to forget. Early link building was not, at its core, deceptive. A site owner who wanted a listing in the Yahoo! Directory or DMOZ was seeking a legitimate editorial endorsement. The deception entered when imitators sold the appearance of endorsement without the substance.

3.4 Reciprocal Links, Web Rings, Link Farms and Blogrolls

If directories were the editorial channel, reciprocal linking was the peer-to-peer channel. A site owner would email another with a proposal: link to me and I will link to you. The practice drew on the older social norms of the Web, in which linking to related sites was both courtesy and navigation, and on the earlier institution of web rings, circular chains of topically related sites. WebRing, founded in 1994 and later acquired by Yahoo!, popularized the model.

Link popularity. By the late 1990s, “link popularity,” the number of inbound links to a site, was an established concept in SEO. Before PageRank, some engines (notably Inktomi and, to a degree, AltaVista) took link counts into account, and after Google’s rise the measure became universal. The simplest strategy was to increase the count, regardless of source. The distinction between a link from an authoritative page and a link from any page, which PageRank in principle encoded, was widely understood by professionals but unevenly applied in practice, because the tools for measuring it were crude and because quantity was, at that time, reliably rewarded.

Reciprocal and three-way linking. Mutual links were easy to arrange and, at moderate scale, difficult to distinguish from the natural linking of related sites. Link-exchange programs, “links pages” and “resources” pages proliferated. A more sophisticated variant was the three-way link (site A links to B, and C links to A), designed to avoid detectable reciprocity. The graph-theoretic response was straightforward. Reciprocal pairs and short cycles are detectable, and a high fraction of reciprocated links is statistically unusual. Google’s early guidelines advised against participating in link farms and “bad neighborhoods,” and the language of its guidelines on link schemes, though it changed over time, retained this core.

Link farms and free-for-all pages. At the manipulative extreme were link farms, networks of pages or sites created solely to interlink, and free-for-all (FFA) pages, which accepted automated link submissions from anyone. Their effect on PageRank was limited, because the farms had little inbound authority from the broader Web to redistribute. Nevertheless, they demonstrate that, as soon as links had value, an infrastructure of link production emerged. Gyöngyi and Garcia-Molina (2005) would later model such structures formally as “spam farms” in which boosting pages funnel rank to a target page.

Blogs, blogrolls and the social Web. From 1999 onward, weblogs, and the platforms that made them easy to run (Blogger in 1999, Movable Type in 2001, WordPress in 2003), changed the shape of the link graph. Blogs were updated frequently, linked liberally to each other and to external sources, and often included a blogroll, a permanent list of recommended blogs in the sidebar, repeated across every page of the site. Blogrolls were a form of social endorsement. They were also a pattern of site-wide links (a single link appearing on thousands of pages), which later became a classic indicator of manufactured or paid linking. Google’s 2003 acquisition of Pyra Labs, the owner of Blogger, reflected the company’s recognition of the importance of this content. Technorati, founded in 2002, indexed the blogosphere’s link structure explicitly.

Google bombing. The blog era also yielded the most vivid demonstration of the power of anchor text: the Google bomb. In 2001, Adam Mathes documented that coordinated linking from many pages to a target with a particular anchor phrase could make the target rank for that phrase even if the phrase did not appear on it. The best-known cases, such as the 2003 campaign that associated a derogatory phrase with a biography page of a sitting US president, drew media attention. Google responded only in 2007, with an algorithmic change that reduced the effect. For this paper, the Google bomb has analytic value because it isolates the anchor-text mechanism: a modest number of coordinated links with identical anchors, with no relevance to the target’s content, could dominate the ranking for a low-competition phrase. The same mechanism, applied commercially, would drive the exact-match anchor text tactics of Era II.

Comment and guestbook spam. The openness of blogs and guestbooks created the opposite problem: anyone could leave a link. Automated posting of links in comments and guestbooks began in the early 2000s, and it foreshadowed the large-scale automation of the following period.

3.5 Algorithmic Responses: Hilltop, Local Interconnectivity and the Florida Update

By 2002 and 2003, the volume of manipulative linking had grown enough to demand more than manual action. The search engines’ responses were to refine the question the link graph was supposed to answer.

Hilltop and expert pages. Bharat and Mihaila (2001) proposed Hilltop, an approach that first identifies a set of “expert pages,” documents with many outlinks to topically related, non-affiliated hosts, and then ranks target pages according to the links they receive from those experts for the query’s topic. Hilltop addressed two weaknesses of simple PageRank. First, it required topical relevance between linking and linked pages. Second, it required the independence of the linking host, by discounting links between affiliated hosts (the same owner or the same network). Google acquired the rights to the technique in 2003, and Krishna Bharat, its co-author, joined the company that year. The degree to which Hilltop was incorporated into production ranking cannot be established from public sources, but its principles (expert sources, non-affiliation and topical relevance) are visible in nearly every later link-quality criterion.

Local interconnectivity. A related Google patent, associated with Bharat and often called “LocalRank” (US 6,526,440), described re-ranking an initial result set on the basis of the interlinking among the results themselves. The idea was that a page linked to by other highly ranked pages for the same query is more likely to be relevant than one linked to by pages unrelated to the query. This is a query-dependent refinement of global link authority.

Named updates and the “Google Dance.” Through 2002 and 2003, Google updated its index on a roughly monthly cycle, which the SEO community called the “Google Dance” because rankings fluctuated while the new index propagated. Updates were named by participants on forums such as WebmasterWorld, in the manner of weather events. Boston (early 2003), Cassandra (April 2003, associated with the devaluation of hidden text and links and large-scale cross-linking among sites owned by the same party), Dominic (May 2003) and Esmeralda (June 2003) were successive changes. Fritz (July 2003) moved Google to an incremental, near-continuous index update, effectively ending the monthly dance.

Florida. In November 2003, the Florida update produced the first dramatic and widely publicized ranking upheaval. Many established commercial sites, particularly those that had optimized heavily for competitive keywords, lost rankings abruptly in the middle of the holiday shopping season. Affiliate and keyword-stuffed sites were hit especially hard, and the effect on small e-commerce businesses was severe. The SEO community responded with intense debate over the cause. Hypotheses included an over-optimization filter, a stemming or semantic expansion of query terms, and the incorporation of Hilltop- or LocalRank-style relevance mechanisms. Google did not publish an official explanation, and the causes remain a matter of inference.

Florida is a watershed in this history for three reasons. First, it showed that a single algorithm change could abruptly eliminate the revenue of businesses that depended on the search engine’s current behavior, which created the sustained mutual suspicion between Google and the SEO industry. Second, it demonstrated a decisive shift in philosophy: the engine would no longer reward whatever signal was easiest to produce, but would try to model naturalness, treating signals that looked artificially optimized as evidence against a page. Third, it marked the end of the era in which link quantity could be treated as a reliable monotonic lever.

3.6 The Economics of the Early Link Market

A complete account of Era I must consider incentives. Three structural conditions explain why link building became a commercial activity.

1. Search traffic had measurable commercial value. The introduction of pay-per-click advertising, notably Google AdWords (2000), gave every query a price. Organic rankings on the same page were therefore a free alternative to a priced good, and the price of the paid slot indicated the value of the organic one.
2. Monetizing content became simple. AdSense (2003) allowed any publisher to earn revenue from traffic. This produced “made-for-AdSense” sites designed solely to attract search traffic and display ads, many of which relied on scraped content and artificial links. It substantially increased the demand for rankings by low-quality content.
3. Link value was observable. Toolbar PageRank (2000) and the link query operator gave practitioners a feedback loop for experimentation. In an adversarial environment, observability benefits the manipulator more than the defender, because the defender must protect the signal while the attacker only needs to find a profitable regularity.

By 2003, link buying and selling had begun. Early brokers and text-link marketplaces appeared, and by the middle of the decade the practice had grown large enough to become a policy issue (Section 4.4). The first generation of link builders were, in effect, publicists, directory submitters and webmasters writing emails. They worked manually, at small scale, with relatively little concern for detection. Their successors would work with software.

3.7 Assessment of Era I

Era I established the conceptual framework within which all later link building operates. Four conclusions follow from the evidence reviewed above.

1. The unit of value was the link, and its dominant dimension was quantity. PageRank in principle weighted links by source authority, but the weight was a global scalar, not a topical, contextual or trust-based judgment. The practical rule for a practitioner was: more links from more domains, and especially from higher-PageRank domains, produce better rankings.
2. The link’s two functions, vote and label, were fused. Anchor text was both a descriptor and a signal of endorsement, which made the descriptor easy to counterfeit. The Google bomb exposed this fusion before the industry exploited it.
3. The principal defenses were structural and heuristic. Detection of reciprocal links, devaluation of link farms, and the beginnings of source-topic and host-affiliation analysis (Hilltop) were the principal tools. Trust propagation, spam classification by machine learning and link attributes had not yet appeared.
4. Information asymmetry favored the optimizer. Because Google published the PageRank concept and exposed toolbar values, while practitioners could observe the effects of their own experiments, the early period was a time of unusually transparent ranking logic, and the cost of manipulation was low.

The next era would see the cost of manipulation fall further through automation, and the search engines’ defenses become more systematic in response.

4. Era II: Industrialized Manipulation and First-Wave Defenses (2004–2011)

4.1 Context: Scale, Automation and the Professionalization of SEO

Between 2004 and 2011 the Web’s economics and tooling changed in ways that transformed link building from a craft into an industry. Broadband adoption, the spread of content management systems and the growth of user-generated platforms multiplied the number of places where a link could be placed. The SEO profession consolidated around trade conferences (Search Engine Strategies, PubCon), forums (WebmasterWorld, Digital Point, later Warrior Forum and BlackHatWorld), and agencies that sold ranking improvements as a service. Google, for its part, became a public company in 2004, and its dominance in search made it the effective regulator of web visibility.

Two developments define the era. The first is automation: the cost of producing a link fell toward zero for anyone willing to use software. The second is the institutionalization of defense: for the first time, Google, Yahoo! and Microsoft publicly adopted policies, new HTML attributes and academic research programs specifically aimed at link spam. The era is therefore best read as an arms race with an explicit, public front line.

4.2 Tools of Industrialization

The defining instruments of Era II were programs that automated the creation and placement of links. It is worth describing them precisely, because they explain both the scale of the spam problem and the nature of the defenses that followed. The tools named here are cited as historical facts about the ecosystem, not endorsed.

Forum, comment and guestbook posters. XRumer, first released around 2006 by a developer group associated with BotmasterLabs, became the most notorious. It automated registration and posting on forums, wikis, blogs and guestbooks, solved or circumvented CAPTCHAs of the time, and spun message text to avoid duplicate-content detection. Its output was profile pages and posts containing signature links across tens of thousands of sites. Other tools handled comment posting at scale, and ScrapeBox (2009) combined a harvesting component, which found sites by footprint (identifying text patterns of a platform), with an auto-poster. Comment spam on blogs became a crisis severe enough that Akismet (2005) and similar services emerged specifically to filter it.

Article spinning and article directories. A large class of link builders produced low-cost articles, often by taking one source article and “spinning” it into hundreds of superficially distinct versions by substituting synonyms and rearranging sentences, then submitting them to article directories such as EzineArticles (founded 1999) in exchange for a keyword-anchored “author bio” link. Article directories were a rational response to the observation that user-generated content sites accepted links in bulk. They became a major source of near-duplicate content and were an obvious target for the quality-focused Panda update of 2011.

Web 2.0 properties, social bookmarking and profile links. Free blogging platforms, social bookmarking sites, and profile pages on forums and community sites could be created at will and used to host links or short articles. “Link wheels” and “link pyramids,” tiered structures in which low-quality properties linked to higher-quality ones that in turn linked to the money site, were marketed as methods of laundering low-quality links into authority.

Blog networks. Rather than rely on third-party sites, some operators built their own networks of sites (later called private blog networks, or PBNs) that existed to link to client sites. Early blog networks of the late 2000s, sold as subscription services, were a more sophisticated and more detectable evolution of link farms.

Directory and bookmark submission software and link-building suites. A large category of software (including tools such as GSA Search Engine Ranker, which appeared in the late 2000s) combined target discovery, account creation, content generation and submission across many platform types. It systematized what had been a manual process.

The aggregate effect was a tiered link ecosystem. At the bottom was a mass of automated, near-worthless links, which in the aggregate still moved rankings for non-competitive queries. At the middle were semi-manual guest posts, article placements and directories. At the top were editorial placements, rare and expensive. The relative returns of the tiers were what the search engines needed to change.

4.3 Exact-Match Anchor Text and the Anatomy of the Dominant Tactic

The single most characteristic practice of Era II was the engineering of exact-match anchor text. The logic followed directly from Brin and Page’s (1998) design. If a link’s anchor text labels its target, then the way to rank for “cheap hotels in Paris” is to build many links from different domains whose anchor is “cheap hotels in Paris.” The Google bomb had proven that the mechanism worked in the absence of any other relevance signal. Practitioners learned that, for commercial queries, an anchor profile dominated by the target keyword correlated strongly with rankings.

The tactic had three attractive properties. It was controllable, because the link builder chose the anchor text. It was scalable, because forum signatures, article bios, directory descriptions and blog comments all allowed custom anchors. And it was measurable, because rankings for a target phrase moved visibly in response.

Its weakness, which Google exploited in Era III, is that natural anchor profiles do not look like this. Real-world links to a commercial site are dominated by the brand name, the bare URL, generic phrases (“click here,” “this site”) and varied descriptive text. A profile with a high concentration of identical commercial anchors, drawn from low-quality sources, is a statistical outlier and easy to flag. The practice was, in other words, effective only for as long as the search engine did not measure its own input’s distribution. Era III would show what happened when it began to.

4.4 Paid Links and Google’s Policy Response

Link buying moved from a fringe activity to a large market in this era. Text-link marketplaces connected publishers who wished to sell links on established sites with advertisers who wanted the PageRank. Prices were often explicitly tied to toolbar PageRank, which exposed the market’s logic: the link was being sold as a ranking input, not as advertising.

Google took the position that links intended to manipulate PageRank, as opposed to links that advertise or editorially recommend, violated its guidelines. In 2007 its webspam team, led by Matt Cutts, publicly stated its approach to paid links, introduced a form for reporting them, and began demoting the toolbar PageRank of sites caught selling. The enforcement was visible, but it was limited, because distinguishing a paid link from an editorial one at scale is difficult without the seller’s disclosure. Google therefore relied on a mixture of detection of selling patterns (sponsored sections, networks of sites sharing templates), user reports, and manual review.

This period fixed a distinction that persists to the present: links that are editorially given versus links that are compensated for their ranking effect. The distinction is a policy distinction, and its enforcement depends on the search engine’s ability to infer intent from outcomes. The introduction of nofollow in 2005 offered the first technical mechanism for publishers to signal that a link should not be treated as an endorsement.

4.5 The Introduction of nofollow (2005)

On 18 January 2005, Google announced, jointly with Yahoo!, MSN Search and several blogging platforms, a new value for the HTML rel attribute, rel="nofollow", which instructed search engines not to credit the link as a ranking vote (Google, 2005). The initial motivation was the control of comment spam: if comment links no longer passed ranking value, the incentive to spam comments would, in theory, disappear. Blogging platforms, including Blogger and WordPress, quickly applied the attribute by default to comment links.

The design is notable for its simplicity and for its strategic implication. It acknowledged, in the form of a standard, that the open link graph could not be trusted unconditionally, and it gave the publisher, rather than the search engine, a means of declaring a link’s non-editorial status. That was a shift of responsibility.

Effects and side effects.

1. Comment spam did not disappear. Spammers continued to post in volume, partly because their economics did not depend on any single link counting, and partly because the effect of nofollow was not uniformly known. The volume of spam, though, did decline relative to what it would otherwise have been, and the attribute became a permanent part of web publishing.
2. PageRank sculpting. Some practitioners, believing that PageRank was divided among a page’s outlinks, used nofollow on internal links to concentrate internal authority on preferred pages. In 2009, Google’s Matt Cutts explained that nofollowed links still consumed a share of a page’s outbound rank, so that sculpting no longer conserved equity (Cutts, 2009). The episode illustrated the difficulty of publishing implementation details: public statements about behavior can retroactively invalidate tactics built on earlier statements.
3. Reduced link supply. As large social and content platforms adopted nofollow on user-generated links, an increasing share of links to commercial sites came from sources that transferred no PageRank. The link graph became, in effect, bifurcated into a “followed” core and a “nofollowed” periphery, with consequences for how practitioners valued platform links.
4. The signal moved. Over time, nofollow links on authoritative sites came to be recognized as still valuable for traffic, discovery and brand exposure. Google formally redefined the attribute in 2019 as a “hint” rather than a directive (Section 6.2).

4.6 The Jagger Update (2005) and Big Daddy

The autumn of 2005 brought the first update broadly understood as an assault on poor-quality link building. Jagger rolled out in three phases from roughly September to November 2005. SEO practitioners’ analyses at the time converged on its targets: reciprocal link exchanges, link farms, and paid or purchased links. Sites that had built their rankings on large numbers of low-quality, reciprocal or site-wide links saw heavy drops, while those with fewer links of higher editorial quality were relatively unaffected or gained.

Google did not publish a formal explanation of Jagger, but its spokespersons acknowledged the broad objectives in blog posts and conference remarks, and the contemporaneous commentary by Cutts described changes to how links from sites with poor quality were evaluated. The update has since served as the usual reference point for the beginning of systematic link-quality evaluation in production ranking.

Big Daddy (rolled out between late 2005 and early 2006) was principally an infrastructure update that changed crawling, URL canonicalization and the handling of redirects. It is relevant here because Cutts explained that the number of pages crawled and indexed on a site depends partly on the PageRank of the site and the quality of its inbound links. Link quality therefore influenced not only ranking but indexation, a connection that remains important today.

Between 2005 and 2011 the broader pattern was that Google applied successive refinements of link evaluation without announcing most of them. Austin (2004) and Brandy (2004) were seen as adjustments to on-page and anchor-text relevance, and the “sandbox,” a hypothesized period during which new domains were suppressed in competitive results, became a widely discussed phenomenon, though Google never confirmed it as a deliberate mechanism and suggested that it might be an emergent side effect of how ranking signals were combined. The evidence is thus indirect, but the practical effect on practitioners was real: new sites could not rely on a short burst of links.

4.7 TrustRank and the Academic Defense Program

The scientific counterpart to these practical measures was a body of research on web spam, much of it conducted by or in cooperation with the search engines.

TrustRank. Gyöngyi, Garcia-Molina and Pedersen (2004), at Stanford with Yahoo!, proposed TrustRank as a response to the vulnerability of PageRank to link spam. The key observation was that good pages rarely link to bad ones, whereas bad pages often link to good ones in an attempt to appear legitimate. TrustRank begins with a small seed set of pages manually judged to be reputable (for example, well-known institutions), assigns them initial trust, and propagates that trust along outlinks with attenuation. A page’s trust score therefore declines with its link distance from the seed set. The method required human review of only a small number of seeds, scaled to the full graph, and had a clear interpretation: spam pages end up far from trusted sources. It is an application of the principle introduced in Section 2.2 (recursive authority) with an added assumption (approximate isolation of spam from the trusted core). The paper reported effective separation of good and bad pages in experiments on Web samples.

Counterparts and extensions. Krishnan and Raj (2006) introduced Anti-TrustRank, which propagates distrust from a set of known spam pages backward along inlinks. Benczúr and colleagues proposed SpamRank, which examines the distribution of PageRank contributions to a page and flags unnatural sources. Wu and Davison (2005) developed methods to identify link farms through the detection of dense bipartite structures. Fetterly, Manasse and Najork (2004) showed that spam pages could be identified by statistical outliers in properties such as duplicated content and link degree distributions. Becchetti and colleagues (2006) combined link-based features (such as degree, PageRank, and TrustRank) in a machine-learning classifier. Gyöngyi and Garcia-Molina (2005) provided the taxonomy that organized this work.

Significance for link building. TrustRank’s legacy for practitioners was a mental model: distance from trusted sources matters. Though Google has never confirmed using TrustRank itself, the principle that links from sites close to authoritative seeds, such as universities, government sites, major publications and recognized institutions, are worth more than a larger number of links from the periphery became the working premise of high-end link building. It is also embedded in third-party “trust” metrics that appeared later (Section 5.4). In 2024 and 2025, evidence from antitrust proceedings reportedly indicated that Google still uses a PageRank-derived measure interpreted as distance from known good sources, though public descriptions are brief and incomplete (see Section 7.2).

4.8 Brand, Relevance and Quality Signals Begin to Appear

During the second half of the era, Google began to incorporate signals that were not link counts and that foreshadowed the trajectory toward entity authority.

Universal Search (2007) blended news, images, video and local results into the main results page, reducing the relative prominence of the ten blue links. The Vince update (February 2009) was widely interpreted as favoring established brands on competitive generic queries, and Google’s then-CEO Eric Schmidt remarked publicly in 2008 that brands were a way to sort out the “cesspool” of the Internet, a statement that practitioners read as a signal of intent (the exact wording is paraphrased here). Caffeine (completed in 2010) rebuilt the indexing infrastructure for freshness. Google also deprecated the meta keywords tag as a ranking input, formalizing a recognition that self-declared metadata was unreliable.

The final and most significant of the period’s changes was the Panda update of February 2011, which targeted low-quality content, particularly “content farms” that produced large volumes of thin articles designed to rank for long-tail queries. Panda was not a link algorithm, but it mattered for link building in two ways. First, it undermined the article-directory and spun-content ecosystem that supplied a large share of the era’s links. Second, it signaled that Google was prepared to build site-level quality classifiers and apply them across an entire domain, a model that would later apply to links.

4.9 Enforcement by Example: Public Penalties

Because Google could not penalize all link manipulation, it used exemplary public enforcement. In 2006, the German BMW site was temporarily removed from the index for using doorway pages, an action publicly discussed by Cutts. In February 2011, a New York Times investigation (Segal, 2011) reported that J.C. Penney’s strong rankings for a wide range of product queries had been supported by a large number of paid, keyword-anchored links on unrelated sites. Google applied a manual penalty, and the retailer’s rankings dropped substantially. Overstock.com was similarly penalized that year in connection with a promotion that solicited links with commercial anchor text from educational sites.

These cases are instructive for two reasons. First, they show that the brands most exposed to link-based penalties were those whose agencies purchased links at scale, a vulnerability that did not depend on technical sophistication. Second, they demonstrate that, at the end of the era, Google’s primary defense was still reactive. It could detect and punish egregious cases, but it lacked a systematic, scalable, automatically applied algorithm for devaluing unnatural link profiles. Penguin would supply that in 2012.

4.10 Measurement Infrastructure

Era II also saw the emergence of the data infrastructure that would define professional link building for the next decade. Yahoo! Site Explorer (launched in 2005) allowed anyone to query the inbound links of any URL, an unprecedented transparency; it closed in November 2011 after Yahoo! transitioned its search to Microsoft. Google’s webmaster tools (2006) provided site owners with data on their inbound links. Third-party crawlers filled the gap: Majestic, which grew out of a distributed crawling project beginning in 2004; SEOmoz’s Linkscape (2008), later Open Site Explorer, with its Domain Authority and Page Authority metrics; and Ahrefs, whose crawler appeared in 2011. These services created the first widely available independent link indices and gave practitioners comparative data on competitors’ backlink profiles. Competitor backlink analysis became standard practice, with the corollary that practitioners could replicate a competitor’s links directly, which tended to homogenize link profiles within verticals.

4.11 Assessment of Era II

The principal findings for Era II are as follows.

1. Automation collapsed the cost of link production. A signal that had been expensive enough to be informative (because it required human effort) became cheap, and its informational value decayed accordingly.
2. The anchor text became the main manipulation surface. The label function of links proved easier to counterfeit than the vote function.
3. Defenses diversified. The era introduced publisher-side attributes (nofollow), trust propagation (TrustRank and variants), statistical spam detection, exemplary manual penalties, and site-level quality classifiers (Panda).
4. Google’s enforcement remained incomplete. The algorithm could devalue many spam links, but a sufficiently large and diverse link profile could still produce rankings. The profitable equilibrium for black-hat practitioners persisted until a systematic penalty arrived.

The tension was reaching a limit. By 2011, the search results for many commercial queries were widely seen as degraded by link-driven manipulation, and Google was under public pressure to respond. Its response, in April 2012, ended the era.

5. Era III: The Algorithmic Reckoning and Quality Shift (2012–2017)

5.1 Context: From Devaluation to Punishment

If Era II was the era of cheap links, Era III is the era of expensive mistakes. The defining change was not that Google began to detect spam, which it had done for years, but that it began to punish sites for the links they had acquired, systematically and algorithmically. Previously, a webmaster who built poor-quality links risked the links being ignored. After 2012, the same webmaster could be demoted. This changed the risk calculus of the entire industry, and it made the quality of a link profile a strategic liability as well as an asset.

5.2 Google Penguin (2012–2016)

The first Penguin. On 24 April 2012, Google announced an algorithm change designed to decrease the rankings of sites that were violating its Webmaster Guidelines through manipulative tactics, which it described as “webspam” (Google, 2012a; Cutts, 2012). The change affected roughly 3.1 percent of queries in English, according to Google’s statement, a very large number for a single update. It was named Penguin by the SEO community, following the earlier Panda, and was understood to target two categories of violation: keyword stuffing and manipulative link schemes. The second category was by far the more consequential.

The targets were precisely those the preceding sections describe: unnatural link profiles dominated by exact-match anchor text, links from low-quality directories, article sites and blog networks, paid links, and large-scale link exchanges. Sites with over-optimized anchor distributions saw sudden and sometimes catastrophic traffic losses. The effect was not gradual. Because Penguin was run as a periodic recomputation, affected sites could not recover until the next refresh, even after cleaning their profiles.

Subsequent versions. Google updated Penguin repeatedly: Penguin 1.1 (May 2012), Penguin 3 (October 2012), Penguin 2.0 (May 2013, affecting approximately 2.3 percent of English queries by Google’s account), Penguin 2.1 (October 2013) and Penguin 3.0 (October 2014). The long intervals between the refreshes, sometimes more than a year, increased the penalty to affected businesses, and the SEO industry’s frustration with the system was a notable feature of this period. The last major step, Penguin 4.0 in September 2016, is treated in Section 6.1, because it changed the nature of the algorithm.

Manual actions. In parallel, the webspam team ramped up manual penalties. Webmaster Tools began notifying site owners of “unnatural links” actions, and a large wave of notices in the spring of 2012 caused widespread alarm. Google also took public action against its own properties: in 2012 it acknowledged that a marketing campaign for Chrome had used paid blog posts, and it demoted the Chrome landing page in its own results for a period. The action was interpreted as a demonstration of impartiality.

Why Penguin worked, and what it implied. From an information-retrieval standpoint, Penguin represented the shift from a positive-only link model (links can only help) to a signed model (links can help or hurt). The practical significance of that shift is large. A positive-only model allows an optimizer to experiment without risk, since a bad link is merely wasted. A signed model punishes experimentation, which in turn raises the value of caution, of expertise, and of tools that evaluate risk. The industry that grew to meet that need is the subject of Section 5.4.

5.3 The Disavow Tool and the Cleanup Economy

Because Penguin punished past link-building, webmasters needed a means of repudiating links they did not want or could not remove. In October 2012, Google introduced the Disavow Links tool in Webmaster Tools, announced by Matt Cutts at a trade conference. It allowed a site owner to upload a file listing URLs or domains whose links Google should ignore when assessing the site.

The tool’s introduction created a recognizable process: audit the backlink profile, identify toxic links, attempt manual removal by contacting webmasters, and submit a disavow file for those that could not be removed. A new service line, the link audit and “link cleanup,” arose almost overnight. Many agencies charged substantial sums for the work, and the volume of outreach emails asking webmasters to remove links reached the point of being a nuisance in its own right.

The tool also had an ambiguous status. Google described it as an advanced feature to be used with caution, and Google staff subsequently said that most sites did not need it. Practitioners nonetheless used it widely, sometimes disavowing very large numbers of links based on third-party “toxicity” scores that Google never endorsed. This is an example of a structural feature of the search-optimization market: because the engine does not disclose the model, the industry substitutes proxies, and the proxies can become authoritative in their own right.

The tool also made visible a problem that had previously been theoretical: negative SEO, the deliberate building of harmful links to a competitor’s site in the hope of triggering a penalty. Google’s position was that this was rare and that its systems were designed to ignore most such links, a position it strengthened in 2016 (Section 6.1). The disavow tool was part of the answer to the concern.

5.4 Link Risk Analysis and the Rise of Third-Party Authority Metrics

Penguin made link quality a first-order business problem and created demand for metrics. Between 2012 and 2017, a set of commercial metrics took on the role that toolbar PageRank had played in Era I. Google stopped updating the displayed toolbar PageRank value after December 2013 and removed it in 2016, which left a vacuum.

Moz Domain Authority (DA) and Page Authority (PA) predicted the likelihood that a domain or page would rank, on a 100-point logarithmic scale, using a machine-learned model of link features. Majestic Trust Flow and Citation Flow (introduced in 2012) were explicitly modeled on the TrustRank idea. Trust Flow measures a site’s link distance from a manually curated set of trusted seeds, while Citation Flow measures raw link volume. The ratio of the two was widely used as a heuristic for spamminess. Ahrefs Domain Rating (DR) and URL Rating measured the strength of a site’s backlink profile on the basis of the number and strength of the referring domains. LinkResearchTools (including its Link Detox product) and similar services offered risk scoring of individual links.

Two cautions apply. First, these metrics are models of Google’s behavior built by third parties. They are not Google signals, and Google staff have repeatedly said that Google does not use metrics such as Domain Authority. Second, the metrics had a self-fulfilling aspect. Because practitioners optimized for them, and because prices for placements were often quoted in terms of DA or DR, they created a new “currency” that could itself be manufactured, a Goodhart effect on the proxy. Link sellers advertised sites with inflated DA, some built by purchasing expired domains with existing link histories.

Despite these limitations, the metrics served a purpose. They gave the industry a shared vocabulary for link quality, and they allowed practitioners to prioritize prospects, to benchmark against competitors and to estimate risk. The theoretical lineage from TrustRank through Trust Flow to practical outreach prioritization is direct.

5.5 Semantic Foundations: The Knowledge Graph, Hummingbird and RankBrain

Penguin was the era’s most visible algorithm, but three less conspicuous developments reshaped the context in which link building operated and prepared the ground for Era V.

The Knowledge Graph (May 2012) was introduced by Google with the slogan that search should deal with “things, not strings” (Singhal, 2012). It was a database of entities (people, places, organizations and concepts) and relationships among them, and it was the beginning of Google’s explicit entity model. For the first time, Google was representing the world as entities rather than only as documents.

Hummingbird (announced in September 2013, deployed a few weeks earlier) was a rewrite of the core search algorithm to better interpret the meaning of whole queries, particularly conversational queries, rather than matching individual keywords. RankBrain (revealed in October 2015) used machine learning to interpret unfamiliar queries by mapping them to known concepts, and in 2016 a Google representative stated that links, content and RankBrain were the top three ranking signals, a statement widely cited by the industry. It confirmed that links remained a principal factor, but it also indicated that they were now one of several.

The significance for link building is cumulative. As search interpreted queries semantically and resolved entities, the context of a link became more interpretable, because the engine could better evaluate whether the linking page was about the same topic as the target. This laid the foundation for the relevance-focused practice of Era IV.

5.6 The Pivot to White-Hat Content Marketing

The strategic response of the legitimate part of the industry was to rebrand and reorient link building around content and outreach. The phrases “link earning” and “content marketing” gained currency, and the stated principle was that the best links were those that arrived because the content deserved them. In practice, the era’s methods were a mixture of editorial outreach and tactics that approached the boundary of policy. Several are described here.

Guest blogging. Writing articles for other sites in exchange for a byline link had been a standard tactic of Era II, and in the early 2010s it was the principal form of “white-hat” link building. As it was commoditized, with operators selling bulk placements of low-quality articles on any site that would accept them, Google acted. In January 2014 Matt Cutts declared in a widely read post that guest blogging done for SEO had become spammy, advising webmasters to stop. Google then penalized a prominent guest-blogging network, MyBlogGuest, a few weeks later. The episode demonstrated a principle that recurs throughout the history: any tactic that scales as a ranking lever will be treated as manipulation once scaled. The post’s distinction, between guest contributions for audience-building and exposure and guest contributions for links, remained fuzzy but influential.

Infographics and link bait. The visual summary of data, distributed for embedding, was another widely used technique, with an embed code that included a link back to the creator. Its overuse led to low-quality infographics and automated embeds, and Google’s guidance in 2013 on large-scale infographic distribution with keyword-rich anchors classified some such practices as link schemes.

Resource pages and broken link building. Resource-page outreach asked the curator of a list of links to add a relevant site. Broken link building identified dead outbound links on a page and offered a working replacement. Both are theoretically value-adding, because they improve the page, and they remain mainstream. Their quality depends on the relevance of the match.

The Skyscraper Technique. Popularized in 2013 by Brian Dean of Backlinko, the Skyscraper Technique had three steps: find content that already earned many links, create something better, and ask those who linked to the original to link to the new version instead. Its appeal was that it provided a repeatable, evidence-driven process. It also illustrates the arms-race logic in a new form. As it spread, the recipients of outreach were inundated with near-identical requests, and the technique’s conversion rates fell. Its enduring contribution was the principle of link prospecting from competitors’ proven link-earners, which remains in standard use.

Outreach at scale and tooling. Email outreach software, contact-finding tools and relationship-management systems standardized the process. The quality of outreach varied widely, and the practice of mass templated outreach produced the same saturation effect as other scalable tactics.

Private blog networks. Despite Penguin, PBNs persisted as a high-risk, high-reward tactic, using expired domains with existing link equity. Google periodically took action against network operators, deindexing identified networks, but detection of the networks relied on footprints such as shared hosting, shared ownership signals and link patterns, which sophisticated operators tried to avoid.

5.7 Mobile, Authorship and the Social-Signal Interlude

Two other developments are noted for completeness. Google Authorship, introduced in 2011 and discontinued in 2014, allowed authors to link their content to a Google+ profile, with byline photos in results. Its failure showed the difficulty of establishing author identity through a single platform, and its legacy is visible in later E-A-T work. Social signals were hypothesized by many practitioners to influence ranking. Google consistently denied using likes and shares as direct ranking signals, and the correlational evidence was ambiguous, since heavily shared content also tends to earn links. The period nevertheless drew attention to the idea that popularity on social platforms could be a precursor to link acquisition, which is where its actual value lay.

5.8 Assessment of Era III

The central lessons of Era III can be summarized as follows.

1. Links acquired a sign. After Penguin, a link could be a liability. This transformed link building into a risk-managed discipline, and it created the cleanup, audit and risk-scoring industry.
2. Proxies replaced the visible signal. With toolbar PageRank gone, third-party metrics (DA, DR, Trust Flow) became the working currency, and they inherited the problems of proxies.
3. Scalable tactics were consistently downgraded. Guest blogging, infographics and mass outreach each followed the same lifecycle: innovation, adoption, saturation, abuse and de-valuation or penalty.
4. The semantic foundations were laid. The Knowledge Graph, Hummingbird and RankBrain gave Google the ability to interpret context, which was a prerequisite for ranking by relevance rather than by volume.
5. The strategic rhetoric changed. “Earn links with great content” became the official philosophy, though it was often a euphemism for outreach.

Era IV began when Google moved from periodic punishment to continuous, real-time handling of links.

6. Era IV: Relevance, Editorial Integrity and Digital PR (2018–2022)

6.1 Penguin 4.0 and the Real-Time Turn

The final iteration of Penguin, released on 23 September 2016 and therefore straddling the boundary between Eras III and IV, changed the algorithm’s character in three ways that governed the later period. First, it became real-time: Penguin was incorporated into Google’s core ranking pipeline, so that its evaluation was updated as pages were recrawled and reindexed, rather than in periodic batches. This ended the long waiting periods for recovery that had defined the earlier versions. Second, it became more granular, affecting specific pages or sections rather than entire sites. Third, and most significantly, Google staff stated that Penguin now worked principally by devaluing spammy links, that is, by ignoring them, instead of demoting the sites they pointed to.

The third change was the most consequential in practice and the most contested in interpretation. If poor links were simply ignored, the incentive to disavow them declined, the risk of negative SEO fell, and the penalty for past mistakes became, in principle, small. Google representatives repeated this message in the following years, including the claim that Google could simply discount most spam links and that webmasters did not generally need to disavow. The industry’s reaction was mixed. Some practitioners continued to disavow extensively; others abandoned the practice. The evidence from practitioner experiments was inconclusive, in part because manual actions remained available for flagrant cases, so a risk of penalty for unnatural link patterns persisted even if the algorithmic mechanism had shifted.

The conceptual change reflects the progression described in Section 2.3. The search engine’s countermeasure moved from visible punishment to invisible neutralization. This is a strategically sound evolution: if a spam link is ignored, the spammer’s investment is wasted without a signal that would teach him what was detected. It also made link building harder to evaluate for practitioners, because failure took the form of absence of effect rather than the visible loss of rankings.

6.2 Link Attributes: sponsored, ugc and the Redefinition of nofollow

On 10 September 2019 Google announced two new rel values to supplement nofollow (Google, 2019a). rel="sponsored" was intended for links that are advertisements or paid placements, and rel="ugc" for links in user-generated content such as comments and forum posts. rel="nofollow" remained for cases in which a publisher did not wish to vouch for a link and did not fit the other categories. Critically, Google redefined all three as hints rather than directives for ranking purposes. From 1 March 2020 they would also be treated as hints for crawling and indexing. Google would be free to use such links as signals in discovery, understanding and, potentially, ranking.

The change had three implications.

1. A richer taxonomy of link intent. The publisher could now declare why a link was not an editorial vote, which gave Google labeled training data about the nature of links, and which refined its ability to model the editorial subset.
2. The reclassification of nofollow links as informational. The hint model meant that nofollow links could contribute to ranking in some circumstances, a possibility that practitioners had long suspected. This increased interest in links on platforms that applied nofollow universally, such as large news organizations, social platforms and encyclopedias.
3. Formalization of the disclosure norm. Google’s guidance made it explicit that paid links should be qualified. This closed the gap between policy and mechanism that had existed since 2007.

Compliance, though, was imperfect. A paid link without a qualifier is, by definition, an undisclosed one, and the sellers of such links had no incentive to comply. The attributes therefore functioned as a tool for the compliant and as a signal for enforcement against the non-compliant.

6.3 Topical Relevance and the Contextual Link

Era IV is the era in which topical relevance became the dominant qualitative criterion in professional link building. The idea was not new (Hilltop in 2001, topic-sensitive PageRank in 2002), but the evidence and the tools to apply it had matured.

The reasonable-surfer model. A family of Google patents, often referred to by that name (for example, US 7,716,225), describes assigning different weights to different links on a page according to the probability that a user would click them. Features such as the link’s position (a prominent in-content link versus a footer link), the font and visibility, the surrounding text, and the type of page could influence the weight. This contradicted the earlier equal-division model, in which every outlink on a page received an equal share of rank. Practitioners drew the inference that an in-content, editorially placed, topically consistent link was worth substantially more than a sidebar or footer link, consistent with the earlier decline of site-wide links. Whether and how the model operates in production is unknown, but its direction matches Google’s public advice.

Topical authority. As Hummingbird, RankBrain and later BERT (announced for Google Search in October 2019) improved Google’s understanding of language, the contextual fit between a link source and target became more assessable. Practitioners began to speak of topical authority: the idea that a site comprehensively covering a subject, and linked to by other sites about that subject, is seen as an authority on it. In practice, link prospecting changed from a search for high-DA sites in any field to a search for relevant sites in the target’s niche. Link buyers began to pay more for relevance than for raw metric strength.

Evidence and its limits. Industry studies published in this period, such as correlation analyses of millions of search results, found that pages ranking first for a query tended to have substantially more referring domains than those ranking lower (for example, Dean, 2020). These studies are consistent with the continued importance of links but cannot establish causation. Top-ranked pages attract links because they rank, and they rank for many reasons. This caveat applies to nearly all practitioner evidence on link importance in the period.

6.4 E-A-T and the Quality Rater Framework

In parallel with link-algorithm developments, Google’s Search Quality Rater Guidelines gained prominence. The guidelines are instructions for the human evaluators who assess search results for quality, and they provide an indirect window into the properties that Google’s engineers wish their ranking systems to approximate. Their central concept in this period was E-A-T, Expertise, Authoritativeness and Trustworthiness. It was applied with special rigor to “Your Money or Your Life” (YMYL) topics, such as health, finance and safety, where inaccurate content could harm users.

In August 2018 a broad core update, nicknamed “Medic” by the SEO community because it appeared to affect health and wellness sites heavily, brought E-A-T into practitioner awareness. Google’s later guidance on core updates directed site owners to the questions of expertise, authority and trust. Google staff were careful to say that E-A-T was not a single “score” or ranking factor, but a framework describing what its systems tried to reward, and that its systems used a variety of signals as proxies for it. The guidelines specifically cited reputation as established by independent sources, such as news articles, reviews and recommendations from experts, as evidence of authoritativeness.

The relationship between E-A-T and links is therefore indirect but significant. Links, especially from recognized authorities, were an obvious proxy for authoritativeness, and Google stated that PageRank-based signals were among the means by which its systems identified authoritative content. The difference from earlier eras is that link evidence was now embedded within a broader view of reputation that included who the author was, what independent sources said about the site and its authors, and whether the content reflected real expertise. Link building that ignored this wider reputation context became less effective, and the successful practitioners began to think in terms of reputation building.

6.5 The Rise of Digital PR and Data Journalism

The practical culmination of the shift was the rebranding of high-quality link building as digital PR. Its distinguishing characteristics were as follows.

Newsworthiness as the qualifying criterion. Rather than ask for a link, the digital PR practitioner produced something journalists wanted to cover: original research, surveys, proprietary data analysis, expert commentary on current events, or creative campaigns. The link was a by-product of editorial coverage. The rationale was that links from news publications and major media were exactly the type of high-authority, editorially given, topically coherent link that Google’s systems were designed to reward, and that were hardest to counterfeit at scale, because they required a real editor’s decision.

Data journalism. Many campaigns used datasets, public or proprietary, to produce rankings, maps, indexes and trend analyses suited to journalistic reuse. The approach exploited the fact that journalists regularly need data and that regional and niche publications had limited resources to produce their own. It also posed quality risks: surveys with unrepresentative samples, rankings built on arbitrary methodologies, and “data” designed to generate a headline rather than to be accurate. The ethical dimension is taken up in Section 9.

Reactive PR and expert sourcing. Platforms connecting journalists with expert sources allowed practitioners to respond to specific journalist queries. The best known, Help a Reporter Out (HARO), had been founded in 2008 by Peter Shankman and later acquired by commercial media-database companies. It became the standard source for reactive link building throughout the 2010s and early 2020s. Its value, and its decline, illustrate the saturation dynamic once more: as practitioners flooded journalists with generic pitches, many journalists stopped using the service, and its owner reportedly changed its branding and ultimately discontinued it in the mid-2020s, after which a number of alternative platforms emerged. The principle of expert sourcing persisted, but the channels fragmented.

Measurement. Digital PR was assessed by coverage, referring domains and the authority of linking publications, and, increasingly, by unlinked mentions. This last metric reflects a recognition that earning a mention without a link still had value, an early step toward the entity-based view discussed in Section 7.

Skepticism toward link building as a label. Part of the digital PR movement was explicitly positioned against the industry’s reputation for manipulation, with practitioners arguing that the term “link building” connoted bulk tactics. The reframing had a credible basis, because the criteria (editorial discretion, relevance, newsworthiness) align with those the search engine rewards, but it also served a marketing purpose, and some “digital PR” services were simply guest posting rebranded.

6.6 The Persistent Gray Market: Sponsored Content and Link Marketplaces

Beneath the legitimate surface, a large market for sponsored posts and link placements persisted. Marketplaces listed publishers by metrics such as DR, traffic and topical category, and sold “guest post” placements at fixed prices. Many such placements were paid and undisclosed. Google’s policy was unambiguous that such links, if intended to manipulate ranking, violated its spam policies, but enforcement was uneven, and practitioners debated how much Google could in practice detect. The structure of the market is analyzed in Section 8 in terms of information asymmetry. The buyer cannot easily verify a seller’s claims about traffic and quality, and the seller cannot easily verify that the buyer will not be harmed by Google’s response.

During the same period, a parallel phenomenon emerged: parasite SEO, the use of the authority of a large, trusted domain to host low-quality or promotional content that would rank because of the host’s reputation. Google’s later response, the site reputation abuse policy of 2024, is discussed in Section 7.

6.7 Core Updates, Helpful Content and the Convergence on Quality

From 2018 onward Google issued broad core updates several times a year, each of which re-weighted a large set of signals and caused ranking shifts. It stated that there was no specific “fix” for sites affected, other than improving overall quality. In August 2021 and later, Google introduced a Product Reviews system that rewarded in-depth, first-hand evaluation, and in August 2022 the helpful content update introduced a site-wide classifier designed to demote content written primarily for search engines rather than for people. Together these measures moved the center of gravity of SEO away from individual signals and toward holistic quality assessment.

For link building, the implications were as follows. Links remained important, but they were no longer the only means of demonstrating authority, and they were assessed in a context of site quality. A link to a low-quality site conferred less than a link to a high-quality one, and the quality of the target site, assessed by systems that included human-rated examples, mattered for how much benefit it received from links. The link ceased to be an independent input and became a component of a composite judgment.

6.8 Assessment of Era IV

The characteristic features of Era IV can be summarized as follows.

1. From punishment to neutralization. Penguin 4.0 and the hint model of link attributes made Google’s handling of low-quality links mostly invisible, which devalued the feedback available to manipulators.
2. Relevance and context replaced quantity as the leading criteria. The reasonable-surfer logic, topical authority and the quality-rater framework favored contextual, editorial links.
3. Link building was absorbed into PR. The best-performing tactic was to produce something worth covering. The skills required changed from technical to editorial and relational.
4. Links became one component of reputation. E-A-T and the later E-E-A-T framework embedded links in a broader assessment that included authorship, independent reputation and the content’s demonstrable quality.
5. Markets for undisclosed paid links persisted. The tension between policy and practice was unresolved, and it fed into the next era’s emphasis on machine-learning spam detection.

The following period brought a qualitatively different change: the dominant interface of search began to shift from a list of links to a generated answer.

7. Era V: AI Search, Entity Authority and Linkless Signals (2023–Present)

7.1 SpamBrain and the Machine-Learned Defense

Google’s AI-based spam-prevention system, SpamBrain, was first described publicly in the context of its 2018 origins and the company’s annual webspam reporting. It was not a single algorithm but a system for detecting several categories of spam, including hacked sites, scraped content, and link spam. Google’s webspam report for 2021 stated that SpamBrain had detected a substantially larger number of spam sites than at its launch, though the precise figures should be verified against the original document.

On 14 December 2022 Google announced a link spam update that used SpamBrain to neutralize the effect of unnatural links on rankings (Google, 2022a). Google stated that the system was designed to detect both sites buying links and sites used to pass outgoing links, and that its effect would be to nullify the value of the unnatural links, not necessarily to demote the sites involved. The announcement is noteworthy for what it says about the evolution of the defense. It is a learned classifier, applied continuously, that identifies link-spam patterns rather than relying on enumerated rules.

The shift from rules to learned classifiers has analytic implications. A rule-based filter can be probed and circumvented, because its logic can in principle be inferred. A learned classifier trained on examples of spam, including those discovered by manual review, can generalize to variants not yet seen, and it can be retrained as tactics evolve. It does not, however, eliminate the arms race, since adversaries can also use machine learning to generate content and links that mimic natural patterns. Generative AI sharply reduced the marginal cost of producing plausible-looking article text, which fueled a surge in AI-generated content on sites built to host links and to monetize traffic.

Google’s responses in 2024 were policy-based as well as algorithmic. In March 2024 it announced updates to its spam policies targeting three practices: scaled content abuse (producing large amounts of low-value content, however generated, primarily to manipulate ranking), site reputation abuse (hosting third-party content on an authoritative site primarily to exploit that site’s ranking signals, the parasite-SEO model), and expired domain abuse (purchasing expired domains to leverage their residual reputation). Manual enforcement of the new policies began in May 2024. Each of the three maps directly onto a historic link-building tactic. Scaled content abuse is the successor to article-spinning, expired domain abuse to PBNs built on aged domains, and site reputation abuse to the long practice of renting authority.

7.2 What the Leaks and Litigation Revealed

Two public disclosures in 2024 enlarged the evidentiary base for understanding Google’s current systems, and both bear on links.

In May 2024, documentation of an internal Google Content Warehouse API was made public, and practitioners analyzed it in detail. Google confirmed that the documents were authentic but cautioned against drawing conclusions without context, noting that they might be outdated and incomplete. The documentation described numerous attributes, including some associated with link processing and with site-level and author-level signals. Analysts concluded that links remained present in the data model, with indications of link-quality tiering, and that some attributes appeared related to source trust and to how a link’s position or freshness was recorded. The documentation, however, does not specify weights, and the inference to current ranking behavior is uncertain.

The second source is the United States antitrust case against Google concerning search. In August 2024 the trial court found that Google had unlawfully maintained a monopoly in general search. Trial testimony and exhibits described aspects of Google’s ranking infrastructure, including systems that use aggregated user interaction data (such as the system referred to as Navboost) and language-model-based retrieval components. Some exhibits reportedly described PageRank as continuing to be used as a measure of a page’s distance from known reliable sources, which is consistent with the TrustRank-style premise discussed in Section 4.7. The record underscores that the link graph is now one part of a larger system that includes behavioral and learned signals, and that the weighting of links relative to these other signals cannot be established from public materials.

The methodological caution of Section 2.5 applies with particular force. The disclosures confirm the existence of signal families and architectural features. They do not support claims about the exact weight of backlinks in current ranking.

7.3 From E-A-T to E-E-A-T and the Rise of Entity Authority

In December 2022 Google added a second “E,” for Experience, to E-A-T, and it declared trust to be the most important member of the set. The addition recognized that first-hand experience with a subject (having used a product, visited a place or lived through a situation) is a distinct and valuable quality. The change fits a long trajectory, from the Knowledge Graph of 2012 onward, in which Google models the world through entities and attempts to assess the reputation of the entities that produce content.

Entity authority in this context means that a brand, organization or person is recognized as a distinct, consistent and reputable entity associated with specific topics across many independent sources. It is established through a combination of signals:

1. Consistent identity across the web. The same name, description and attributes appear in structured data (such as schema.org markup with sameAs references), in knowledge bases such as Wikidata and Wikipedia where eligible, in business profiles and in authoritative directories.
2. Independent corroboration. Coverage by publications, citations by experts, reviews, awards and references in scholarly or professional works that are independent of the entity itself.
3. Topical association. The entity is repeatedly mentioned alongside particular topics, products or problems, so that it is associated with those topics in the corpus. This is a co-occurrence signal.
4. Author-level credentials. For individuals, verifiable expertise, publication histories and professional affiliations.
5. Link evidence. Links from recognized sources remain one of the clearest forms of corroboration, but they are interpreted within the larger identity and reputation profile.

The brand co-occurrence concept deserves separate discussion. Practitioner literature has argued for many years that unlinked brand mentions and the co-occurrence of a brand with topical terms act as ranking signals. The patent literature gives some support for a general idea of “implied links,” in which references to a site that are not hyperlinks can contribute to ranking signals, as discussed by commentators such as Slawski (2013), but the extent of use in production is unknown. Whatever the situation for traditional search, the argument has strong support for language-model-based systems, which learn associations between named entities and topics from text, without needing hyperlinks at all.

7.4 The Generative Interface: AI Overviews, AI Mode and Chat-Based Search

The most important structural change of the period is the change in the interface of search. Several developments, in rough chronological order, define it.

• ChatGPT (November 2022) demonstrated the appeal of conversational question answering to a mass audience, and Perplexity (late 2022) built a search product around cited generated answers. Microsoft integrated OpenAI’s models into Bing (early 2023).
• Google announced its Search Generative Experience in May 2023 as an experiment, and extended generative summaries to general availability in the United States in May 2024 under the name AI Overviews. In 2025 it launched an AI Mode, a more conversational, multi-step search experience, in the United States.
• OpenAI added search capabilities to ChatGPT in late 2024, and other assistants developed web-grounded retrieval features.

These systems typically combine a language model with a retrieval step. Lewis et al. (2020) formalized the retrieval-augmented generation (RAG) approach, in which relevant documents are retrieved and supplied to the model as context for generating an answer. Retrieval relies on a traditional search index and ranking, and the sources it retrieves are then cited or summarized. For link building, two consequences follow.

First, traditional ranking remains upstream. Google’s own guidance on its AI features has indicated that standard SEO practices remain the foundation for appearing in them. Because many AI answers draw on pages that already rank, the signals that govern traditional ranking, including link-derived authority, still determine eligibility for citation in many cases. Links are thus still instrumental, but at one remove: they help a page get retrieved, and retrieval, not ranking position, is what matters for citation.

Second, the point of exposure moves from the click to the citation. A generated answer may satisfy the user’s query without a visit to any source. Pew Research Center (2025) reported that users were markedly less likely to click on traditional results when an AI summary was present. The value of a link as a referral channel therefore declines for informational queries, while the value of being named or cited in the answer increases. This dissociates two roles that a link played in Eras I through IV, the ranking vote and the traffic route, and it changes what link building optimizes for.

7.5 Generative Engine Optimization (GEO)

A new field, variously labeled generative engine optimization (GEO), answer engine optimization (AEO) or AI search optimization, emerged to address visibility in these systems. Aggarwal et al. (2024) introduced GEO as a research framework and benchmark, and reported that certain content modifications, such as adding citations, quotations from authorities and statistics, could increase the visibility of a source in generative engine responses by substantial margins in their experiments (on the order of tens of percent in some settings). The study’s value lies less in its specific percentages, which depend on the experimental design, than in showing that the content features that affect citation differ from the features that affect traditional rank, and are measurable.

Practitioner studies in 2024 and 2025 attempted to identify what correlates with brand presence in AI answers. Reports from SEO software vendors generally found that branded web mentions and brand-related signals correlated with visibility in AI Overviews and chat assistants more strongly than raw backlink counts, though such findings are correlational, vendor-produced and sensitive to sampling choices (for example, Ahrefs, 2025). They are best treated as hypotheses that agree with the theoretical expectation that language models learn associations between entities and topics from text.

Industry analyses of the sources that AI systems cite found frequent citation of community platforms such as Reddit and Quora, reference sites such as Wikipedia, video platforms, and established publishers, though the distributions varied by system and over time. The implication for strategy is that presence on platforms that language models and retrieval systems draw from is a source of visibility that does not depend on hyperlinks from conventional websites. This has led to a broadening of the scope of what practitioners do under the heading of link building to include citation building: securing references in sources that AI systems are likely to retrieve or have learned from.

7.6 Link Building for Language Models: Mechanisms

It is worth distinguishing the mechanisms by which an entity’s authority can come to be reflected in a language model’s output, because each corresponds to a distinct class of tactic.

1. Training-data inclusion. A model’s pretraining corpus is drawn from large web crawls, curated datasets, books, code and other sources. Documents that appear in widely used crawls (such as Common Crawl) and survive quality filtering contribute to what the model learns. Hyperlinks have long been used as a curation signal for such corpora. The WebText dataset used for GPT-2 consisted of pages linked from Reddit posts that had received at least a minimal level of approval (Radford et al., 2019), a use of links as endorsement that reflects the PageRank intuition in a new setting. Common Crawl itself publishes web-graph rankings, including PageRank-like and harmonic-centrality measures, which dataset builders can use to prioritize domains. A link from a widely linked source therefore can raise a page’s chance of entering a training corpus, though the mechanism is indirect and varies by dataset.
2. Retrieval-time selection. For RAG-style systems, retrieval uses search indexes, which means that classic link-based signals, relevance and freshness all influence which documents are fetched.
3. Entity association. Repeated, consistent mentions of an entity in conjunction with a topic strengthen the association in model weights and in retrieval embeddings. This is the mechanism behind co-occurrence arguments, and it is facilitated by coverage across diverse sources.
4. Structured knowledge. Entities represented in structured knowledge bases are more easily resolved and disambiguated.

Research on link-aware model training suggests that hyperlinks carry information that models can exploit: Yasunaga, Leskovec and Liang (2022), for example, showed that pretraining language models with document links as additional context improved performance on knowledge-intensive tasks. The finding supports the view that a hyperlink retains informational value for machine-learning systems even where it no longer functions as a click route.

7.7 Adversarial Pressure on Generative Systems

The arms-race logic of the earlier eras extends to generative systems. Early research shows that language-model-based search is vulnerable to manipulation. Kumar and Lakkaraju (2024) showed that adding optimized text sequences to a product page could increase the likelihood that an LLM-based recommender would highlight it. Nestaas, Debenedetti and Tramèr (2024) described “preference manipulation attacks” against LLM-powered search engines, using prompt-injection-like content on web pages to bias the system. Research on training-data poisoning indicates that a small, fixed number of malicious documents may suffice to implant unwanted behaviors in language models regardless of the model’s size, as reported in a 2025 study by Anthropic with collaborators. This area is moving quickly, and the specific results should be read as indicative.

These findings matter for the history of link building because they show that the structure of the problem is unchanged. Whenever a system assigns visibility on the basis of a signal that can be manufactured, a market for manufacturing it appears, and countermeasures follow. The 2020s version of the Google bomb is a coordinated effort to place content where models will find it. The countermeasures, by analogy, are likely to include source-reputation weighting, provenance tracking, classifiers for manipulative content, and an increasing emphasis on trusted-source seed sets, which is the TrustRank principle in new form.

7.8 The Changing Role of Links in Google’s Own Guidance

Google’s public communications about links have shifted in tone. In 2023, it published documentation of link best practices describing how to make links crawlable and how to write useful anchor text, and it has repeatedly advised that links should be earned through content worth referencing. Google representatives have, on a number of occasions, said that links matter less than in the early years and that the company needs fewer links than the SEO industry believes in order to understand a site, though these remarks are informal and have not been accompanied by quantification. At the same time Google has maintained that links remain a signal and has continued to prosecute link spam, which would be unnecessary if they were irrelevant. The reasonable reading is that link-derived authority is now one of many inputs, that its marginal contribution is higher for new or ambiguous entities and lower for well-established ones, and that the emphasis has shifted from the count of links to the credibility of the sources and the context of the citation.

7.9 Practice in the Present Era: What Link Building Has Become

In practice, the discipline in the mid-2020s is characterized by the following features.

1. Digital PR as the primary scalable white-hat method. Original research, expert commentary and newsworthy campaigns earn editorial coverage, which supplies links and, increasingly, mentions and citations.
2. Brand and entity building. Practitioners invest in consistent identity, structured data, author profiles, expert contributors, and presence on platforms relevant to the audience and to AI systems, together with monitoring of how AI assistants describe the brand.
3. Citation-oriented content. Content designed to be cited includes clear definitions, original data, quotable statements and authoritative sourcing, in keeping with the GEO research findings.
4. Selective, relevance-first link acquisition. Fewer, better links from topically relevant, editorially independent sources.
5. A continuing gray market, now with AI-generated content as the raw material of low-quality placements, and a continuing detection effort by SpamBrain and manual reviewers.
6. Measurement adaptation. New tools track share of voice in AI answers and the sources cited, though methods are immature and results are volatile because generative responses vary between runs.
7. Licensing and access as strategic variables. Publishers’ decisions about whether to allow AI crawlers, and their commercial deals with AI developers, affect which sources models can access. Infrastructure providers introduced default blocking and pay-per-crawl controls in 2025. Content-licensing agreements between AI companies and publishers (for example, with news organizations and with Reddit) demonstrate that the value of web content for AI systems has acquired a price, in some cases outside the link economy altogether.

7.10 Assessment of Era V

The key conclusions are as follows.

1. Defense is learned and continuous. SpamBrain and associated policies treat link spam as one of several families of manipulation, handled by classifiers and policy, and not by a dedicated link algorithm.
2. The unit of value is the entity. Authority is increasingly attributed to entities whose reputation is corroborated across many sources, with links as one evidence type.
3. Links lose some roles and keep others. The referral function declines for informational queries answered in generated summaries. The ranking and retrieval-eligibility functions persist. A new function, as a curation and training signal for AI corpora, emerges.
4. The adversarial structure persists. Generative systems are manipulable, and a new class of optimization practices is developing with the same dynamics as earlier ones.
5. The evidence is thin. Much of what is claimed about AI visibility rests on vendor studies and correlational data, and the systems change frequently. Conclusions should be held provisionally.

8. Comparative Synthesis: Shifting Paradigms Across Five Eras

8.1 Purpose and Method of the Synthesis

The preceding sections described each era on its own terms. This section compares them along a consistent set of dimensions to identify the structural regularities that the individual narratives suggest. The comparison is interpretive, and its categories are simplifications. Eras overlap, and many practices persist across boundaries. The purpose is to make the pattern visible, not to claim that history divides neatly.

8.2 Comparative Table

Dimension	Era I (1994–2003)	Era II (2004–2011)	Era III (2012–2017)	Era IV (2018–2022)	Era V (2023–present)
Core ranking signal	Link count and PageRank; on-page text; anchor text	PageRank plus anchor-text relevance; early trust propagation	Link profile quality, anchor distribution, semantic understanding	Contextual, topical links within a broader quality and reputation model	Entity reputation, retrieval eligibility, and learned quality; links as one input
Unit of value	The link (vote and label)	The anchor-text link	The link profile (signed: asset or liability)	The editorial, contextual link	The entity and its corroborated mentions
Dominant tactics	Directory submission, reciprocal links, link pages, blogrolls	Automated posting, article spinning, paid links, link wheels, blog networks	Guest posting, infographics, resource pages, skyscraper, outreach, PBNs	Digital PR, data journalism, reactive PR, quality guest content	Digital PR, brand and entity building, citation building, GEO, platform presence
Principal defenses	Reciprocal-link detection, Hilltop-style relevance, manual removal	nofollow, Jagger, TrustRank research, public penalties, Panda	Penguin, manual actions, Disavow Tool	Penguin 4.0 (devaluation), sponsored/ugc, core updates, quality raters	SpamBrain, link spam update, spam policies (scaled content, site reputation, expired domains)
Information visibility to optimizers	High (toolbar PageRank, link operator)	High to medium (Site Explorer, third-party indices)	Medium (proxies such as DA, Trust Flow replace PageRank)	Low (invisible devaluation)	Very low (learned classifiers, variable AI outputs)
Cost of producing a link	High (manual) falling	Near zero (automation)	Falling for low-grade, rising for editorial	Rising for editorial	Near zero for synthetic, very high for credible
Risk of manipulation	Ignored or filtered	Filtered, occasional public penalty	Algorithmic penalty	Silent neutralization; manual action	Neutralization, policy enforcement, and exposure to new generative attack surfaces
Practitioner role	Webmaster, submitter	Technician, tool operator	Outreach specialist, risk auditor	PR and content strategist	Reputation and entity strategist
Success metric	Link count, toolbar PageRank	Rankings for anchor phrases	DA, DR, Trust Flow, referring domains	Coverage quality, topical relevance, referring domains	Share of voice in AI answers, citations, brand mentions, plus classic metrics
Main economic driver	Search traffic value; AdWords and AdSense	Affiliate and ad monetization at scale	E-commerce and lead-generation competition	Content-driven brand marketing	Visibility in answers; licensing of content

8.3 Five Cross-Era Patterns

Pattern 1: The recurring cycle of signal, scale, saturation and response. Each tactic examined in this paper followed approximately the same lifecycle. A practice that produced a genuine editorial signal (a directory listing, a guest article, an infographic, a skyscraper-style improvement, a reactive PR comment) was adopted by early movers who benefited. Its success attracted imitators, and its mass adoption by operators who preserved the form but lost the editorial substance reduced its informational value. The search engine then devalued or penalized the abused version. Directories (2000s), article marketing (2000s to 2011), guest blogging (2014), infographics (2013) and generic skyscraper outreach (2010s) all illustrate it, and the same trajectory is visible in HARO-based pitching by the 2020s. The duration of the cycle has shortened with the cost of scale.

Pattern 2: The relocation of the exploitable surface. The surface of manipulation migrated with the defenses. In Era I, the exploitable feature was link count. In Era II, once link farms and reciprocal links were discounted, it was the anchor text. After Penguin punished unnatural anchors, the surface moved to the apparent editorial quality of the source, expressed through guest posts and PBNs on seemingly legitimate sites. After devaluation of such links, the surface moved to authority borrowed from reputable hosts (parasite SEO) and expired domains. In the AI era, it moves to the corpora and retrieval layers that generative systems consult. This pattern is what Goodhart’s Law predicts. Optimization pressure flows to the nearest controllable proxy, and when that proxy is defended, it finds the next.

Pattern 3: The shift from observable to opaque defenses. In Era I, the algorithm was understandable and its main input was displayed on a toolbar. In Era II, defenses were announced (nofollow) or publicly demonstrated (penalties on well-known brands). In Era III, they were explicit and punitive (Penguin) with a published repair tool (disavow). In Era IV, devaluation was silent. In Era V, learned classifiers add intrinsic opacity, since the decision criteria are not even specified in advance as rules. From a game-theoretic perspective this is a rational move for the defender. An adversary who can observe the defense can tune against it, and the move from penalties that teach (a visible ranking drop) to neutralization that does not (a silent absence of effect) reduces the information available to the attacker. It also reduces transparency for honest publishers, which is an ethical cost discussed in Section 9.

Pattern 4: The decreasing weight of the link as a standalone signal and the increasing weight of its context. The unit moves from the raw link (Era I) to the link with anchor text (Era II), to the link as part of a profile (Era III), to the link as a contextual and editorial act (Era IV), to the link as one form of corroboration for an entity (Era V). Each step adds conditioning variables: source authority, topical fit, position, intent, independence, and finally the identity and reputation of the entity being linked. The link graph has not been replaced. It has been absorbed into a richer model, in which it serves as a primary input but is interpreted using many other signals.

Pattern 5: The convergence of link building and public relations. The practices that scaled in Era II (automation) and that were punished in Era III were technical in nature. The practices that remain effective in Eras IV and V are relational and editorial. This convergence is not accidental. It follows from the logic of the signal. If a link’s value derives from its being an uncoerced, independent editorial decision, the only way to obtain it at scale without manipulation is to give an editor a reason to make that decision. The legitimate method is therefore the one traditionally called public relations: producing something worth covering, and ensuring that the right people know about it.

8.4 A Theoretical Lens: Costly Signals and Markets for Lemons

Two concepts from economics clarify why the cycle has the shape it does.

Costly signaling. Spence (1973) showed that a signal can convey reliable information about an unobservable quality only if it is more costly for low-quality than for high-quality senders to produce. A link functioned as a quality signal in the early Web because creating one required deliberate effort by an independent party, and because people were reluctant to link to things they did not respect. This was the implicit assumption behind PageRank. The industrialization of link production in Era II violated that cost differential: for the manipulator, the marginal cost of a link fell to near zero, whereas for an honest publisher it was unchanged. The signal’s informational content therefore declined. Many of the search engines’ later countermeasures can be interpreted as attempts to restore a cost differential: by discounting links that are cheap to produce (comment links, directories, reciprocal links, site-wide links), by emphasizing links that are expensive to counterfeit (editorial coverage by established publications), and by shifting attention to signals such as brand search demand and entity reputation, which are costly to fake at scale. Digital PR can be understood as the practice of producing the costly signal honestly.

Markets for lemons. Akerlof (1970) showed that when buyers cannot distinguish quality, low-quality goods can drive out high-quality ones. The market for paid links and sponsored posts exhibits the same structure. Buyers cannot readily verify a placement’s true traffic, editorial independence or risk of future devaluation, so sellers advertise metrics (DR, traffic) that are themselves manipulable. The resulting market tends toward low-quality offerings sold at prices that obscure their risk. Search engines’ disclosure attributes (sponsored) attempt to reduce the information asymmetry on the signal side, but compliance is voluntary for exactly the sellers who benefit from non-disclosure.

8.5 What Persisted Through All Five Eras

A comparative synthesis should also record continuities. Four ideas remained stable across the entire period.

1. Recursive authority. The principle that a reference from a reputable source is worth more than a reference from an obscure one has been present from PageRank (and, before it, from bibliometric influence weights) through TrustRank, Trust Flow, E-E-A-T’s emphasis on reputation, and entity-authority models. Its mathematical form has changed, but its content is constant.
2. Independence. Value attaches to endorsements from parties not controlled by the beneficiary. From the Hilltop requirement for non-affiliated experts to modern emphasis on independent coverage, the logic of independence is unchanged.
3. Relevance. The idea that a link is more informative when the source and target are topically related, present in Hilltop and topic-sensitive PageRank in the early 2000s, became a principal criterion in Era IV and is encoded in entity co-occurrence in Era V.
4. The adversary. Throughout, a population of actors has sought to simulate the signal, and the search engines have responded. Nothing in the evidence suggests that this has ended or will.

9. Ethical Considerations, Challenges and Future Outlook

9.1 Sponsored Content, Disclosure and the Ethics of Undisclosed Placement

The ethical core of link building is a question about honesty toward two audiences: readers and search engines. A link from a publisher to a client’s site is, in the ordinary understanding of readers, an editorial recommendation. When the link is paid for and presented as a recommendation, readers are misled about the commercial relationship, and search engines are misled about the independence of the endorsement.

Search engine policy. Google’s spam policies treat links that exist primarily to manipulate ranking, including buying or selling links for ranking purposes, excessive link exchanges and large-scale guest posting campaigns with keyword-rich anchors, as violations. The recommended handling of legitimate paid or promotional links is to qualify them with rel="sponsored" or nofollow. This convention allows commerce to take place without distorting the signal, and it is the principal mechanism by which advertising and editorial links are separated at the technical level.

Consumer protection law. Distinct from search policy, advertising law imposes disclosure obligations on material connections between advertisers and endorsers. In the United States, the Federal Trade Commission’s Endorsement Guides, revised in 2023, require clear disclosure of material connections and address fake and misleading reviews and endorsements (FTC, 2023). Regulators in the United Kingdom and the European Union apply similar principles to native advertising and influencer marketing. The relevance to link building is that a sponsored article that reads as independent journalism but is paid placement may raise concerns under such rules, independent of any search-engine rule. Link-building practitioners have historically paid more attention to the search-policy risk than to the consumer-protection risk, though the legal exposure is of a different order.

Disclosure at the link level and at the page level. An important subtlety is that the rel attribute qualifies the link, whereas consumer-protection disclosure attaches to the content. A sponsored post may carry a disclosure label for readers while still containing followed links, or may qualify the link technically while hiding the commercial relationship from readers. Best practice requires both: a visible disclosure for the human audience and an appropriate attribute for the machine audience.

The ethics of the gray market. The sale of placements on sites that present themselves as independent publications is the central ethical problem. Even if the buyer does not intend to deceive readers, the structure of the market encourages publishers to hide their commercial arrangements. The more successful a site becomes as a link vendor, the less its content can be trusted as independent, and the less valuable its links become. In effect, the market consumes the asset it sells.

9.2 Ethical Questions in Digital PR

The migration toward digital PR does not eliminate ethical problems. It transforms them.

1. Manufactured newsworthiness. Surveys with unrepresentative or tiny samples, rankings built on arbitrary criteria, and “studies” designed to produce a predetermined headline put misleading information into the public sphere for the purpose of earning links. The harm is borne by readers and by the credibility of the journalists who repeat the claims.
2. Fabricated experts and synthetic quotes. Reactive PR created an incentive to supply quotes from experts, and, with generative AI, to produce plausible but unverified expert commentary or invented personas. Journalists and platforms have responded with verification requirements, and some outlets have publicly retracted articles containing fabricated sources.
3. Source pressure on journalists. Mass outreach imposes costs on journalists, who must filter large volumes of irrelevant pitches. The aggregate effect of scaled outreach, as with comment spam, is a tragedy of the commons in which individually rational efforts degrade a shared channel.
4. Transparency about intent. A journalist who covers a study is entitled to know who funded it and why. Concealing a commercial motive, particularly in YMYL topics such as health and finance, may cause real harm if the published information is inaccurate.
5. AI-generated content at scale. Using language models to produce large volumes of articles to host links, a practice squarely within Google’s scaled-content-abuse policy, raises quality and honesty issues. The wider concern is the degradation of the information environment that both search engines and language models depend on.

9.3 Web Rot and Link Decay

The reliability of links as a long-term infrastructure is itself in question. Berners-Lee (1998) argued that persistent URLs are a basic obligation of web publishers, yet the evidence is that links decay. Studies in the early 2000s found substantial turnover of web content, including Fetterly et al. (2003) on the evolution of pages and Ntoulas, Cho and Olston (2004) on how the Web changes. Zittrain, Albert and Lessig (2014) documented the extent of link and reference rot in legal and scholarly citations, and they proposed the Perma.cc archiving service as a remedy. A 2024 Pew Research Center analysis of web content found that a substantial fraction of pages that existed in the early 2010s were no longer accessible, and that many news and reference pages contained at least one broken link (Pew Research Center, 2024). The exact figures should be checked against the source, but the direction of the findings is consistent across studies.

Link decay has several consequences for link building and search.

1. Depreciation of link assets. A link on a page that disappears, or on a domain that expires or changes hands, ceases to provide any value, so a link profile is a decaying asset requiring maintenance. Professional practice includes monitoring for lost links and reclaiming them through outreach or redirects.
2. The market for expired domains. Domains that lapse still carry inbound links. Buying them to inherit the reputation, or using them to build PBNs, exploits link persistence at the expense of the original linkers’ intent. Google’s expired domain abuse policy responds to this.
3. Citation and trust. The reliability of references is central to E-E-A-T, to scholarly credibility and to AI systems that cite sources. A model that cites a dead link provides a user with a degraded answer, which raises demand for archiving and for stable identifiers.
4. Preservation infrastructure. The Internet Archive’s Wayback Machine, which has archived web pages since 1996, and tools that automatically replace dead links with archived versions provide partial mitigations, but they depend on voluntary and under-resourced institutions.

The sociological point is that link-building economics assumes durable links, whereas the Web is an ephemeral medium. The future of link-based authority will partly depend on whether the infrastructure of persistence improves.

9.4 Links as Ranking Votes Versus Links as Training Weights

The most consequential open question for the discipline is whether the hyperlink’s role in AI systems will be that of a ranking vote, a training signal, or neither. Four scenarios are plausible, and they are not mutually exclusive.

Scenario A: Continuity with erosion. Search engines retain link-based authority as an important input, and generative interfaces draw on traditionally ranked results. Links remain essential for retrieval, but the proportion of user attention that reaches the underlying sources declines, and the economic value of links falls except for navigational and transactional queries. Under this scenario the discipline continues much as in Era IV, with a shrinking return on investment for informational content.

Scenario B: Links as corpus curation. As the quality of pretraining and retrieval corpora becomes a competitive advantage for AI developers, the link graph is used more heavily as a quality filter, as it was for WebText (Radford et al., 2019). The authority of a page is assessed on the basis of who links to it and in what context, and highly linked sources are over-represented in training data. In this scenario, links matter less for ranking visible results and more for determining which sources a model “knows” and trusts, with effects that are lagged, opaque and difficult to measure. The research on link-aware pretraining (Yasunaga, Leskovec and Liang, 2022) suggests technical interest in this direction.

Scenario C: Substitution by direct licensing and provenance. Instead of relying on the open link graph, AI developers sign licensing agreements with publishers, and systems rely on a curated set of trusted sources and on provenance metadata. Reputation is established by contract and by cryptographic or institutional verification, not by links. Early indications include news-publisher licensing deals, platform data agreements and infrastructure providers’ pay-per-crawl proposals. If this scenario dominates, the open web’s link economy is displaced by a smaller number of bilateral relationships, with significant implications for small publishers and new entrants, who have no leverage to negotiate.

Scenario D: Adversarial equilibrium. Generative systems are targeted by manipulation at scale, and developers respond with seed-set trust, source reputation and classifiers, recapitulating the history of link spam and TrustRank. The result is an arms race that mirrors Eras II through IV, with a new vocabulary.

The evidence available does not allow a confident forecast. A reasonable expectation is a blend, with Scenario A or D characterizing the near term for general web search, and Scenarios B and C affecting how AI developers select sources and which publishers remain economically viable.

9.5 Structural Challenges for the Discipline

Several unresolved problems follow from the analysis.

Measurement. Practitioners lack reliable ways to measure influence in generative systems. Outputs vary between runs, between users and over time, and vendors’ visibility metrics are based on sampled prompts of uncertain representativeness. The measurement problems that applied to links (proxies, correlations, vendor metrics) apply to AI visibility with greater force.

Concentration. If AI systems rely on a small set of highly authoritative sources, and those sources are heavily over-represented in training and retrieval, then the benefits of authority concentrate further, as in the preferential-attachment dynamics discussed in Section 2.1. New and small entities face a bootstrapping problem: authority is needed to be cited, and citation builds authority.

Economic sustainability of publishing. If AI answers reduce clicks, ad- and subscription-funded publishers lose revenue, which threatens the content ecosystem on which AI systems depend. Pew Research Center (2025) found that users clicked results less often in the presence of AI summaries. The long-term effects of reduced traffic on the supply of high-quality content are uncertain, but the dependency is real.

Governance. Questions of who may crawl, for what purpose and on what terms are being settled through technical standards (the Robots Exclusion Protocol and proposals for AI-specific controls), contracts and litigation. The outcomes will shape the conditions under which any link-based or citation-based authority operates.

Regulatory exposure. Disclosure rules, rules on AI-generated content, and competition law (including the remedies arising from antitrust proceedings concerning search) may alter the landscape in ways that the search-optimization industry has little power to influence.

9.6 Directions for Further Research

The analysis suggests several research priorities. First, controlled experiments on the causal effect of links of different types on rankings and on AI citation, which are difficult but not impossible given the availability of test domains and reproducible query sets. Second, longitudinal studies of AI citation sources to identify the stability of the distributions and the factors that govern inclusion. Third, work on provenance and reputation systems for AI retrieval that might supplement or replace link-based trust. Fourth, studies of the economic effects of AI answers on publishers across categories. Fifth, ethical and legal analysis of sponsored content, disclosure and synthetic expertise in an AI-mediated information environment.

10. Conclusion and References

10.1 Conclusion

This paper has argued that the history of link building from the 1990s to the present is best understood as a prolonged negotiation about how an open, decentralized network can communicate trustworthy judgments about itself. The hyperlink was designed without a mechanism for expressing the reason for its existence. Search engines assigned it meaning, first as a vote (PageRank) and a label (anchor text), and in doing so they created an incentive to counterfeit it. The history that followed is a sequence of cycles in which each new use of links produced new forms of manipulation, and each form of manipulation produced a new defense.

Five eras structured the account. In the first (1994–2003), link quantity was the dominant lever, directories and reciprocal links were the characteristic tactics, and defenses were heuristic. In the second (2004–2011), automation collapsed the cost of links, anchor text became the principal manipulation surface, and Google, Yahoo! and Microsoft introduced nofollow and academic trust-propagation methods while relying on exemplary penalties. In the third (2012–2017), Penguin and the Disavow Tool turned links into potential liabilities, third-party metrics replaced toolbar PageRank, and the industry pivoted toward content marketing and outreach. In the fourth (2018–2022), link devaluation became continuous and silent, link attributes formalized the distinction between editorial and commercial links, topical relevance and E-A-T embedded links in a broader reputation model, and digital PR absorbed the best of link building. In the fifth (2023–present), learned spam classifiers and spam policies address the manipulation of links and content together, and generative search shifts the object of optimization from the ranked list to the generated answer, and from the link to the entity.

Four conclusions bear emphasis.

First, the answer to RQ1 is that tactics have consistently tracked the signals and defenses of their time, with a lifecycle of innovation, adoption, saturation and devaluation that has accelerated as the cost of scale has fallen.

Second, in answer to RQ2, the information-retrieval principles that explain the trajectory are recursive authority, independence, topical locality and trust propagation. They have been stable. What has changed is the sophistication with which they are implemented, from global scalars (PageRank) to contextual models (reasonable surfer, topical authority) to learned classifiers (SpamBrain).

Third, in answer to RQ3, non-link signals have supplemented rather than replaced links. Brand mentions, entity co-occurrence and AI citations are increasingly important, especially in generative systems that learn from text, but the evidence for their relative weight is mostly correlational and vendor-produced. Links remain part of the infrastructure through which sources are retrieved and, plausibly, through which they are selected for training corpora.

Fourth, in answer to RQ4, the principal challenges are ethical (disclosure, synthetic expertise), infrastructural (link decay) and economic (the effect of AI answers on the publishers whose content sustains them). The future role of the hyperlink is uncertain and may differ between search ranking, retrieval and model training.

The practical implication for practitioners is that the sustainable form of link building is the production of credible, independently verifiable authority: work that earns editorial attention from sources a reader would trust, and that makes an entity consistently and accurately represented across the web. This has been the direction of the search engines’ defenses for twenty years, and it is also the principle on which the original link-analysis algorithms rested. The change from Era I to Era V is less a change of objective than a progressive, and perhaps final, closing of the gap between the signal that search engines wished to measure and the signal that practitioners could manufacture.

10.2 References

Note on verification. This reference list is drawn from the author’s knowledge of the literature. Before submission, verify volume, page and DOI details, the exact titles and dates of Google blog posts and documentation, and the figures cited from industry and news sources. The status of rapidly changing AI-search material (Section 7) should be re-checked against current sources.

Scholarly and technical works

• Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., & Deshpande, A. (2024). GEO: Generative engine optimization. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. (arXiv preprint 2023.) https://doi.org/10.1145/3637528.3671900
• Akerlof, G. A. (1970). The market for “lemons”: Quality uncertainty and the market mechanism. The Quarterly Journal of Economics, 84(3), 488–500. https://doi.org/10.2307/1879431
• Albert, R., Jeong, H., & Barabási, A.-L. (1999). Diameter of the World Wide Web. Nature, 401, 130–131. https://doi.org/10.1038/43601
• Barabási, A.-L., & Albert, R. (1999). Emergence of scaling in random networks. Science, 286, 509–512. https://doi.org/10.1126/science.286.5439.509
• Becchetti, L., Castillo, C., Donato, D., Leonardi, S., & Baeza-Yates, R. (2006). Link-based characterization and detection of web spam. Proceedings of AIRWeb. Google Scholar
• Benczúr, A. A., Csalogány, K., Sarlós, T., & Uher, M. (2005). SpamRank: Fully automatic link spam detection. Proceedings of AIRWeb. Google Scholar
• Berners-Lee, T. (1989). Information management: A proposal. CERN. https://www.w3.org/History/1989/proposal.html
• Berners-Lee, T. (1998). Cool URIs don’t change. W3C. https://www.w3.org/Provider/Style/URI
• Bharat, K., & Henzinger, M. R. (1998). Improved algorithms for topic distillation in a hyperlinked environment. Proceedings of ACM SIGIR. https://doi.org/10.1145/290941.290972
• Bharat, K., & Mihaila, G. A. (2001). When experts agree: Using non-affiliated experts to rank popular topics. Proceedings of the 10th International World Wide Web Conference (WWW10). https://doi.org/10.1145/371920.372162
• Bianchini, M., Gori, M., & Scarselli, F. (2005). Inside PageRank. ACM Transactions on Internet Technology, 5(1), 92–128. https://doi.org/10.1145/1052934.1052938
• Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems, 30, 107–117. https://doi.org/10.1016/s0169-7552(98)00110-x
• Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., & Wiener, J. (2000). Graph structure in the Web. Computer Networks, 33, 309–320. https://doi.org/10.1016/s1389-1286(00)00083-9
• Bush, V. (1945). As we may think. The Atlantic Monthly, 176(1), 101–108. https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/
• Campbell, D. T. (1979). Assessing the impact of planned social change. Evaluation and Program Planning, 2(1), 67–90. https://doi.org/10.1016/0149-7189(79)90048-x
• Fetterly, D., Manasse, M., Najork, M., & Wiener, J. (2003). A large-scale study of the evolution of web pages. Proceedings of WWW2003. https://doi.org/10.1145/775152.775246
• Fetterly, D., Manasse, M., & Najork, M. (2004). Spam, damn spam, and statistics. Proceedings of WebDB 2004. https://doi.org/10.1145/1017074.1017077
• Garfield, E. (1955). Citation indexes for science. Science, 122, 108–111. https://doi.org/10.1126/science.122.3159.108
• Goodhart, C. A. E. (1975). Problems of monetary management: The U.K. experience. In Papers in Monetary Economics (Vol. I). Reserve Bank of Australia. https://doi.org/10.1007/978-1-349-17295-5_4
• Gyöngyi, Z., & Garcia-Molina, H. (2005). Web spam taxonomy. Proceedings of AIRWeb. http://ilpubs.stanford.edu:8090/771/
• Gyöngyi, Z., Garcia-Molina, H., & Pedersen, J. (2004). Combating web spam with TrustRank. Proceedings of VLDB. https://doi.org/10.1016/b978-012088469-8.50052-8
• Haveliwala, T. H. (2002). Topic-sensitive PageRank. Proceedings of WWW2002. https://doi.org/10.1145/511446.511513
• Henzinger, M. R., Motwani, R., & Silverstein, C. (2002). Challenges in web search engines. SIGIR Forum, 36(2), 11–22. https://doi.org/10.1145/792550.792553
• Hubbell, C. H. (1965). An input-output approach to clique identification. Sociometry, 28, 377–399. https://doi.org/10.2307/2785990
• Katz, L. (1953). A new status index derived from sociometric analysis. Psychometrika, 18, 39–43. https://doi.org/10.1007/bf02289026
• Kessler, M. M. (1963). Bibliographic coupling between scientific papers. American Documentation, 14, 10–25. https://doi.org/10.1002/asi.5090140103
• Kleinberg, J. M. (1999). Authoritative sources in a hyperlinked environment. Journal of the ACM, 46(5), 604–632. https://doi.org/10.1145/324133.324140
• Krishnan, V., & Raj, R. (2006). Web spam detection with Anti-TrustRank. Proceedings of AIRWeb. Google Scholar
• Kumar, A., & Lakkaraju, H. (2024). Manipulating large language models to increase product visibility. arXiv preprint. https://arxiv.org/abs/2404.07981
• Langville, A. N., & Meyer, C. D. (2006). Google’s PageRank and beyond: The science of search engine rankings. Princeton University Press. https://doi.org/10.1515/9781400830329
• Lawrence, S., & Giles, C. L. (1998). Searching the World Wide Web. Science, 280, 98–100. https://doi.org/10.1126/science.280.5360.98
• Lewis, P., Perez, E., Piktus, A., et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems (NeurIPS), 33. https://arxiv.org/abs/2005.11401
• Marchiori, M. (1997). The quest for correct information on the Web: Hyper search engines. Computer Networks and ISDN Systems, 29. https://doi.org/10.1016/s0169-7552(97)00036-6
• Nestaas, F., Debenedetti, E., & Tramèr, F. (2024). Adversarial search engine optimization for large language models. arXiv preprint. https://arxiv.org/abs/2406.18382
• Ntoulas, A., Cho, J., & Olston, C. (2004). What’s new on the web? The evolution of the web from a search engine perspective. Proceedings of WWW2004. https://doi.org/10.1145/988672.988674
• Page, L. (2001). Method for node ranking in a linked database. US Patent 6,285,999. https://patents.google.com/patent/US6285999B1/en
• Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank citation ranking: Bringing order to the Web. Stanford InfoLab technical report. http://ilpubs.stanford.edu:8090/422/
• Pinski, G., & Narin, F. (1976). Citation influence for journal aggregates of scientific publications. Information Processing & Management, 12, 297–312. https://doi.org/10.1016/0306-4573(76)90048-0
• Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI technical report. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
• Robertson, S. E., & Spärck Jones, K. (1976). Relevance weighting of search terms. Journal of the American Society for Information Science, 27, 129–146. https://doi.org/10.1002/asi.4630270302
• Robertson, S. E., & Walker, S. (1994). Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. Proceedings of ACM SIGIR. https://doi.org/10.1007/978-1-4471-2099-5_24
• Salton, G., Wong, A., & Yang, C. S. (1975). A vector space model for automatic indexing. Communications of the ACM, 18(11), 613–620. https://doi.org/10.1145/361219.361220
• Small, H. (1973). Co-citation in the scientific literature. Journal of the American Society for Information Science, 24, 265–269. https://doi.org/10.1002/asi.4630240406
• Souly, A., et al. (2025). Poisoning attacks on LLMs require a near-constant number of poison samples. arXiv preprint (Anthropic, UK AI Security Institute and Alan Turing Institute). https://arxiv.org/abs/2510.07192
• Spärck Jones, K. (1972). A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28, 11–21. https://doi.org/10.1108/eb026526
• Spence, M. (1973). Job market signaling. The Quarterly Journal of Economics, 87(3), 355–374. https://doi.org/10.2307/1882010
• Spertus, E. (1997). ParaSite: Mining structural information on the Web. Computer Networks and ISDN Systems, 29. https://doi.org/10.1016/s0169-7552(97)00033-0
• Strathern, M. (1997). “Improving ratings”: Audit in the British university system. European Review, 5, 305–321. https://doi.org/10.1002/(sici)1234-981x(199707)5:3<305::aid-euro184>3.0.co;2-4
• Wu, B., & Davison, B. D. (2005). Identifying link farm spam pages. Proceedings of WWW2005 (Special Interest Tracks). https://doi.org/10.1145/1062745.1062762
• Yasunaga, M., Leskovec, J., & Liang, P. (2022). LinkBERT: Pretraining language models with document links. Proceedings of ACL 2022. https://doi.org/10.18653/v1/2022.acl-long.551
• Zittrain, J., Albert, K., & Lessig, L. (2014). Perma: Scoping and addressing the problem of link and reference rot in legal citations. Harvard Law Review Forum, 127. https://doi.org/10.1017/s1472669614000255

Search engine communications, legal and regulatory sources

• Cutts, M. (2009). PageRank sculpting. mattcutts.com/blog. https://www.mattcutts.com/blog/pagerank-sculpting/
• Cutts, M. (2012). Webspam algorithm update announcement. Google Webmaster Central / Inside Search, April 2012. https://developers.google.com/search/blog/2012/04/another-step-to-reward-high-quality
• Cutts, M. (2014). The decay and fall of guest blogging for SEO. mattcutts.com/blog, January 2014. https://www.mattcutts.com/blog/guest-blogging/
• Federal Trade Commission. (2023). Guides concerning the use of endorsements and testimonials in advertising (16 CFR Part 255, revised). https://www.ecfr.gov/current/title-16/chapter-I/subchapter-B/part-255
• Google. (2005). Preventing comment spam. Official Google Blog, 18 January 2005. https://googleblog.blogspot.com/2005/01/preventing-comment-spam.html
• Google. (2012a). Another step to reward high-quality sites. Inside Search, 24 April 2012. https://developers.google.com/search/blog/2012/04/another-step-to-reward-high-quality
• Google. (2012b). Disavow links to your site. Google Webmaster Central, October 2012. https://developers.google.com/search/blog/2012/10/a-new-tool-to-disavow-links
• Google. (2016). Penguin 4.0 announcement. Google Webmaster Central Blog, 23 September 2016. https://developers.google.com/search/blog/2016/09/penguin-is-now-part-of-our-core
• Google. (2019a). Evolving “nofollow”: New ways to identify the nature of links. Google Search Central Blog, 10 September 2019. https://developers.google.com/search/blog/2019/09/evolving-nofollow-new-ways-to-identify
• Google. (2022a). December 2022 link spam update. Google Search Central Blog, 14 December 2022. https://developers.google.com/search/blog/2022/12/december-22-link-spam-update
• Google. (2022b). Google Search’s core updates and E-E-A-T guidance; Search Quality Rater Guidelines (various editions 2015–2025). https://developers.google.com/search/docs/fundamentals/creating-helpful-content
• Google. (2023). Link best practices for Google. Google Search Central documentation. https://developers.google.com/search/docs/crawling-indexing/links-crawlable
• Google. (2024). Core update and new spam policies (scaled content abuse, site reputation abuse, expired domain abuse). Google Search Central Blog, March 2024. https://developers.google.com/search/blog/2024/03/core-update-spam-policies
• Singhal, A. (2012). Introducing the Knowledge Graph: Things, not strings. Official Google Blog, 16 May 2012. https://blog.google/products/search/introducing-knowledge-graph-things-not/
• United States et al. v. Google LLC (D.D.C.), memorandum opinion on liability (August 2024), and associated trial exhibits and testimony. https://www.justice.gov/atr/case/us-and-plaintiff-states-v-google-llc

Industry, news and practitioner sources

• Ahrefs. (2025). Studies on brand mentions and visibility in AI Overviews and AI assistants. Ahrefs Blog. https://ahrefs.com/blog/ai-overview-brand-correlation/
• Dean, B. (2020). We analyzed 11.8 million Google search results. Backlinko (conducted with third-party data providers). https://backlinko.com/search-engine-ranking
• Mathes, A. (2001). Filler Friday: Google bombing. Uber.nu (blog). https://web.archive.org/web/2001/http://uber.nu/2001/04/06/
• Pew Research Center. (2024). When online content disappears. https://www.pewresearch.org/data-labs/2024/05/17/when-online-content-disappears/
• Pew Research Center. (2025). Google users are less likely to click on links when an AI summary appears in the results. https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/
• Segal, D. (2011, February 12). The dirty little secret of search. The New York Times. https://www.nytimes.com/2011/02/13/business/13search.html
• Slawski, B. (2013). Analyses of Google patents on implied links and site quality. SEO by the Sea. https://www.seobythesea.com/
• Documentation analyses of the May 2024 Google Content Warehouse API disclosure by independent SEO researchers (for example, R. Fishkin and M. King), with Google’s response. https://sparktoro.com/blog/an-anonymous-source-shared-thousands-of-leaked-google-search-api-documents-with-me-everyone-in-seo-should-see-them/