An important quirk of the first 30 years of the Internet is that search has not been a composable feature of Internet applications.
Search and discovery is arguably the most important thing the Internet is useful for, the Emperor of All Internet Applications. The Internet levels the playing field so niche content creators, sellers, producers of all kinds can find an audience anywhere in the world. Wrangling this massive messiness requires Aggregators, which get more valuable as the web gets more massive. That's why the largest Internet companies provide a search and discovery role, and are rewarded with the biggest market caps and highest profits for doing so.
Thanks for reading Rishi’s Newsletter! Subscribe for free to receive new posts and support my work.
A notable reign
Google is of course, as Ben Thompson noted, the king of aggregators, with the superlatives to prove it. Yet after 25 years, Google is stale, rather shockingly so given the evolution of the web in other categories and the company’s enormous resources. It's still roughly a list of blue links. It's gamified to the nth degree by creators and companies of all kinds, and in so many cases doesn't actually give you what you want - which is a pointed answer to a question.
Google's monopoly, brand, network effect for creators and advertisers, and most importantly, search result quality (relative to competition) has made it untouchable, so all the economic value of “search on the web”, the most important Internet application, accrued to them.
And only them. The search algorithm is a trade secret. The only way to access its power is through Google's search bar. And the only answers are the ones they provide. Google employs no less than 10,000 people in maintaining and expanding its knowledge graph. If an application wants to search the whole of human knowledge on the Internet to enable its own service, they play by Google's rules. Which so far have been: if you want search, you’ve got Google.com’s website, and little else.
There have been many efforts to peel off verticals from Google, particularly in certain commerce verticals like travel. The mid-2000s saw a variety of high quality vertical search companies like Kayak, Shopping.com, and Yelp re-implement the search bar paradigm, but importantly, they not only had to build on top of proprietary data (the Internet as such was too hard to wrangle for a startup), but also could only build their businesses entirely on Google's distribution via SEO, SEM, or both.
Enter the magic Generative AI machine
While most of the media attention has focused on the arresting creative power for new images, text, and video that generative AI models have enabled, the bigger implication of the transformer model is much more profound.
The state of AI now creates a general purpose computer, trained on all the unstructured data on the Internet, that in the course of building a model, must understand relatively deeply the meaning of concepts represented as words, and as such can return high value answers to the same kinds of queries we have typically seen on Google, for any application, as an embedded service. As Andrej Karpathy, head of AI at Tesla, noted on Twitter:
Already the power of GPT-3 is leading to tweets like this:
What if search was an embedded, callable super power, placed in service of any end user application? Further, imagine that proprietary data of any kind can be used to personalize, enhance, and target what an application delivers, while creating the basis for differentiation in the service?
This is an unprecedented power shift into the hands of entrepreneurs and developers everywhere, which can ultimately lead to the unbundling of Google.
To make this happen, the cost to build, train, and employ models has to be low and getting lower, and we're seeing that not just from OpenAi, but also from emerging model platforms like Stability.ai, Midjourney, and others, who are demonstrating that the ability to leverage the entirety of human knowledge on the web as a service will not be in the hands of a select few, but rather available and open sourced at reasonable cost for any entrepreneur.
When searching the web to find answers is a composable feature of any application, you can create wholly new applications that change the way consumers seek information. Today we're already seeing early signs of this phenomenon, even though current text models are in their relative infancy (but still eye-poppingly articulate). Virtually any category where there is 1) huge information asymmetry between consumer knowledge and a stable set of canonical knowledge; and 2) something meaningful is at stake, has the potential to birth a new product where a user can harness the power of searching the entire web to find just the right answer.
Here are a few of my favorite early examples that start to poke at what's possible:
- CoPilot (Code): 1.2M developers use the AI assistant to write 30% of their code, a vast number and huge improvement over prior attempts to put Stack Overflow in the IDE. Built by Microsoft on OpenAI’s codex, it’s a great example of how a ton of hard product goes into to actually making an AI bot that’s right only 30% of the time useful to developers.
- Kyron Learning (Education): tackling tutoring in the K-12 range, Kyron, founded by AI leadership from Google, is building a 2-way AI-based tutor. The future of education, and really all learning, potentially comes from moving from the current world of “you can watch videos of the best lectures from the best teachers on anything” to “we’ve trained an AI service to be the best teacher in the world to you for your subject, on a conversational basis”: Jaime Escalante as a service.
- Adept.ai (AI assistant): founded by two of the original authors of the Attention paper, building a new interface layer for the Internet, with a big vision to harness a model for action, turning commands into the mouse clicks and Internet pathways that generate actual actions. Check out this announcement on A CT-1, their “action transformer” to see a compelling vision for how our relationships with computers, and the knowledge on the Internet, could meaningfully evolve.
- Mem.ai (Productivity): founder Dennis Xu describes the company’s vision as “Imagine if you had a Google search bar but for all nonpublic information, for every piece of information that was uniquely relevant to you.” Combining transformer based AI with Internet knowledge with your personal information enables a self-organizing workspace, a personalized search engine with answers instead of links.
Mem’s concept of a personalized set of data working with all the knowledge on the web leads to a bunch of exciting potential reinventions, especially in categories where search fails but could be particularly useful, where the right knowledge at the right time would REALLY help.
For example, therapy: there is a clear canon of knowledge on specific ways to handle crucial conversations and prevent relationship-busting arguments in favor of a more constructive approach. Problem is, you can’t remember any of that when you’re seeing red with your spouse. Using audio cues and sentiment analysis, a personalized therapist in a live-listening AI bot has the potential to interject at just the right time to make relationships better at scale. This was an underground problem being worked on at the MIT Media Lab, which has been waiting for the AI to catch up.
Similarly, imagine a Personal Health bot, trained on your apple watch and lab results, and a full trove of printed medical knowledge. With the right tuning and input, from Doctors, people may be able to more effectively learn about their bodies and ailments, without relying on what feel like random SEO results.
Building startups into durable companies
A key question naturally becomes: if search can be part of every app, how does anyone differentiate their service, and earn the right to acquire customers? And won't incumbents capture most of the value from the emerging technology since they already have the users (indeed, were Google to meaningfully shift and potentially subvert its own search paradigm, could they not retain market position, if not necessarily the size of their business)?
It’s worth noting here that since Google began leveraging PageRank in 1998, virtually no non-hardware Internet company has built a massive business primarily on meaningful technology differentiation. Rather, staying power for the Squares, Lyfts, Facebooks, and Doordashes of the world has been about 1) great execution; 2) network effects if available; 3) speed of growth; and 4) customer insight leading to an incredible product. During the mobile revolution of 2008 - 2013, they built rapid businesses on the back of mobile adoption and created new categories. In the case of Facebook and Linkedin, they invented their own new platforms.
Thick vs. Thin Applications
As in the early days of the browser and mobile, most early applications today are thin layers on platforms like OpenAI. But the opportunity ahead is substantial precisely because of the new superpower that entrepreneurs and developers now have in building off of these services to create 1) “thick”, highly considered applications with deep user insight; 2) which combine proprietary or personal data with the Internet's knowledge to create a moat; 3) and the creates as positive feedback loop as the application harnesses user behavior insight and data to deepen it's value and customer relationship.
You can see that this is starting to happen because emerging products have clear value propositions that feel like magic, are easy to describe in two sentences, and that change the game for users. Examples like: "Copilot helps write 30% of your code, saving you hours each day", or "Jasper helps you create blog posts 10X faster with AI."
This kind of exciting product clarity has been rare of late; but it is more common when new platforms excite the minds of entrepreneurs. In these instances, the AI model is a necessary but hardly sufficient feature — the key is in building the significant software and product context to make the overall service truly useful to customers.
An Important Moment
It's this level of opportunity, where companies can see 100s of $Bns of potential market cap up for grabs because the rebels have the keys to the castle of search on the Internet, that is stoking the energy of the most ambitious entrepreneurs, and for good reason.
Whether Google responds to this moment aggressively or not, the shape of technology in huge categories will change. Indeed Google responding will likely signal a meaningful platform shift, from a closed modality to one where Google is willing to take the risk of subverting their own magic money machine.
This is a heady time in technology history, an “I’ll remember what that was like decades from now” moment, because despite inherent bureaucratic headwinds, technology capability and innovation has for the last 15 years been accelerating at the largest tech companies.
Now a come-from-nowhere technology is available to all an upending the game, and the rules of the chessboard may be rewritten, for Google and startups alike. Whatever the result, it will be a wild and wonderful ride from here.
Thanks for reading Rishi’s Newsletter! Subscribe for free to receive new posts and support my work.