What is Google Bard?
Google Bard is an experimental, conversational artificial intelligence based on the Large Language Model LaMDA. This is how Google describes the technology on the Google blog. It is intended to make it possible to gain access to complex information and facts in dialog form. An example from the developer’s blog Preparing discoveries from the James Webb Space Telescope for a 9-year-old child.
What distinguishes Google’s Bard from OpenAI’s ChatGPT is that Bard can fall back on current information from the Internet right from the start. Since ChatGPT is based on the GPT-3.5 model, it only knows about texts written up to mid-2021 and can only fall back on these. Our expert Pieter Buteneers told us in an interview about ChatGPT that it is “basically a summary of the Internet until the end of 2021”. Adding up-to-date information from the Internet is the next big step, he said, “If ChatGPT can search the Internet, Google and Stack Overflow will be history. It makes sense to assume that will happen.” That’s the path Google is now apparently taking with Bard.
Microsoft, which is said to have recently invested 10 billion dollars in OpenAI, will likely announce soon that ChatGPT will be integrated into the search engine Bing. Currently, Google is and remains largely unrivaled in the search engine business. In December 2022, Google’s search engine had a market share of just under 85%, followed in second place by Microsoft’s Bing, with just under 9%. With the new developments, however, some movement could occur, depending on who succeeds in implementing the respective AI better.
Google’s conversational AI Bard is currently not yet available to the public, but only to a select group of “trustful testers”. In the coming weeks, the AI will be made available to a wider audience. To what extent the AI will be usable for free remains to be seen. ChatGPT has only recently introduced a payment model that promises better access and shorter loading times for $20 per month.
Stay up to date
Learn more about MLCON
The models: GPT and LaMDA
An AI is only as good as the model it is based on. OpenAI’s GPT model (“Generative Pretrained Transformer”), on which ChatGPT is based, has been in development for some time. GPT-3 was introduced in mid-2020 and is a Large Language Model (LLM) that was initially designed to complete texts. In an interview, Christoph Henkelmann told us more about GPT:
“GPT stands for “Generalised Pretraining for Transformers.”GPT is a family of architectures and various resulting models. Many neural networks follow this architectural pattern. GPT models are trained to simply complete text. They perform the same functions as old Nokia phones with T9. GPT attempts to predict what the next text block will be as you type. To train GPT, you collect vast amounts of text, for example, with web scraping. Then you let a neural network make predictions, sometimes for months.”
This LLM was fine-tuned to dialog progressions, eventually modeling it into ChatGPT.
Google’s LaMDA (“Language Model for Dialogue Applications”) was released in May 2021. It is an LLM that was trained on dialog progressions from the beginning. Earlier in 2018, Google presented another model called Bert (“Bidirectional Encoder Representations from Transformers”). Similar to the GPT model, Bert is based on transformers, a technology that uses a neural network architecture to fill in cloze texts: “(…) with BERT, the big breakthrough came because developers trained the model to fill in missing text parts. If you want to fill in a cloze, first you have to understand what the text is about.” That’s what Pieter Buteneers told us in an interview about Dall-E and image-generating AI.
Buteneers continues, “The big breakthrough that BERT brought was to take the old models that work well on a CPU and put them into a new architecture that computes fast on a GPU. That’s how we were able to make these huge leaps forward. Since 2018, these models have gotten bigger and smarter, so you can do more and more exciting things with them.”
What is Google doing in the AI field?
With all the excitement about ChatGPT, there hasn’t been much talk about the progress Google has already made in the AI field, especially with regard to LLMs. As previously mentioned, Google is at the forefront of natural language processing with BERT and LaMDA.
Google is also active in the area of text-to-image AIs with Imagen and even claims to produce better results than OpenAI’s Dall-E when measured against human evaluation. Work is also underway to extend Imagen to video generation. Music is also no longer out of reach for Google’s AI. With MusicLM, Google Research introduces artificial intelligence that can generate music based on text input. There is no accounting for taste, but the technical implementation is highly exciting.
Considering how incredibly versatile Google is in the artificial intelligence field, it’s fair to ask how the tech giant seems to have been overtaken by OpenAI. Not that OpenAI is a small software company or an underdog. Its market value is in the tens of billions.
However, due to Google’s dominance for years, you can still wonder how it came to this. Our expert Pieter Buteneers also told us in an interview: “I know from my contacts at Google Deep Mind that there was a bit of a crisis in the team there when GPT-3 was released. They were desperately trying to find a solution as quickly as possible. And that was just GPT-3. But with ChatGPT, I think the whole company is shaking, not just Google Deep Mind, but all of Google, or at least those who understand what ChatGPT can do.” CNBC also reported panic in Google’s executive suite.
The CNBC article reported Google’s concern of losing their reputation, which is understandable. As linguistically convincing as OpenAI’s ChatGPT may be, the AI can reproduce misinformation. This is due to the model itself, says Christoph Henkelmann: “The answer is not the result of a conclusion, but the result of training. ChatGPT has seen texts, (…) and learned from them. The closer my wording was to the ‘read’, the better the answer was.” Henkelmann went on to tell us, “It doesn’t even understand that it’s answering a question. It simply wants to complete or continue the text. And that’s the problem: GPT will continue the text even if it doesn’t have an answer. The model is not able to reflect that. There will always be a text that is statistically a likely answer for GPT, and so it can’t state, ‘I don’t know something.’”
For Google, which primarily earned its reputation as an accurate and precise search engine, this is a major problem. Additionally, certain limits have to be imposed on an AI like ChatGPT, which, for example, prohibit it from playing out or generating certain information. Therefore, ChatGPT does not answer some queries at all if they violate the guidelines. If you ask the AI how to build a bomb, you will (fortunately) not get the answer you were hoping for.
And that doesn’t even touch on the issue of sexist, racist, or other discriminatory and misanthropic outputs of an AI, which in part only reproduces what it already has. Microsoft had a similar experience in 2016 with its chatbot Tay.
We experienced similar concerns during the release of OpenAI’s image AI Dall-E, when accusations were made that the release was irresponsible because restrictions were missing. Image generation by artificial intelligences in particular presents the problem of ethical and moral boundaries in the truest sense of the word in a very graphic way. The concern that an AI trained and managed by Google could produce reprehensible results of whatever kind would be reputationally damaging is justified.
However, it is much more likely that the new chatbots are a problem for Google’s business model, which is largely financed by advertising revenue. It’s not impossible that Bard or even ChatGPT as a Bing implementation could phase out advertising. Classic search engine advertising is based on clicks, and these could fail to materialize if the AI response is enough for users.
If the answer to a question is resolved on Google itself, there is much less motivation for end users to visit or look at the actual source. This is not only a problem for Google, which promises to drive traffic to the advertisers’ site through ad placement, but also for the operators of websites themselves, whose content may now only be rendered by an LLM. The fear is that the source will not be visited.
Of course, Google already displays answers to certain questions directly on their site. But it is clear that the search engine advertising model, at least in its current form, is no longer sacrosanct and will undergo a transformation if the search engine operators want to implement AI.
What’s next?
The release of ChatGPT, which hit many experts unexpectedly, has marked a new battleground for the tech giants. This is not a reason to panic yet, nor is it a reason to worry about an AI revolution. However, the developments are likely to be especially exciting in terms of how we search for information on the Internet in the future and how we are presented with the results. It remains to be seen what this means for the major search engine operators’ business models, content optimization, and SEO.
At least for us as viewers, turning the search engine business upside down on the way to artificial general intelligence (AGI) is pretty interesting. We will see what long-term challenges and improvements for users will result from this little AI war.
As long as Bard is not yet publicly available, it’s hard to evaluate what the AI is really capable of. We don’t know yet if it is as user-friendly as ChatGPT. If the rumors are true, we should learn more about possible implementation of ChatGPT in Bing very soon. Until then, we will have to be patient, but we can be excited about the very near future.
Be part of this exchange with leading experts at the heart of current AI developments, at MLCon Munich 2023.
