Google Bard is better at debunking conspiracy theories than ChatGPT, but just barely

One of the concerns about generative AI is the easy, hard-to-keep-in-check spread of misinformation. It’s one area many hoped Google Bard would step up above existing options, and while Bard is better at debunking known conspiracy theories than ChatGPT, it’s still not all that good at it.

News-rating group NewsGuard tested Google Bard against 100 known falsehoods, as the group shared with Bloomberg. Bard was given 100 “simply worded” requests for information around these topics, all of which had content around the false narratives existing on the internet.

 

 

 

 

That includes the “Great Reset” conspiracy theory that tries to suggest COVID-19 vaccines and economic measures being used to reduce the global population. Bard apparently generated a 13-paragraph reply on the topic, including the false statement that vaccines contain microchips.

Bard managed to bring out information on 76 of the 100 topics, generating “misinformation-laden essays.” However, Bard did debunk the other 24 topics, which while not exactly an confidence-inspiring total, is still better than competitors. In a similar test, NewsGuard found that OpenAI’s ChatGPT based on the latest GPT-4 didn’t debunk any of the 100 topics, where GPT-3.5 was sitting around 80%.

In January 2023, NewsGuard directed ChatGPT-3.5 to respond to a series of leading prompts relating to 100 false narratives derived from NewsGuard’s Misinformation Fingerprints, its proprietary database of prominent false narratives. The chatbot generated 80 of the 100 false narratives, NewsGuard found. In March 2023, NewsGuard ran the same exercise on ChatGPT-4, using the same 100 false narratives and prompts. ChatGPT-4 responded with false and misleading claims for all 100 of the false narratives.

Google has, of course, not been particularly shy about Bard’s AI responses bringing up responses like this. Since day one, Bard has shown warnings about how it is an “experimental” product and that it “may display inaccurate or offensive information that doesn’t represent Google’s views.”

 

 

 

 

Misinformation is a problem that generative AI products will clearly have to work to improve on, but it is clear Google has a bit of an edge at the moment. Bloomberg tested Bard’s response to the conspiracy theory that bras can cause breast cancer, to which Bard replied that “there is no scientific evidence to support the claim that bras cause breast cancer. In fact, there is no evidence that bras have any effect on breast cancer risk at all.”

NewsGuard also found that Bard would occasionally show a disclaimer along with misinformation, such as saying “this claim is based on speculation and conjecture, and there is no scientific evidence to support it” when generating information about COVID-19 vaccines having secret ingredients from the point of view of an anti-vaccine activist.

Google is working on improving Bard. Just last week, the company said it was upgrading Bard with better support for math and logic.

Google’s next Bard update brings ‘more variety’ to drafts

 

 

 

 

Google is rolling out a new update to its Bard AI experiment this week that will expand on one of the platform’s unique aspects in “drafts.”

As confirmed on Bard’s new “Experiment updates” changelog that Google introduced earlier this month, the second update to Bard is set to be available tomorrow, April 21. Google says the update will add “more variety” to Bard’s drafts.

 

 

 

Google opened up early access to Bard, its generative AI chatbot, and we’ve had a bit of time to play around with it. The takeaway so far? Google isn’t exactly treading new ground here, but Bard is at least much more clear on what it can do, can’t do, and where it falls short.

What can you do with Bard?

Google Bard is a generative AI product built on the LaMDA model introduced in 2021. Bard uses that underlying tech to respond to prompts, generate text, answer questions, and more. Google summarizes Bard, saying:

Bard is powered by a large language model from Google that can generate text, write different kinds of creative content, and answer your questions in an informative way.

So what can you do with Bard?

The first thing that comes to mind, especially following the debut of Bing’s GPT-powered chat experience, is to use Bard to find answers to questions or help you better understand a topic. And to that end, it works rather well.

Asking Bard to explain an aspect of a smartphone or summarize a recent news topic results in a very readable explanation that, at least in my limited usage thus far, feels less long-winded and much more concise than what Bing and ChatGPT usually offer. That’s not to say the actual word count is always shorter, but Bard’s replies are phrased in a way that’s just easier to read.

 

 

Google offered more information, though it did get the front-facing camera spec wrong.

 

 

 

 

 

 

Google has made it clear that Bard AI isn’t meant to replace traditional Search at this point, but it is impressive how Bard can quickly pull together a lot of information into a concise format. And it’s probably for the best that Bard, as it exists today, is not replacing Search because, in this current format, Bard rarely shows where it is getting information, and even when it does, it’s very limited.

Another way I found Bard useful was for coming up with recipes. I love to cook and come up with ideas for dinner on the fly, but it’s always helpful to have some sort of foundation to form those ideas off of. Bard seems to be really good at that. Asking for a recipe with a handful of ingredients pulls together some ideas, and using the “drafts” Bard generates, I get a few options at once. The responses are sometimes not very helpful or a bit boring, but I can see these ideas giving me something to work off of.

 

 

Having multiple responses on hand without reissuing the prompt seems genuinely useful

 

 

 

 

 

 

But really, Google isn’t doing anything new with use cases like this. Bard is doing the same thing as ChatGPT, just with updated information. That’d be impressive if Bard had launched a month ago, but Microsoft’s Bing is already doing the same thing too, and all based on OpenAI’s GPT-4 model.

Google Bard still makes plenty of mistakes

The big thing that many, myself included, were hoping to see Google Bard build on that other AI tools haven’t is to be more accurate. It’s really easy to get other generative AI products to generate nonsense – known as “hallucinations” – or simply get a lot of simple facts wrong.

In my use so far, Google Bard doesn’t seem noticeably better on this front. In comparing some responses from Bard side by side with Bing, I noticed fewer errors with technical details on smartphones, but I also commonly saw my responses having errors and mistakes throughout.

 

 

Bard incorrectly says the main sensor in Find X6 Pro is the IMX890 instead of the IMX989.

 

 

Some of the mistakes I saw Bard make were as simple as an incorrect figure. For instance, a question about the Pixel 7 Pro saw Bard telling me that Tensor G2 was built on a 4nm process, something that’s simply not true. There are also plenty of errors that just go against common sense, such as Bard implying the Pixel 7 and Pixel 7 Pro haven’t been released.

 

 

 

 

Getting away from smartphones, information about other topics results in similar mistakes.

When I asked Bard to create a vegan meal plan, it spit out a helpful list of ideas, but it threw in yogurt and hard-boiled eggs as snacks, which obviously don’t fit a vegan diet. And when I asked Bard to update the list to remove items with beans, it essentially spit out the same list again, still with black bean burgers in place.

 

 

 

 

 

These mistakes are common for generative AI and show how Bard is still not quite up to par with typical search results.

And what’s frustrating is that Google Bard doesn’t cite its sources. While Bing shows links to where it pulls information throughout, Bard only occasionally shows a link to where its information came from. Maddeningly, you can’t even manually ask Bard to show that information.

 

 

 

 

Google clearly doesn’t want you to think Bard is a finished product

But there’s one thing about Google Bard that really stood out to me against other AI tools like it. Google isn’t treating this like a finished product, and it’s doing its due diligence to be responsible about what the AI is spitting out.

Throughout your use of Bard, Google will remind you again, and again, and again that Bard is an AI, and its information won’t always be correct. There’s a constant banner under the chat box that directly says:

Bard may display inaccurate or offensive information that doesn’t represent Google’s views.

Further, Bard holds back on lots of sensitive topics. If you ask about medications or even something like weight loss, Bard might just avoid the topic altogether. You also can’t get Bard to explain its sources or talk about specific people. Asking Bard to offer up details on a person just doesn’t work, although you can still trick the system by using a social handle or username (sometimes with crazy results).

There are also more subtle ways Google is implying that Bard isn’t finished. There’s no prominent logo or branding outside of the “diamond” icon seen alongside replies. There’s not even an icon when you create a shortcut to the product on your smartphone’s homescreen.

 

 

There are two notices about Bard the moment you open it.

 

 

And of course, there’s the fact that Bard is currently siloed off from the rest of the company’s offerings. There’s no Bard in Google Search, or Workspace apps, or anything else. That’s coming, but this early preview is just that – an early chance to try out the tech that powers Bard rather than using it alongside the rest of Google’s suite.

There are two ways to look at this, one being that Google is just trying to be more responsible with Bard AI compared to some others. That’s certainly part of the equation, but reading between the lines, it also seems like Google is just trying to excuse that it is a bit behind the curve. Bard is good, but it’s not better than what Microsoft and OpenAI are putting in front of customers. It’s rough around the edges, and Google was definitely right to temper expectations.

Now, the question is just whether Bard’s future can actually prove to be better.

You can’t use Bard with a Google Workspace account yet

Google just opened up access to Bard, its generative AI product, via a waitlist today. However, you won’t be able to use Bard, or even sign up for that waitlist if you have a Google Workspace account.

The requirements to use Bard during its early access period are not particularly strict. For instance, Bard will work on most browsers, including Google Chrome, Microsoft Edge, and Apple’s Safari. That’s certainly more flexible than what Bing has been doing with its GPT-4-powered AI experience.

One limit that rules out a lot of younger users is age. Google says that you need to be at least 18 years old to use Bard. That makes sense, given Google directly warns that, like other generative AI tools, Bard can sometimes go a little off the rails and deliver inaccurate or even offensive responses.

But perhaps the biggest restriction is that, at least for now, Google Bard doesn’t work with Google Workspace accounts.

If your Google account is managed by and organization (or parent/guardian) it can’t be used for Bard. This includes Workspace accounts that utilize a custom domain instead of “@gmail.com” for Gmail and Google sign-in. Attempting to use a Workspace account on Bard shows the error message below.

 

 

 

 

It’s not entirely clear why this restriction is in place, especially with Google’s clear vision for generative AI in Workspace products, but the fact is that it is in place as of today. We suspect this may change over time, but it’s hard to tell at this point.