Garbage in, garbage out — How health marketers can ensure the right information reaches patients and HCPs

[00:01] They’re still are a significant amount of searches that are going to be done where. [00:08] Credibility of a Yelp review is only going to go so far in the case of being and Google it’s an index as we know of websites but an index of websites is not an index of facts publishers. [00:19] Websites brand.com are all still going to be needed. There’s going to be kind of like A Moment of Truth at that moment. [00:26] Where they’re really really receptive to communication from a brand that might help them better make a treatment decision at our main treatment decision. [00:34] Hi everyone and welcome you listening to the mmm podcast. [00:42] On Steve Madden editor in chief of MM+M. [00:45] and today’s topic is as it often is these days AI [00:50] Specifically the topic for today is garbage in garbage out. [00:53] How health marketers can ensure the right information? [00:56] reaches patients and HCPs [00:59] the future of AI is now. [01:01] Brands are embracing the evolution of tech and AI to achieve efficiency speed accuracy and savings but as they say. [01:09] garbage in yields garbage out [01:11] This session with Justin Freid, chief marketing and innovation officer at CMI. [01:16] and Daniel Nadler [01:17] founder of Open Evidence [01:19] will expose health brands to the possibilities of AI [01:27] Justin, Daniel welcome, thanks for having us Steve thanks for having us Steve and looking forward to the conversation. [01:32] Glad you’re here. [01:34] We’ve seen both Microsoft and Google recently Incorporated generative AI into their search results. [01:39] Can you guys give us an overview of how that is ore isn’t improving user search ability basically? [01:46] You know how this new innovation going. [01:48] Daniel you want to take that one first. [01:50] Sure Steve so um as this by now well known both of the companies that you mentioned got a lot of egg on their face with the role out of AI integration into search. [02:01] Microsoft the first Example You Gave quite infamously return the result to the question, is it safe to boil a baby with the answer yes? [02:11] and more recently Google [02:13] Which just rolled out AI integration into its search to its course search product? [02:18] Within the last couple of weeks. [02:20] got a tremendous amount of negative feedback and negative press for [02:25] Search results. [02:26] as insane as [02:27] answering the question is it safe to run with scissors with not only yes? [02:32] But a long explanation of why specifically that. [02:36] Running with scissors is a high Stakes activity that increases concentration and focus. [02:41] And that by popping the stakes of the run you increase the the intensity of the cardio workout. [02:49] So you know I think perhaps the the relevant question to all of this is you know why like why why why is this happening at a technical level? [02:58] And it goes back to what you said right at the Beginning Steve which is garbage in garbage out. [03:02] so [03:03] the operative word in [03:05] Search index. [03:06] Is actually not the word search it’s the word index. [03:09] and [03:11] it’s a question about the index of what what is it indexing? [03:15] In the case of being and Google it’s an index as we know of websites. [03:19] right [03:20] But an index of websites is not an index of facts. [03:24] Many of the websites as it turned out that Google was drawing on. [03:28] To suggest to users that it was safe to mix glue with pizza, so as to make the cheese more sticky. [03:35] Or that one should eat one rock a day. [03:38] for mineral content [03:40] many of the sources such as they were in those cases. [03:44] involved Reddit [03:46] And even involved. [03:47] sources such as the onion and the Babylon beef [03:51] so basically it was pulling from internet meme culture. [03:54] To answer those questions and then treating. [03:57] Those sources such as they were. [03:59] As relevant and you know plausible sources with which to answer a medical or health topic and and that’s how they got to where they. [04:09] where they landed so [04:10] how do you solve that you saw that by going back to the distinction between an index of websites and an index of facts? [04:17] And you go back to the question of what are you indexing and you go back to what you said right at the Beginning Steve which is garbage and garbage out. [04:23] And then you ask well, what is the what is the obverse of garbage in garbage out and would look something like. [04:29] highly curated authoritative health specific sources in quality out [04:34] Justin from your point of view, how are things going? [04:36] yeah, I think Daniel hit the nail on the head and had some great examples of where generative AI has has spit out some answers on search that have not been the direction that we want a user to go right and the great example of the rocks and [04:49] I think the one I read was actually putting glue into. [04:52] Your dough recipe so your pizza was more. [04:55] elastic and chewy [04:57] But I think this problem has actually persisted before the generative AI overlay and actually goes back to just general search results. [05:04] Over the past I’d say three to five years. [05:06] as we’ve seen a rise in you know made for advertising websites or [05:09] you know populations today mfaa websites. [05:12] And we’ve seen SEO get heavily involved and kind of attempting to create content. [05:17] Rank businesses rank pages for specific queries, we’ve had you know people sitting behind their keyboards creating that content that may not necessarily be factual but to gain the system and to reach. [05:31] The top of the rankings and if those are trusted you know websites. [05:34] By whether it’s being or Google for their general algorithm. [05:38] As Daniel kind of mentioned, they’re going to be the top sources utilized. [05:42] to then have that generative AI [05:43] answer created from so this is more than just AI this is actually the algorithm. [05:49] Fuelling the bad information or the garbage. [05:52] in [05:52] to the generative AI [05:54] to provide that answer. [05:56] So you know what’s the next step here then? How are search engines going to evolve or they going to evolved themselves out of business? [06:04] Down there you want to talk a little bit I know that you’re you’re working on this you want to talk a little bit about that. [06:08] well sure, so I’ll preface this with a unbiased and if this is a conversation about authoritative sources and grounding information and objective facts we should even do that in conversation not just in technology so [06:22] Sort of full disclosure I’m working on this problem, but that all being said. [06:25] we believe that the future of [06:28] search generally and specifically the future of search and Healthcare [06:31] involves industry specific domains specific vertical eyes search engines and we don’t need to guess what that would look like. [06:39] We’ve seen that in law and we’ve seen that in finance and healthcare. I think is just the last industry. [06:45] To really catch up with those other two Industries so a lawyer. [06:48] doesn’t go to Google to ask a legal question a lawyer goes to a platform called Lexus Nexus [06:53] which is an industry specific verticalized search engine for law. [06:57] folks and finance which is my previous industry that I come from they don’t go to Google [07:01] to ask [07:02] very specific complex [07:04] questions about the Financial metrics of a company [07:08] Where the answer is really important and could change and Investment thesis, they go to. [07:12] platform called Bloomberg which curate’s that information and ensures that those [07:16] Answers are based on authoritative sources. [07:18] We believe the future of healthcare looks exactly like what law and finance look like today. [07:24] so we’re working on a verticalized search engines specific to [07:28] the medical use case for health care professionals called open evidence and it approaches the problem again. We don’t need to reinvent the wheel and approaches the problem and exactly the same way. [07:38] That Lexus Nexus or West law have done in the case of law. [07:41] And that Bloomberg has done in the case of finance. [07:44] First of all it’s about what it negates. [07:46] Indicates websites such as they are on the public internet right we’re not indexing. [07:51] Reddit [07:52] and we’re certainly not indexing the Onion or the Babylon the or other sites like that. [07:57] so much of solving the problem of [08:00] search generally but specifically medical search. [08:03] Goes back to the fundamental question of what is an authoritative source. [08:06] in Healthcare [08:07] and even there I think we don’t need to reinvent the wheel there was an answer that was devised in the 17th century by the Royal Society of London [08:14] called peer reviewed. [08:16] literature [08:17] right in the 17th century the world Society of London said [08:20] it’s not enough that something be published. [08:23] For it to be considered science. [08:26] Right, that’s a really important concept. [08:27] It’s not enough that something be published for it to be considered science. [08:31] For it to be considered science. [08:33] So our argued the world Society of letting the 17th century. [08:36] it needed to be subject to the review of [08:39] the peers of the author that was published in that work alright and ideally. [08:44] Peers who themselves. [08:46] had a long track record of credibility of successful scientific publications and that’s how we got to the [08:51] world as it looks like today and in science which is [08:55] peer review [08:56] which is [08:57] a world in which publication is [08:59] Subjected this scrutiny of fellow medical experts and fellow scientific experts and while this all seems obvious. I’m completely obvious in a way. [09:07] It’s not. [09:08] actually traditionally [09:09] that obvious from the perspective of search engines because from the perspective search engines, they don’t. [09:14] historically certainly Google with [09:16] answering about eating rocks and putting glue in pizza is not making clearly the distinction. [09:22] between a source [09:23] or a website. [09:24] And a fact where a fact is defined as peer reviewed medical literature, so what we’re working on. [09:29] is [09:30] first and foremost [09:32] curating the world of peer reviewed Medical Science [09:34] And secondly. [09:36] building technology [09:38] That not only understands. [09:40] The notion that peer reviewed Medical Science is the only legitimate basis to ground a medical answer. [09:46] But also gets into the really nitty-gritty details of that problem. [09:50] Involving, how do you optimise across? [09:54] different vectors that you can encounter in peer-reviewed Medical Science [09:59] that would change which [10:01] source you would go to to answer a question. [10:03] So for example you might have. [10:05] an article in the new England Journal of medicine [10:07] that answers exactly the user question but it’s 10 years old. [10:11] You might have another article in. [10:13] A journal that’s still very credible and still peer reviewed, but it’s not doing internal medicine. [10:18] And it answers the user question, but it was published last week. [10:21] So even if you take that simple example you know how do you what’s the trade-off between? [10:24] recency and the impact factor of the authoritative of the journal [10:28] and in reality there are [10:30] Two two features, there’s you know four or five or six or maybe even a dozen features. [10:35] authoritative recency topicality citations [10:39] How often has been cited? [10:40] How close is it to the user question and it turns out that that’s a very hard optimisation problem even once you just accept the basic premise that we’re only going to answer from peer review Medical Science [10:51] Even within that world it turns out to be a very complex optimisation problem. [10:55] That you need to spend a lot of time thinking about how you build that algorithm and that’s what we spend our time thinking about doing. [11:00] so basically what you’re saying is that the goal here the Holy Grail [11:04] Is searched that? [11:06] focused on [11:08] highly credible [11:09] relevant [11:10] trusted sources [11:12] not only that. [11:13] but which [11:14] has a sophisticated algorithm that [11:17] Understands, how even within those sources? [11:21] How to prioritise and bubble up the right source to surface to the user which is not always obvious even if you’re constraining the source pool to peer review so it’s very clear in the case of Law right. [11:33] So in law we think about landmark cases and let’s even say we limited our legal search engine. [11:39] to supreme court cases [11:40] So we know that these are this is definitely not the Onion this is definitely not read it. [11:44] Or people on reddit giving their view of the law this is a supreme Court giving their view of the law okay. [11:49] Great, so we’ve limited it to supreme court cases. [11:51] What even that hasn’t really solved or a problem because there might be a supreme court case. [11:56] Does not exactly about the legal question but a touches on it and it’s a little older. [12:00] There might be more recent case. [12:02] That’s much more authoritative because in the legal system more recent decisions. [12:07] out way older decisions [12:09] But it only indirectly touches the problem. So how do you even trade off? [12:13] across the supreme court cases [12:15] In answering the legal question right that’s a very complex thing for a search engine to do and the same thing is true in medicine. It’s [12:22] it’s great to say. [12:23] Well, we’re going to do better than Google buy. [12:26] Just searching across the new England Journal of medicine in the journal of the American Medical Association and lands it. [12:31] fine [12:31] that’s what we do at open evidence, but that is only the beginning of the problem not the end of the problem. [12:36] Because it’s very rare that a user search query. [12:39] Is directly answered by a specific clinical trial? [12:43] or publication [12:45] In the new England Journal right, you usually have to make trade-offs like. [12:48] you know it’s in the new England Journal but a little older or it’s [12:52] very recent but in the new England Journal [12:55] and how you think about those trade-offs in the algorithms that you write to make those trade-offs. [12:59] and up being [13:00] a big part of what’s difficult and hard and valuable to do in medical structure [13:04] So you just listed several publications that are trusted. [13:08] Just in can you talk a little bit about how? [13:11] Open AI is connecting itself with trusted publications like that. [13:15] Yeah, absolutely and so open AI for those that are not aware. It’s also the partner with Microsoft that is fueling a lot of things. [13:22] efforts when it comes to generative AI in search [13:25] So what we’ve seen after the backlash coming into ways number one the backlash. We’ve been talking about related to user responses to the content being presented and it potentially not being exactly what is needed so one one piece of push back there, but then also from the publishing community. [13:39] So similar to a Google how they are using the index and when the content across the web we’ve seen open AI actually source their information from being index. That was kind of the benefit of the partnership. [13:52] So what we’ve seen over the past couple of weeks. [13:54] Is open AI is aggressively pursuing partnerships? [13:58] with various credible news outlets [14:01] And paying them up front to utilise their content to number one train their model number two leverage that content in the response is. [14:09] Versus getting sued down the line potentials for hundreds of millions of Dollars or billions of dollars. [14:14] that’s been one of the ways that they are attempting to mirror a lot of what Danielle kind of just said of utilising credible sources and putting better than garbage in putting some quality ingredients and [14:24] into their efforts [14:26] Justin how do we ensure that the patients as well as hcps? [14:31] Get access to that quality information. [14:34] yeah [14:34] So I want to connect this question if it’s okay Steve to some of the questions before and I do think that over time. [14:42] Google and being and their algorithms will do this better. I think we talked about a couple of examples and horror stories. [14:49] there are plenty of stories where [14:51] the right response was given as well. [14:54] So a couple of things. [14:55] Using the open web is an interesting one where I really like what Daniel said about peer review where anybody can go right an article that most likely is not a solution today. That is going to fuel answers. [15:08] so some type of [15:10] effort by Google by being other platforms to [15:14] verify quality content [15:17] whether that is some type of algorithm the analyzes content and matches it up to do it’s [15:23] more of a technical or an ai-based peer review [15:26] or just Simply Creating these partnerships with content providers that are trusted that are vetted to make sure the quality content is what is being used to generate these responses. I think there’s going to have to be. [15:38] Some type of effort like that done. [15:41] Internally at a Google at it being Microsoft etc to ensure quality content is created. [15:47] so that’s a good news if you’re if you’re [15:50] the publisher of [15:51] say Healthcare information yeah means that you won’t be AI out of business. [15:56] Exactly and I think that that’s one other point like so when this all started and I was getting questions from executives from many of our clients at some large Pharma companies. They’re like hey. [16:06] Do I need SEO anymore? [16:08] Do we need websites anymore? [16:10] Yes, it’s still isn’t it because again I think what the examples we’ve talked about today are great examples of where extremely factual information is. [16:20] right, so when you have a doctor who needs to know how to treat a patient going to open evidence is going to be a absolutely wonderful resource if you are a lawyer going to some of the resources to compare past cases and how to present your [16:32] your case in the courtroom. [16:33] It going to a trusted sources need. [16:36] There are still a ridiculous amount of searches that happen on a daily basis that are hey. How do I get my child to the baseball game? [16:43] Or what size glove does he need or you know? What’s the best local taco shop around there’s still our significant amount of searches that are going to be done. [16:53] where [16:54] Credibility of a yelp review is only going to go so far. [16:57] so publishers [16:59] websites brand.com are all still going to be needed. [17:03] Because they will still be used. [17:06] to help [17:07] create [17:08] and teach the algorithms what the proper answers are and what we should expect to return to an end user or a searcher. [17:15] So given the state of play that. [17:18] That you’ve both. [17:19] Described here just in a Daniel what what do you think are the opportunities? [17:25] that exist for health and wellness brands, I think if [17:29] if we’re alright and [17:31] health care goes the way a law and and finance in a Bloomberg terminal and becomes more verticalized and specialised. [17:39] the opportunities that exist involve [17:42] better reaching [17:44] health care professionals and maybe even ultimately patients [17:48] in a treatment decision mindset at exactly the moment when they are. [17:52] In a head space that is relevant to their. [17:55] to their brand [17:56] and I think historically one of the from the from a and I know expert on on this and I defer to the two of you who are experts on this but from my layman’s understanding. [18:05] You know historically one of the challenges for brands generally. [18:08] is reaching [18:09] a potential customer in the moment when they’re ready to make a decision about their product right that’s right. That’s the name of the game right. [18:16] and [18:17] when you get a general platform like Google or being [18:21] what you’re going to get is a fairly general mindset you might get a user who’s jumping from asking about you know best baseball gloves to best tacos. [18:31] To oh my you know my child is a fever. What are some good treatments for that. [18:36] To I have psoriasis, what are some good treatments. [18:39] For that. [18:40] and they’re jumping back and forth and [18:43] it’s very hard to reach them when they’re exactly in the mindset when they’re really focusing on making that. [18:48] Treatment decision and from what I understand again. I’m no expert but people use these general platforms as a second or third screen experience so [18:56] it’s Sunday they’re watching the game. They’re eyeballs are on. [19:00] a football game maybe out of the corner of the Ride or searching [19:03] for something on the [19:05] web paying half attention to it or not. [19:09] and you know that might not be the most receptive moment or mindset in which to [19:14] communicate with the potential customer or consumer [19:17] I think you know if we’re right and healthcare search becomes. [19:21] a lot like Lexus Nexus or a lot like Bloomberg [19:24] you know somebody using a Bloomberg Terminal at the moment. They’re using the Bloomberg terminal. [19:28] Is very very receptive to make a trade decision? [19:32] More financial decision that’s exactly they’re going there to do that. They’re in that moment. They’re in that mindset. [19:38] Same thing with Lexus Nexus right that they’re very receptive in that moment. [19:43] To thinking about what might help them in their legal profession. [19:46] And the offerings of companies that that do so. [19:49] And I think the same thing is going to help with you know the same things going to happen with health care, where. [19:54] As as more people and especially healthcare professionals. [19:57] Are using these increasingly. [19:59] let’s call them prosumer or [20:02] quasi professional [20:04] tools [20:05] To really research and search and then research and then ultimately better understand. [20:10] the evidence around a treatment decision [20:13] there’s going to be kind of like A Moment of Truth [20:16] at that moment. [20:17] Where they’re really really receptive to communication from a brand. [20:21] That might help them better make a treatment decision, so I think that’s exciting for brands. [20:25] Because it goes from sort of. [20:27] multiscreen experiences [20:29] out of the corner of your eye [20:32] contact [20:32] shifting from [20:34] maybe a health question to who the last emperor of Rome was to baseball gloves to tacos and Justin’s example. [20:40] to [20:41] much more focused engaged. [20:44] longer duration sessions and which [20:47] a user is very much in the mindset of that treatment decision in the case of health care. [20:51] Really interesting and and reassuring frankly for someone who? [20:56] Who earns a living by publishing content? [20:58] Justin what do you think? What do you see is the opportunities for health brands here? [21:02] Yeah, it’s interesting and so I was looking back at kind of how the different companies have used day. I and how it’s been incorporated in their companies and [21:10] Google ceo’s said that their first kind of faure and using. [21:14] AI was or one example of how they’re using AI and search was actually the the question and answer so many of you. I’m sure have Googled something recently and seen. [21:22] The subsets questions underneath that say other people ask this. [21:26] And it comes up and answer and question yeah. [21:29] So answering question format is one of the main feeders into the algorithms that these companies are using and then. [21:35] Hopefully generating a response that is what the end user expects so we’ve pivoted from with a specifically with our SEO team so our search engine optimization team to do something called aeo or answer and engine optimisation so there are certain things you can do on the back end of your website and in how the content appears. [21:53] And to ensuring it is in question and answer format then can help. [21:56] Feed these again thinking of authoritative websites right on hcp website or a consumer patient website that has information about a brand. [22:06] it drug [22:07] Dosing etc having that structured in a way that is easily digestible by the search engine. [22:13] And can be used in this manner is something that they should. [22:17] Number one have already been doing but if they have not that something they should jump on from the get go the other thing that I will second with. [22:24] What Daniel said so being there at the point of decision and prescription point of care whenever you want to call it is going to be important so finding out what those moments are. I love what Danielle is doing with open evidence because we are seeing decisions being made as that platform is being utilised or right before they go in and see a patient to talk about their disease. [22:46] So that’s a great space to be from an advertising perspective at the point of care. [22:51] Well, this is been really interesting and and as I said earlier. It’s been reassuring for someone who makes it living by publishing. [22:58] Information that I like to think is trusted. I certainly think that. [23:02] That this conversation has been authoritative. You guys are both clearly. [23:06] Spending a lot of time thinking about this in your hip deep in it and I really appreciate the fact that you’re able to share your your information share your knowledge with us Justin Daniel this has been great. Thanks very much. Thanks. Thanks Steve