Reddit Citations Drop, Unstable AI Search Data and Agentic Shopping

0:03Hello everyone, welcome to our next episode in marketing to agents podcast and as always I have Lalit with me. Hey Lalit, great to have you here.

0:13Hey Ashwani, great to be here.

0:16Fantastic. So Lith a lot like always again this space is evolving so fast and as always we have tons of things to talk about tons of interesting news to cover and so let's dive right in. The first news is very interesting and I think is touching a few segments and a lot of players are also involved in that. So we are seeing that u there is something going on with AI search. So for example the core AS search there is some risk in measurability now and it's not as stable or constant as as it was meant to be. So the first data that we saw coming out of chat GPT was that suddenly the citations for Reddit dropped by almost 86%. So whatever Reddit was wherever Reddit was getting cited in chat GPD it saw a significant drop. We did cover this deep analysis on AirPulse as well. But it was interesting to see that suddenly just with one change on the backend algorithm suddenly now Reddit links are no longer being getting cited in charg. another interesting news at the same time which happened was search console. So Google admitted that there was some search console error. So wherever we see

1:36Generative AI results, impressions, there was some error in the data there. So again, AI search is already such a new domain and with these inconsistency in data and measurement and logging, it's hard to understand what's the correct metric that we're looking at that we're not looking at. U so let first of all, let's get started with this and then we'll discuss more offshoots coming out of just this news. what do you think about this? What's happening? Why is there so much shifting going on in the way it's getting measured or the way the citations or things are getting reported?

2:12Yeah, I think AI search and the change in the search paradigm that is getting prime time spotlight and that is putting lot of these numbers the the expectation of accuracy under stress test. Right? Earlier it was maybe getting discussed in some niche blogs but now these are all part of mainstream conversations. People are reporting real revenue impact. Public companies are getting affected. So I think when when you know anything gets spotlight I think initial few days or initial few phases I think takes for stabilization. That is one. second is I think with the spotlight PE people like Charg are are basically sort of revisiting their quality criterias when it comes to let's say you know trusting various sources and on the same on the same timeline what is also happening is that since people knew that Reddit is getting cited a lot of let's say spam attention or people trying to game the system all of those kind of entities have gone extremely active on Reddit. Although their spam control and moderation you know what what you say moderation systems are pretty decent but still I think you know this these scammers are a step ahead right so

3:32Quite obvious that they the charg would have seen let's say a slight dip in quality and they might have internally sort of you know the AI might have internally decided to sort of give preferences to other sources of information which are more editorial right and not UGC. So it's a dynamic surface area and I think right from let's say some bugs getting surfaced because people are now looking at data a lot more you know intentfully and sort of cross-checking data and then secondly I think quality parameters getting revisited and third is I think u it is it is going to be a race between scammers and these AI engines right so this is probably all all of what we are seeing is kind of like everything coming together into some of these issues getting surfaced Absolutely. However, a couple of points that now that you delve deeper into the scammer part. So for the longest period of time at least I've known this for about 6 to 10 months for the brands, Reddit was a holy grail, right? So if you are supposed to build your AI search, AI overviews or visibility in AI searches, Reddit was supposed to be the holy grail where if you do well, if you're able to get good answers or good mentions there, AI searches would pick you up. Now

4:50You mentioned about there could be scammed and therefore could be scams which are getting captured there. Now couple of points there. So number one is that first of all so I'm assuming that chat GPD is still or any any such AI search would still be crawling all of these sites and maybe they are still composing their answers using that information but they're no longer citing that would you think that or you think that now even the crawl that may be happening on Reddit that itself is going to significantly go lower that's one part but let's yeah discuss that and then I have another theory on why this could have happened.

5:33Sure. So I think crawling and so there are three ways in which you know an AI system can interact with a publisher like Reddit right. First is where they have to index the index the content and so that the traditional search that we know of right where if you hit a web search API your content has to be indexed and then you should be able to appear in the search results. So that I think every every publisher allows but is also a a walled publisher where you have to have an account and so not everything is public totally depends upon the subreddit policy and overall content policy. So we can safely assume that the content is available for indexing but not for let's say pre-training and it so pre-trending is a separate licensing agreement [snorts] and the third thing is inference time instant inference time access where someone is trying to as a as a user if someone has a query and then real time the agents are fetching the results and composing an answer that is also now governed by content content licensing policy. Right. So I think Reddit still has an agreement with CH GP to the best of my knowledge. I think they have not

6:49Revoked that. Google I think has not renewed the the licensing agreement with Reddit. But what we're also seeing is that you know Reddit has has been in the spotlight with respect to let's say one of the levers to improve AI visibility for for a while now. And you you'll be surprised to know that just last week I came across a website where Reddit accounts with good karma were up were up for sale. So you could basically buy an old Reddit account with good karma and then use that account to start sort of you know generating content or publishing content. So such white hat or whatever you might call you know these let's say scammy practices or the the as these practices sort of grow in volume the systems are intelligent enough to sort of figure out that you know user engagement is dipping people are maybe citing it as incorrect right so it's a self-corrective measure where systems are you know preferring slightly more editorial more authoritative information sources

7:53I agree I agree And what you mentioned about u their deal with Google following through actually that was my other theory behind this that maybe somewhere their talks with Chad GP although they're still partners but maybe that have those talks have fallen through or maybe those have ended. We are seeing the impact immediately now but maybe in a few weeks that gets announced where either Reddit would have signed an exclusive massive deal with some other bigger LLM player and thereby it's cutting off from the generic ones or it whatever deal they had with Chav that would have fallen through and therefore Chad Gypy might have taken them out as a preferred publisher. was just thinking that maybe there is some other bigger motivation as well because this would again this would be exactly immediately get noticed and interestingly no other LLM engine has yet taken any action against or hasn't seen any change in the citations from Reddit so it's interesting that this happened only for yeah

8:58Yeah so I think Reddit has taken some really visible steps in the direction of building their own AIcentric products. So they're working on their own some sort of an answer answer product and also AI generated podcast from threads and AI generated videos from let's say you know summarized content. So whatever is breaking news and whatever people have discussed Reddit would be ready with let's say a video right where all of what was discussed in the Revit Reddit communities for that topic it would be summarized as a podcast or or as a video. So they are themselves optimizing for let's say eyeballs and you know advertiser revenue. So not really sure what might happen there but my personal theory is that you know all content creation right will be conversational. I think world has discovered podcasting as a as a great way for naturally letting the conversations flow and all information access will be personalized right publishing will be through conversations and the information access will be personalized through you know your own agent or your own preferred AI answer agent. I agree that's a good that's an interesting point and then let's so just now taking an example from so now for brands for brands now how what does

10:19What changes for the brands so for example now of course Reddit seems to be from charge yes it's now not quoted that much now for brands how do they figure out where should the focus be so do we not generally go after such UGC content like Reddit was one example but it could be Kora, it could be Twitter, it could be many other places where a brand today publishes or tries to get the third party authority by having mentions for their brands. So again whatever is can be done organically where if there are news articles which can quote you that's separate but from a brand then how should the brand strategy change now if they want to get ranked on an AI search or get mentioned there because looks like this publisher agreements or these could be just these could still be very unstable. So what should the brand takeaway be from this particular aspect?

11:14Yeah. So u so first of all I think this is a third party study which has cited that you know for certain sample queries the Reddit citation share has decreased. So it may not be a very significant event, right? Maybe it could have been incidental also. Maybe it's a internal update into charge's way of ranking. So it's not something that that changes our perspective of Reddit with respect to AU activities. but principally I think brands should continue to focus on all genuine efforts right anything which they are particularly doing for let's say optimizing for AEO or let's say scaling some efforts by deploying agents. I think those activities will continue to get arrested and would get let's say you know even if they get unnoticed they would they would not really move the needle genuine efforts of a brand to sort of address any grievance that have been you know expressed on social media or Reddit that has to be there you people need to deploy AI to at least extract more and more of those if they have not been able to do it. Second is you know across all social media platforms I think now default setting for anything that you publish is public right whether it's LinkedIn or Instagram it's default public so that people can crawl so so

12:34I think the surface area where reputation can be affected for any brand is now much much much wider not just on relate so all genuine effort should continue to be invested into and slightly scaled in terms of making it more efficient but anything which is particularly particularly being done to gain the system or scale automate automation for generating content much rather just stop investment investing because from all directions we are getting the signal that the content which is purely AI generated for AI is getting zero signals right

13:13Oh yeah absolutely that is 100% true so I think the third part the third the for example We covered in our last episode. We did cover about Trust Pilot, G2. Those seem to be still relevant because those are your champions, your customers who are giving positive feedback and those are still valid signals. But yeah, could be and but yeah, other third party UGC content publishers could be if it is genuine should still continue doing it. and anything which is done specifically to get AI mentions or AI content just be wary about not investing too much into it because as Google has taken steps against it there would be others who will follow in suite and might might not u accommodate so much again similar to what during SEO era the same same rule of thumb was that don't do things just to attract the you know bots and the crawlers rather do things for the user general organic things.

14:15Yeah. So Ashwani I I think this is going to be a very fine balance which the social media platforms would have to strike because they certainly want more content to be produced because the economics really work out right more content means more engagement, more creators, more advertisers. So they definitely want to remove the friction from content creation and AI definitely helps. U so my theory is that they might almost every publisher might u launch their own suite of AI tools which would which would which would be positioned for let's say helping a genuine human creator reduce the friction of editing or reduce the friction of create creation and all third party content creation tools where you can just prepare something and just you you're using the platform to upload the content they would be at the outset discouraged Unless something is extremely viral, extremely good that you know sort of scales the organic visibility, organic let's say distribution on it on its own. But by default the starting weightage of externally created and edited content will be much lower as compared to let's say something which is getting created on the platform using the suite of that the platform itself provides like we have seen very recently YouTube is

15:33Launching a lot of these creatorcentric editing tools. you can now edit your you now you can use Google omni to edit your YouTube shots add some effects so those are the things which that would help the the platform to at least capture the signals that this is a genuine human effort being put behind the creation of the content and it's not something that is automated and scaled so we've already seen LinkedIn right if you are using their AI I think they still continue to give you distribution but if you sort of copy paste something from outside I think the distribution is like much lesser. They they're able to find out that it has signatures and patterns. And we recently also saw the news of cloud AI which which which which is going to embed some let's say watermarking

16:19In the content. Yeah. In the content that they're going to be

16:22Correcting. Correct. So so there is see engagement is the golden goose for these platforms. Right. If the engagement drops they they they probably are worthless, right? So they will do everything and anything under the sun to to protect the engagement and in fact grow the engagement. And myself, you know, just speaking as me as one single data point as a user for on Twitter, if I if I know for sure, if I've somehow got to know that a certain account is fully AI powered, right? I just ignore what what they have posted regardless of what how long they how long it is, how interesting it might be. But somehow my filter of ignoring that post when it is in my feed is much higher. So I think they have to really you know like aspire to strike a really fine balance where the friction is reduced while human genuine effort is also you know let's say captured and scaled.

17:21Completely agree one good example of this is this AI generated mini clips mini series. So we've been seeing a lot of those in shots in YouTube shots or in Tik Tok. There are these 30 one minute typical clips and they'll have like 10 such episodes and then they'll that'll cover one story line and it's interesting sometimes it catches your interest. So you're right the quality and the engagement finally decides how good is even if it's a high authority research article where you have original findings but you've used AI to help write that article I think that'll still have a lot of value in it. yeah eventually I think engagement and u the authority the value that you're giving to the user I think will will drive the final decision of whether it gets distribution or not. Perfect. On distribution we we did talk about Google introducing a new feature where you could where you could mention a preferred publisher. Do you want to cover something around that?

18:24Yeah. Yeah. So this is something just u is like breaking news kind of thing. So just yesterday Google announced that publishers can basically add a little widget to their it's kind of like an action button where u the viewers or readers can can sort of you know express that this publisher should be in my preferred list of publishers. Very similar to the Facebook like button. It's a small embed that any publisher can add to their website or any page. And that would then signal u Google right and that this is assuming that everybody's using Chrome they're already logged in so single click works right so but it they made it largely frictionless to just signal from the user that this publisher is something that they want to see more of in their AI overview answer composition right so I think this is a this is a good step because u if if my favorite publisher is getting preferred in the answers that I'm looking for. it obviously also result into me, you know, clicking that right and then going to the website. So in a way this is coming across as a win-win situation. Hats off to the product management team at Google which sort of you know did something which was not very obvious but out of the blue but really nice.

19:42Fantastic. So I as a publisher or as a brand I should enable that button on my site irrespective because because even if those are my customers even if I have you know x number of customers or people who come and check out my website even if they do even if they you know do that I can hope that when they ask something in Google next time and there is some content around it on my site my site will get quoted more. So bas basically I'm one of the trusted sources according to according to the accord that's a signal for the Google that I'm a trusted source. So that should automatically increase my visibility of my content number one. Of course that gives me slightly preference for that user but in general also. So I think I think for every brand even if it's a SAS if it's a B2B any kind of brand if you're publishing even a blog on your website you're publishing your podcast I think you should still have that button because if there's a user who wants to hear more this is something similar to akin to what YouTube has like subscribe so if there's a new episode you'll get notified similarly if there is a content which is relevant and you're asking about it so Google can show results from that so interesting

20:54Yeah and just like the Facebook like and follow and all of these these ma let's say things became social

21:00Created your feed created your Facebook feed yeah

21:03Yeah but in so my point is more about this becoming a social currency in in a way that you know there could be a race between publishers to sort of flaunting that you know in this category we have the highest number of let's say people who prefer us and advertisers would then get their revenue and then it's it's akin to something as you rightly said YouTube subscribers right but YouTube as a medium, it's very hard to get subscribers in the sense that people largely keep consuming the content. The subscribe ratio is is very abysmal. But

21:35This might this might be something that can happen because people are starting to rely more and more on the text text and the factual information which they can easily scan, right? And over there the I would definitely want to read things from a preferred publisher.

21:55M there's a lot of user value as well.

21:59Yeah. Yeah. So again just a quick anecdotal now that we are talking about this. So I heard about this only last month. So now you know this right that you cannot open your iPhone if you're glancing on it from sideways. So iPhone somehow tracks your eyeballs and knows when you're exactly looking on your looking into your phone only then it opens the phone. Now this is a patented technology and when this patent was filed like something so meta engineers at meta figured this out that okay that you can track eyeballs of the user through your phone camera. So they used this to actually track what's happening on Instagram. So even if you're not engaging on Instagram, even if you're not liking, clicking, nothing you're doing, even if you spend one second extra on a post, that automatically is a signal for them within a post also where you're looking is also a signal for them already. So again like just building on that that you're actually the eyeballs are getting tracked and thereby creating signals. So this is again this is of course lot more explicit and powers more search for you. But yeah, interestingly that the eyeball behavior is captured pretty rigorously on mobiles.

23:14Yeah. Okay, perfect. switching gears now, shifting to our next news from this week. Target reported their earnings and very interesting data. So they are suggesting and very they have come out as a very strong supporter of Agentic Commerce. So they are saying that they saw a 3.5 times growth in AI searches AIdriven searches which brought shoppers to their site and thereby doing the commerce. So again it was not purely agentic agentic commerce is what I understand. I still have to delve deeper into what they reported but to me it seems like that there were more buyers coming who've done research on AI they done their research on AI algorithms and AI search and then coming to target and then and then doing we heard similar things from some other retailers e-commerce retailers who had reported their earnings earlier so it seems that their AI commerce or agentic commerce is significantly picking up and now we are seeing more and more validations of these things as well. Yeah, I think u see beginning of this year the open claw epoch moment I think that is where I think it was very clear that agents are going to be a a

24:32Significant entity on the internet and if you look at the the true benefit of agents is basically delivering productivity and just yesterday I was reading somewhere some VC analysis where I think on the B2C side right the the scope of adding Productivity benefit is not too much except for the category of shopping right because people shop, people travel, people want to buy products and definitely for high-risisk categories they want to do research and nobody enjoys that process but then somehow you have to do it right just to avoid you know sitting with a bad choice. So there's a huge anticipation and in fact we are seeing results as well where these agents can significantly improve the efficiency of how let's say people shop right so this data is not surprising at all in fact you know before we end this year I think this number itself should be twice of what it is because we in some sense I think with some recent developments in grogbot And ultimately what I'm trying to say is that agents are becoming a lot more user friendly. They are transcending from the territory of early adopters, techies, nerds into mainstream. Right? So once

25:53That happens, you will see this number where consumers are using this for shopping skyrocketing. Very interesting. So for again just a key takeaway for the brands then again if you are e-com we've been talking about this theme over and over again I think if you're brands you can't ignore agentic commerce we are already we already talked about Visa Mastercard coming out with payment solution to help to help process aentic commerce payments. So we are seeing more of that coming and all the ecosystem participants coming out and supporting this and of course retailers some of these online retailers are talking about having increased searches increased performance through that. So I think that's a trend that is worth catching on if if you are a retailer and and yeah if you are any facilitator in that journey as well and the buyer journey as well I think it's a great time to be focusing on this part. Yeah.

26:55Yeah. Perfect. So for the last news on our side this week today is around so we've been seeing that although AI search we've been talking so much about AI search and we there are platforms of course like chargity Google Gemini but very little amount of data coming out of them very little data that they share very little data that they provide around okay how many searches where searches is very little analytics only one I think one structured analytics that we get we've started getting from May is what we get from Google search console nothing else so in all of this where it's becoming so important organic traffic is dropping AI searches are increasing with no data at all we are seeing a plethora of now new tools independent tools getting built which are trying to tackle this problem what is your view on that and again if you want to talk about couple more to a couple of these tools which are interesting and what's your view on how sustainable is that finally do do the platforms eventually will need to come in and and provide some relevant tools to the to retailers to publishers to brands what what do you think about

28:13This

28:14Yeah I think the the since the search surface area has been fragmented right I think the last 18 to 24 months months you're starting to see the the growing fragmentation in the search space from 85% or close to 90% Google as the dominant player in search now seeing search getting fragmented at chat GPD perplexity Gemini and then various agents that people are using so information discovery is what I'm calling as search and as it gets fragmented the the understanding of what is getting searched is also getting lost Earlier a brand could very confidently rely on what data Google is showing in their search console and could very confidently assume that this is exactly what people are searching and this is what they have to optimize for. Right? Today since it is fragmented there [snorts] is no one single source of truth. Right? And people are almost you know for for various categories if a certain niche has totally sort of sailed off of Google the advertisers or publishers in that domain would have no clue right and this could happen maybe if you talk about let's say certain very let's say early internet

29:31Cohort let's say teenagers or people are just in a niche who are very much on let's say agents and they're only searching through agents for example developers who are using cloud, right? So that for that segment as an example, right? A lot of what is getting searched will be totally lost from the search analytics which traditionally is available on Google because the maybe for example cloud does not use Google search API they use the duct go right so people who are operating in that niche or particular industry they have to know what is happening right and right now there is no s no single source or no better source of giving this information that what exactly is getting searched so client side capture right so we have tools like similar web which exists as a browser plug-in in a decent number of let's say browsers there's a decent percentage and stat statistically significant presence is there of people like similar web and there are other players also like Alexa and then there are also browsers which are having decent share so client side capture of user behavior about what queries are being asked what percentage of let's

30:49Say you know queries result into an AI overview what what percentage of let's say chat GPT questions result into an ad impression so all of that information as long as the advertise the these platforms are not providing the client side you know ecosystem players can still capture and provide this information and that is what we're seeing so similar web Microsoft clarity these guys have sort of started to launch advertiser products where people can get an answer that if I am an publisher or if I'm a business in this particular industry. What is my chances of getting impressions on charge or what are people even searching in my domain in charge? So right now these this vacuum is getting filled by whatever can fill it. but I think very soon maybe maybe mid of next year or in the best scenario early this this end of this year we should be able to see APIs or MCPS for each of these platform like JP should definitely provide an MCP for advertisers and all of them all of this data can then fill the vacuum that that I just mentioned. Absolutely length. Yep. And yeah, interesting you mentioned about similar web. Similar web launched a tool to track chat GBD advertisements. So as

32:07The ads are we are going to see more ads across platforms. So similar web launched a way to show which categories and what percentage of search queries are now seeing ads at chargpd. So it's interesting again an independent company building some sharing some level of analytics and industry understanding u to help us understand how and what to make of the AI search. Yep. Perfect lit. this has been a great episode. Lot of learnings lot of new news covered. thank you again for spending and sharing your insight spending the time and sharing your insights. listeners, we as always we're looking for your thoughts, your feedback, your comments about about what we are talking about and if there's anything in particular you would like us to delve deeper on cover, we are happy to do that as well. Okay, perfect. Thank you. Absolutely.

33:02Thanks Ashini. Bye.

33:03Thank you. Bye.