For more crisp and insightful business and economic news, subscribe to The Daily Upside newsletter. It's completely free and we guarantee you'll learn something new every day.
EBay wants to track your scrolls, clicks and zooms to build a personalized shopping heatmap.
The company wants to patent a system using computer vision to increase "efficiency in user interaction" by tracking interactions when online shopping. Essentially, eBay's technique is another way to monitor user interaction using AI-based technology.
Using this system, eBay monitors user interaction with "digital content." That can be anything, including results from a search query, a digital ad, or an AR or VR shopping experience. Using computer vision, this system identifies images and objects that a user seems to interact with, such as if they're clicking or zooming in, and the characteristics of those objects.
For example, if a user is browsing in an AR shop and they zoom in on a dress, the system will take note of the style, color or material. That information, in turn, is used to categorize that user within a "user segment," which helps eBay "control output of subsequent digital content to the user," such as customizing recommendations and digital marketing.
EBay also notes that its computer vision system can be used for "missing item determination," or determining if items are "missing from a particular context" when you're using AR to shop. For example, if you show this system a photo of a living room with a blank wall, it may use what eBay calls "complete-the-look techniques" to suggest wall art.
EBay is leveraging computer vision the same way that many companies are using other forms of AI: to track user behaviors with the goal of hyper-personalizing targeted ads. Patents for AI monitoring tech from Walmart, Uber, Visa and more prove that companies are desperate to understand what actually makes a user click.
But eBays' tech offers a more personalized solution, said Toby Awalt, VP of marketing at computer vision checkout company Mashgin. Because this tech relies on object image detection, rather than language-based tags or descriptions, it allows eBay to basically perform "advanced heat mapping through watching what somebody consumes."
While eBay can -- and likely will -- use this for targeted advertising, this has the potential to greatly improve search queries by understanding the specific attributes of items a user commonly searches for, said Abhinai Srivastava, founder and CEO of Mashgin. Because most search engines rely on metadata and keywords associated with an image or product listing to decide what surfaces, the results sometimes aren't as relevant as a user would like.
Computer vision like this could benefit any e-commerce company that relies on a search bar, Srivastava said. "The ability to have signals from the image impact the ranking ... I would be surprised if at least the top players are not already doing it."
What may make this tech difficult to put into practice is that eBay and other major e-commerce brands sell "everything under the sun," Srivastava noted. Training AI to catch thousands of different attributes won't be an easy endeavor.
One last consideration: eBay's patent filing mentioned that it can use its computer vision technology with "digital images ... scraped from a social media profile of the user, a photo sharing web page of the user, an email of the user, and so forth."
While computer vision within eBay's own platform can vastly improve search results and ad placements, using that tech on images scraped from a user's social media may take it a bridge too far, said Srivastava. "The line between very relevant and very creepy is a fine one."
Have any comments, tips or suggestions? Drop us a line! Email at [email protected] or shoot us a DM on Twitter @patentdrop. If you want to get Patent Drop in your inbox, click here to subscribe.
