Improving Retail Checkout Experience With Object Detection

Blog Image 2

by Dimitris Tsapis

February 5, 2021

Ready to transform your retail business?

Schedule A Demo

As a retailer, one of the most critical things for you to understand is that the true foundation of your competitive advantage rests less in the products you sell and is more about how you sell them. In other words, there are probably a lot of stores that do what you do - but nobody does it quite like how you do it. That, in essence, is what the retail experience is all about.

According to one recent study, the average customer tells an average of nine different people about a positive experience they have with a brand. This is notable, as they tend to tell about 16 people whenever they have a negative experience. This, coupled with the fact that about 73% of companies with above-average customer experiences tend to perform financially better than their competitors, illustrates why this is a part of your operation that is definitely worth paying closer attention to.

In recent years, many retailers have tried to improve the shopping experience of their customers by paying particular attention to both speed and convenience. One of the opportunities they've capitalized on in order to do this involves retail object detection, otherwise known as object recognition software. One example of this in action takes the form of Amazon GO, which uses object detection to automatically count up the number and value of products that a customer will be buying the moment they place them in their cart.

But how exactly does object detection work, and why is it the future of the retail checkout experience? The answer to these questions require you to keep a few key things in mind.


The Power of Retail Object Detection: Breaking Things Down

The concept at the heart of retail object detection is called computer vision, which itself is a subset of artificial intelligence. Computer vision essentially allows computers to not only "see" but also identify and process images largely in the same way a human would. Then, based on the data it has collected, it can then provide whatever appropriate output it has been programmed with.

To return to the example of Amazon GO stores, experts needed to solve six core "problems" in order to provide the retail checkout experience that these locations have become known for:

  • Sensor fusion. Using only computer vision, developers were able to aggregate signals across many different sensors and cameras in a store. In other words, as a person moved from the view of Camera A to Camera B, the larger system still recognized them as the same person rather than two separate people because the sensors and cameras were constantly creating and sharing information with one another.
  • Calibration. Each camera in the store was programmed to "understand" not only everything in its field of view, but its position within the larger store itself.
  • Person detection. Each camera could continuously identify and track each person as he or she moved across the store.
  • Object recognition. Again using computer vision, the cameras were able to identify and distinguish between the different items being sold.
  • Pose estimation. This one is particularly important, as it is what allowed the cameras to detect precisely what each person near a shelf was doing with their arms.
  • Activity analysis. This is how the cameras can tell the difference between a person who just picked up an item to examine it and a person who put the item in their cart.

Once those variables had been accounted for, Amazon GO was able to totally eliminate checkout counters for retail stores because the system could then automatically (and accurately) bill customers for every item placed in their cart. It instantly created a faster, more effective and more enjoyable retail checkout experience for customers, all while dramatically cutting costs and improving efficiency at the stores themselves.

A More Creative Point of Sale Experience Begins With Retail Object Detection

Within the context of the retail checkout experience, this can be deployed in a variety of different ways depending on the store. One of the more creative point of sale applications that a lot of retail stores are using involves creating a true cashier-less checkout experience. In a lot of scenarios, this is completed by installing just a single camera directly above a checkout conveyer with no additional sensors needed.

The type of software needed for this type of configuration can run on any computer already present at a retailer counter, making it easy to install nearly anywhere. That software then processes its video stream in real-time, to the point where after a particular product crosses the "counting line" its quantity and price are added into the customer's bill.

So why is it important to look for technology like computer vision and retail object detection in your future POS system? Because it brings with it a host of unique benefits, chief among them being that fast and convenient in-store checkout experience that customers crave.

Unlike a cashier who has to scan items one at a time, customers can put their items on the conveyor belt side-by-side and let them be counted all at once - dramatically reducing the amount of time they have to spend in the store. This is of course if you're making them interact with a checkout counter at all, rather than going with a "grab and go" format like Amazon GO.

Of course, this type of technology can impact other areas of the store, too. In-store cameras can be configured to optimize the layout of a retail store by tracking zones of customer movement with heat maps. Once you know where your most heavily trafficked areas are, you can better choose which products are visible to customers in an effort to increase sales. This data can be used to optimize product pricing and placement in a way that doesn't just make it easier for people to find what they're looking for (thus creating a better checkout and in-store experience), but that also makes it possible to increase sales by between 5 and 20% in many cases.

retail checkout experience

This type of retail object detection has even grown so sophisticated that it can be used to monitor shelves in a variety of ways. All too often in most retail stores, shelves are either empty for too long because employees aren't aware that they need to be restocked, or items are misplaced and are thus difficult for consumers to find. Yes, human employees can make a visual check of the shelves to make sure everything is in order - but every minute they're doing that is a minute they're not focusing on those more important matters that really need them.

It's a problem that costs retail businesses an estimated $1.1 trillion every year, and retail object detection helps tackle it by instantly alerting employees when items are misplaced or when a particular shelf of hot items need to be replenished. Obviously, this helps preserve the retail checkout experience by making sure that people are actually able to find what they're looking for as quickly as possible.

The Future of the Retail Checkout Experience is Here

In the end, all of this technology is working towards the most important goal of all: creating an effortless, checkout-free experience for retail customers everywhere. Based on how far this trend has come, it's easy to see where it's headed over the next few years: particularly when it comes to solving a lot of the other challenges that are harming the retail experience for customers, like shoplifting.

By combining artificial intelligence with a related concept called action recognition, computer systems can be trained to automatically identify whenever a person is shoplifting on a surveillance camera - all by tracking not people, but the items themselves as they move throughout the store. Whenever the movement of an item deviates from whatever is being defined as "normal," alerts can be sent to appropriate personnel to investigate the matter further - thus potentially helping to solve a problem that costs retailers in the United States a combined $16 billion every single year.

Regardless, it's clear that the retail checkout experience has changed dramatically even in just the last ten years alone. More and more, people are turning to online businesses and other e-commerce organizations to get the items they need delivered in an easy, enjoyable way. For traditional brick-and-mortar retailers, there is a major opportunity on the table to re-gain a lot of that ground not by focusing exclusively on what they're selling, but on the genuine experience that they're offering their customers. Not only do companies that take a customer-centric approach tend to be about 60% more profitable than those who don't, but customers have indicated that they're willing to spend about 17% more on average for a good experience regardless of where it comes from.

At this point, the next step becomes clear: retail businesses like yours need to use retail object detection and other opportunities presented by artificial intelligence to your advantage, not only using them to create better experiences on behalf of shoppers (as outlined above) but to also eliminate a lot of the problems that come with running a retail location in the first place. Doing this isn't just how you differentiate yourself from your competitors in an admittedly crowded field - it's how you offer the types of unique experiences that breed customer trust and loyalty at the exact same time.

Ready to transform your retail business?

Schedule A Demo