May 4, 2014

Lessons from Amazon’s Personalisation Algorithm

Amazon has been at the forefront of everything related to eCommerce since the term was coined twenty years ago. They have been online since 1995and while many companies from that period have faded into history, Amazon have thrived. They grew from being an online bookseller to becoming the largest retailer in the world. They have successfully moved with the times. Where they have led, many have followed. On average, an Amazon customer will spend $968 with Amazon annually* so it is worth taking notice of what they do. There is much that we can learn from such a successful company.

A key goal of every business is to sell more to existing customers. After all the business has spent the time and money to acquire that new customer, so why not use upsells? Amazon has become well-known for bundling up products in an effort to sell more to each customer. Everyone who has used Amazon will be familiar with the recommendations on their site and the recommendations that Amazon sends by email…..

It is very interesting when Amazon pull back the curtain and describe how they operate. A report by Greg Linden, Brent Smith, and Jeremy York** gives insight into their recommendations algorithm.

The authors describe how they use recommendation algorithms to personalise the online store for each individual. The end result is that click-through rates of web and email based advertising vastly exceed banner ads and best seller lists.

However completing this task is not straight forward. The challenge is to succeed in producing personalised recommendations that are accurate in under a second.

The difficulty lies in dealing with huge amounts of data for millions of customers. When we place side by side the gargantuan amount of customer data with huge product catalogues, it is of little wonder that a lot of thought has to go into getting fast and reliable recommendations.

Added to this there is the issue of having very small amounts of data for new customers and sometimes too much data for long term customers to make accurate recommendations. Customer preferences are also volatile. Customer behaviour might bring new information that the algorithm has to adjust to.

With that in mind, the authors describe the shortcomings of traditional recommendation algorithms. A traditional algorithm will seek to match a user’s purchase preferences with a set of past customers who have a similar purchase history. The algorithm will then disregard any items in the set that have been purchased already and will make recommendations of the remaining items. However this type of algorithm can quickly run into performance and scalability issues. If you have large amounts of customers and they in turn have purchased large amounts of different items the number of iterations becomes very large very fast. This has an adverse effect on the speed and accuracy of the recommendations.

Amazon has developed what they call “item to item collaborative filtering”. This involves making recommendations on the product level rather than on the user level. Amazon has developed their algorithm to recognise products that customers generally purchase together. One approach would be to look at complementary products such as golf clubs and golf balls. However there are many products that do not fit so neatly into such pairs. Therefore the amazon algorithm looks at the similarities between one product and all related products. Customers can also go in and change their preferences manually to ensure Amazon is making the correct recommendations. This helps them get around the challenge of dealing with such huge data sets. It requires only sub-second processing time to generate online recommendations. This algorithm can react immediately to changes in a user’s data, and makes recommendations for all users regardless of the number of purchases and ratings. This type of algorithm gets around the issues of matching huge amounts of user data with huge amount of product data that we discussed earlier.

*http://www.cnbc.com/id/101299263

**www.cs.umd.edu/~samir/498/AmazonRecommendations.pdf‎

Leave your comment