How to Build a Machine Learning Based Recommendation System for Your Marketplace

“When you think about recommending something to someone, there’s a real business reason why you might want to do that.” - Jack Chua, Director of Data Science at Expedia.

Machine learning recommendation systems are not just a trendy feature of online stores. It is a mighty tool that can propel your business to the next level if used strategically. No wonder Jack Chua suggests always having “a great tie-in to the underlying KPI of what you want to drive”. For example, increase sales, customer retention, cross-selling, and so on.

If you’re still hesitating on how exactly to use recommendations to invigorate your business, we invite you to learn from the experience of those who already made it work brilliantly!

We collected the best examples of machine learning implementation in recommenders (including our own development projects) and explain in plain English how to build a machine learning recommender system from scratch. Are you in?

What is Recommender System?

Okay, let’s start with a short quiz. Try to remember as many types of recommendation systems as you can. Think about all the times you shop online or receive notifications from retailers.

We predict that you’ll name:

• Similar product recommendations as ‘You may also like’
• Autosuggestions when you search for some goods or create listings for selling your own products or services
• Recommendations for accompanying goods like ‘Frequently Bought Together’
• Personalized notifications based on your preferences or purchase history

And more other different types.

So let’s give them a common definition.

Recommenders, or recommender engines, are tools that are able to filter the stored information about products, prices, and services and predict what a particular user may like.

Recommender systems are usually associated with online stores, however, they are used in almost all existing domains:

The biggest area of machine learning recommendation algorithms use is eCommerce, which is why we focus on it, paying special attention to marketplaces.

How the Recommending Systems Can Benefit Your Marketplace

Now, you may think about where exactly you can use the recommending system in your marketplace to have the biggest impact. The first thing that comes to mind is definitely a product recommendation that we often see visiting any online store. However, it is not the only way to use a recommender tool. We can help you build a recommender of one of the following types the will bring you visible benefits:

Let’s dig deeper into each of them.

1. Listings categorization: help sellers make the listings searchable!

Let’s regard it from the user's viewpoint. When creating a listing in the marketplace, sellers need to link the product to a category. Though it seems like an easy task, in practice, Facebook found up to 9% of sellers abandon the process without completing the listing because of difficulties with categorization. Some products are hard to classify, either because of their uniqueness or insufficient understanding of the categories tree used by the platform. Moreover, Facebook marketers say that when the categories are selected by the seller it is often less effective than when they are recommended by the ML tool.

Besides, if sellers add a lot of different products, it is a time-consuming routine to find a category for each one again and again. And time is money. Nobody finds it funny to spend it in vain.

How to implement

Facebook strongly believes in the power of an automatic category suggestion tool that is implemented in their marketplace. The AI-powered system allows users to compare uploaded photos of the products to the existing ones in the database to detect the category and approximate price.

The next steps are title suggestion to a listing, automatic photo enhancement, and detecting if the title does not correspond to the image.

"This autosuggest model runs on each of the millions of for-sale listings on Marketplace and classifies each listing into one of the 29 top categories at post time," said Lu Zheng, Rui Lui, and David Kim, AI experts at Facebook.

Airbnb also deals with the categorization challenge. When travelers are looking for lodging, they pay the greatest attention to the photos. If the content of the photo does not correspond to the description, it may alienate a potential customer from booking a room. Airbnb experts decided to implement AI technologies to detect the elements of rooms on photos to categorize them properly.

Categorization makes home tours more structured by grouping photos of the same room type. Also, AI-powered categorization allowed one to check if the information about the rooms added by the owner is correct.

The same is used for checking the indicated home amenities by detecting them by photo.

Benefits for you

As you can see from these examples, machine learning-based recommendation systems for categories allow to:

• reduce abandonment by sellers
• enhance discoverability of listings/products for buyers
• improve the relevance of the category to the product description. 

2. Product recommendations: suggest strategically to keep them coming back!

Buyers love when the system helps them find something they need or, on the contrary, proposes unexpected, ‘serendipitous’ suggestions - to try something that they have never thought of. Both ways can significantly boost sales.

About 50% of shoppers surveyed say they have purchased a product that they were not initially going to buy after they saw a personalized recommendation.

That’s why, first of all, the machine learning used in recommendation systems should be based on your marketing strategy. For instance, they can suggest:

• Frequently repurchased products (food, cosmetics, food for animals, automotive products, sports nutrition, etc.)
• Newly added products (clothes, accessories, electronics, trendy products, books)
• Cross-sell and upsell products that are bought together with basic items.
• Push the goods that stay in stock for too long (‘only one size left’ or ‘near to expire’)

Well, you got the idea!

Secondly, the recommender system based on consumer products should take into account all user’s preferences and their interactions to provide the best-personalized matches to the products. ML-based recommendations learn and improve with every next user’s action proposing more accurate search results.

For example, online fashion stores may ask their customers to add more than just common profile information. They may also gather such data as body shape, age, favorite colors, lifestyle, occupation, attitudes to fashion. This helps the recommendation system to narrow down the scope of items to be more precise with recommendations reaching really high-quality personalization.

Thirdly, to add more value to your recommender, you can implement the more sophisticated features that may:

• Show customers the product in the application (like show the furniture in an interior)
• Help them decide what to buy and why (compare goods and prices)
• Provide the combinations with other products (‘frequently bought together’)
• Propose education materials on how to use the product (video instructions)
• Engage customers with interactive elements (like an online trying on clothes or makeup with the help of AR).

How to implement

One of the eCommerce projects that at one time understood the benefit of data collection and analysis is Stitch Fix. This is a popular online subscription box retailer in the US that combines the outfit individually for each person according to their preferences.

To pick up the right clothes, the website asks its customers to go through a quiz to identify their tastes. Then, the information is processed by machine learning algorithms and recommends the best fitting items based on customer answers. On the final step, a human stylist verifies the adequacy of the recommender choice adding expert corrections and completes a parcel.

As for the additional features described above, there is an example of the Facebook Marketplace’s camera.

This feature allows customers to take a photo of some item (for example, your friend’s sneakers) and then process it with a recommendation system based on product purchase analysis to find similar goods in the online marketplace database. Easy and fascinating, isn’t it?

To make it work, Facebook uses a multimodal ranking system combined with Lumos, an image understanding platform, and DeepText, a text understanding engine. Thus, each item consists of layers of images and text.

Benefits for you

The described features and examples bring us to the conclusion that product recommendations’ main goal is to increase customer retention. As far as the machine learning algorithms evolve with each new purchase and feedback, the recommendations become more and more personalized and accurate. And it attracts loyal customers like a magnet on your customers' loyalty. When your customers feel that the system knows them well and gives them exactly what they want, there is no need to go to competitors where they need to start growing their customer history from the beginning.

Сustomer loyalty is one of the main profitability drivers because it is cheaper to retain old customers than attract new ones. Thus, due to the product recommendation system using machine learning, in just 6 year period, Stitch Fix managed to earn more than $1.2B a year (as of 2017). Stunning!

3. Price suggestions: help sellers and yourself to earn more!

When a seller creates a listing, s/he needs to define the price of the product or service. For sellers, it is difficult to keep a balance between a competitive price and good margins.

The price recommender’s role is to analyze the previous records on the prices for similar goods and statistics of won deals and find the golden mean. Having such a hint can seriously support the intention of the seller to continue the process without hesitations on the price.

How to implement

Let us explain it with an example from our client, Arcbazar.

Arcbazar is a global crowdsourcing marketplace and competition platform for architects and designers. Customers can post their request for a design or re-building of a house project and select from the proposed variants provided by the registered specialists.

We have been working with Arcbazar for several years already helping them with an improvement of their website.

Usually, Arcbazar’s customer path looks like this:

1. Selecting a project type
2. Adding a description and photos
3. Setting the price - an award to the competition winner
4. Posting the request to the platform

The pricing step was a pain in the neck for many customers.

The problem was that sometimes it was hard for a person to adequately identify what the award for their design project should be. They often abandoned the process for fear of estimating wrong and either have hardly any applicants or pay too much when they could save.

We created an AI-driven price recommendation engine to help the client decide on the optimal price based on the history of the previous projects.

The Machine Learning tool underlying the process learns from the 1,500 competitions conducted on the platform since launch. The system is looking for the relations between the project award selected by the customer and the characteristics s/he defined. Each time a new project is completed the database is updated and, thus, the recommender engine learns again and again to perfect itself. 

After the first phase, we went further and suggested an update.

As you can see on the screen above, there are fields for a text description for each competition where a user can type anything he thinks can help designers to understand the project aims. The descriptions can be written in any language and even be written with slang or mistakes. These parameters are not predefined in the system but added by the clients according to their type of mind, which means that the same house could be depicted in different words.

We implemented the Natural Language Processing tool that picks up meaningful words to add their numeric value to the rest of the parameters that take part in pricing. The system should find the pattern dependencies of words and price by itself. This allows making price predictions more accurate. If you need more tech details you are welcome to read the research in our article.

Benefits for you

Price recommendation greatly helps the customers obtain assurance in their pricing decisions and reduce abandonment. They will be able to set a fair price and earn more and as a result, the platform owner also will obtain a bigger commission. 

4. Real-time target notifications: increase your customer retention!

44% of customers are eager to become regular purchasers on the website if they are treated with a personalized shopping experience. Knowing your users and their purchase history as well as some demographic data and geolocation, the system can send group-targeted or individually personalized notifications with special deals and recommendations.

How to implement

You must have heard about Target’s personalization case. They decided to start analyzing the products bought by their customers and define the patterns and user groups. Thus, they were able to identify pregnant women by the products they bought and push them more personalized offers.

For example, if a woman suddenly started buying supplements like calcium, zinc, and other vitamins, the system assumed that she was in her first 20 weeks of pregnancy. And if she bought an unscented lotion, she was probably at the start of the second trimester. Thus, the system could send personalized notifications with discounts and special offers for moms-to-be.

Care should be taken to not cross the line of ethics and prevent uneasy situations. One of them happened when a 16-year-old girl, who had not told yet her parents about her pregnancy, received a notification from Target with suggestions of goods for future moms, and this message was read by the girl's father. Oops!

Another example is Starbucks that focuses on real-time hyper-personalization sending out up to 400,000 unique variants of emails to their customers each week. Instead of suggesting the same snack to everyone just because of its popularity, the system analyzes the individual customer's behavior and preferences and picks up relevant products.

Collecting a lot of diverse data, you may create hyper-personalized notifications as Starbucks does. They collect not only personal preferences, purchase history, and demographics but also take into account third-party data like the weather. That’s why if it is hot outside, the system will recommend cold drinks. Isn’t that sweet?

Benefits for you

Want to boost up your sales? You definitely need the notifications with a personalized recommendation that will encourage your customers to spend more money. By implementing this ML algorithm, Starbucks noticed a 3x increase in the revenue from individuals who came through the special offers. 

5. Time and other valuable resource estimation: get instant estimations for complex services!

Whatever the service of your company, your resources can be measured and optimized. For instance, some services marketplaces can benefit from smart cost estimation for services. Having previous records on already closed deals, the system can predict the time necessary for new projects.

How to implement

In our agency, every day we deal with several requests for project estimation. It takes a lot of resources to collect initial requirements, ask specialists, and calculate the time necessary for each new project.

However, during more than 12 years of being a web development agency, we made hundreds of estimations and as we track all fulfilled tasks during our work, we decided to use this data to quickly predict the time necessary for this or that new project or task according to the previous experience.

We use three tools that help us manage the development process - Clockodo for time tracking, Trello for tasks assignment and management, and Fleep for chat and notification. Combining the data from all three and switching on the power of Machine Learning we created a new tool that allows us to estimate tasks and helps us save time, be more productive and accurate in our estimations. 

Benefits for you

If your marketplace also provides some services (for example, tradesmen services), we can integrate the tools that are used by providers to estimate the requests to quicken the work of your customers or employees. Saving your customers' time for routine tasks may help them become more productive and increase profits. That, in turn, allows you to grow your revenue as well.

6 Steps to build ML recommender from scratch

An ML-based recommendation system works according to the chosen mathematical method and an algorithm that uses the data stored in the database. To create an ML-based recommender the following steps should be done: 

1. Data collection

Data is the king, and the new oil as many entrepreneurs put it. The difference between oil and data is that oil cannot reproduce itself and data can be used to create more data and is endless in its mining. And besides, eco-friendly (unless it is some toxic information).

There is hardly any electronic system that does not collect data into electronic databases. And the more data the more chances to build an accurate recommendation system. We can collect data from user personal profiles, interactions on site, history of deals, surveys, etc.

We can use the following types of data:

• User profile data (user id, user preferences received from quiz)
• User behavior data (rates, purchases, and other interactions of users with the product)
• Demographic data (age, gender, ethnicity, education, income, location)
• Product data (product features and attributes)

Investing in data and its analytics is one of the major steps for any business that can bring insights and solve many business problems.

In the beginning, when you have no data to work with, you may start collecting it right away asking new users to register or fill out a small survey to be able to target your recommendations according to these data.

2. Data analysis

The next step after we collected the data is to analyze it. This is one of the most important stages in the ML model creation process. We need to understand what data we have, what is the size, quality, and nature of the data, if the data is complete or if there are some empty spaces that will make processing that data difficult. Only after making this overwhelming research can we pass over to the selection of the ML model.

3. ML model selection

There are three main ML tasks: regression, classification, and clustering. They are subdivided into many different algorithms.

Understanding the diversity of data and interdependencies in data we can choose the most fitting method and algorithm.

4. Data formating

Depending on the selected ML mode, we need to process the data to bring it to the necessary format.

For example, the data may be collected from structured drop-down menus, unstructured description fields, numeric fields. As the data from all these fields will be heterogeneous, we may need to transform one type into another to make it the one (text into digits). This preparation step is very important and usually takes most of the development time.

5. Coding

When the original data is cleaned and standardized, we may start coding. In fact, it does not take long to code if all the previous steps are made thoroughly. We prefer to code in Python which works seamlessly with ML and has many pre-built components that facilitate the development.

6. Testing and verifying

Of course, before releasing the program, it should be tested and fixed to pass the QA. At this step, we also make a posteriori verification of the ML model that we chose. Even after all the above-mentioned steps, it may turn out that the primarily selected ML model does not exactly fit. It is really hard to predict which algorithm should perfectly fit without testing in reality. But we can quickly change the model to the other one. No worries. Thanks to the previous data preparation, the model change happens really quickly.

How to Apply This to Your Business

Now you know what benefits of recommendation systems you may experience. If you are intending to implement an ML-based recommendation system to your website you need to do the following actions:

• Identify your KPIs that can be improved with recommenders and decide on the strategy you want to use - how you would like to use the data. For example, you may want to increase repeat sales, sell more new products, or reduce operational costs.
• Identify the areas where your customers may need to be guided more.
• Check what data you already have collected: personal, behavioral, attributive, demographics.
• Send your requirements to Greenice for estimation and get a free quote in a few days.

Hopefully, this article makes it easy to understand all aspects of recommenders and gave you a vector for further steps for building your own recommendation system!