Using Machine learning for text classification and building a user collaborative recommendation system
App is live on https://sharukhshaikh.herokuapp.com/ you can use username from user_final_rating.pkl for testing the application. You can use "00sab00" as test user also.
The e-commerce business is quite popular today. Here, you do not need to take orders by going to each customer. A company launches its website to sell the items to the end consumer, and customers can order the products that they require from the same website. Famous examples of such e-commerce companies are Amazon, Flipkart, Myntra, Paytm and Snapdeal.
Suppose you are working as a Machine Learning Engineer in an e-commerce company named 'Ebuss'. Ebuss has captured a huge market share in many fields, and it sells the products in various categories such as household essentials, books, personal care products, medicines, cosmetic items, beauty products, electrical appliances, kitchen and dining products and health care products.
With the advancement in technology, it is imperative for Ebuss to grow quickly in the e-commerce market to become a major leader in the market because it has to compete with the likes of Amazon, Flipkart, etc., which are already market leaders.
As a senior ML Engineer, you are asked to build a model that will improve the recommendations given to the users given their past reviews and ratings.
In order to do this, you planned to build a sentiment-based product recommendation system, which includes the following tasks.
I. Data sourcing and sentiment analysis
II. Building a recommendation system
III. Improving the recommendations using the sentiment analysis model
IV. Deploying the end-to-end project with a user interface
The steps to be performed for the first task are given below.
Exploratory data analysis
Data cleaning
Text preprocessing
Feature extraction: In order to extract features from the text data, you may choose from any of the methods, including bag-of-words, TF-IDF vectorization or word embedding.
Training a text classification model: You need to build at least three ML models. You then need to analyse the performance of each of these models and choose the best model. At least three out of the following four models need to be built (Do not forget, if required, handle the class imbalance and perform hyperparameter tuning.).
- Logistic regression
- Random forest
- XGBoost
- Naive Bayes
Out of these four models, you need to select one classification model based on its performance.
Building a recommendation system As you learnt earlier, you can use the following types of recommendation systems.
-
User-based recommendation system
-
Item-based recommendation system
Your task is to analyse the recommendation systems and select the one that is best suited in this case.
Once you get the best-suited recommendation system, the next task is to recommend 20 products that a user is most likely to purchase based on the ratings. You can use the 'reviews_username' (one of the columns in the dataset) to identify your user.
Improving the recommendations using the sentiment analysis model Now, the next task is to link this recommendation system with the sentiment analysis model that was built earlier (recall that we asked you to select one ML model out of the four options). Once you recommend 20 products to a particular user using the recommendation engine, you need to filter out the 5 best products based on the sentiments of the 20 recommended product reviews.
In this way, you will get an ML model (for sentiments) and the best-suited recommendation system. Next, you need to deploy the entire project publically.