Our client, a leading international e-commerce business, was investing a significant amount of manpower in the curation of its extensive product catalogue. They are receiving thousands of new products every week, which need to be processed to make the products available in the shop. The manual tasks include standardizing product data as well as selecting the correct type, brand and attributes for each product. Since these tasks are resource-intensive, repetitive, and error-prone, the client was looking for ways to streamline the process.
OpenWT was asked to support the product data process using Artificial Intelligence to automatically detect the type, brand and relevant attributes of each new product. With the planned AI engine in place, errors would be reduced, and new products could be made available in the shop much more quickly, leading to increased revenue and a higher overall quality of the product descriptions.
To turn this vision into reality, the OpenWT team followed a three-step approach:
1. Analyze business processes and pain points
In close cooperation with the client team, we analyzed the business processes, with a special focus on the manual tasks and the most time-consuming pain points. We identified the use cases that should be handled by the Artificial Intelligence engine and prioritized them together with the project sponsor. Finally, we created the full functional specifications on which the AI development is based.
This phase also allowed us to gain an overview of the historical product data and to define the cornerstones of the Artificial Intelligence approach that would leverage these data.
2. Train the Artificial Intelligence engine
Based on the historical data about products that had entered the e-commerce platform in the past, we built a supervised Artificial Intelligence engine able to learn from historic data and apply the same processing to new products. The learning was based on carefully engineered features from the raw input data. Leveraging an ensemble combination of Gradient Boosting and Neural Network algorithms, we created a powerful processing pipeline capable of handling a large number of products.
3. Manage data lifecycle and set up production deployment
In order to integrate the AI engine with the productive infrastructure and to keep the learning up to date with any changes in our client’s product catalogue, we set up a number of workflows that automatically fetch the most recent product data, update the Artificial Intelligence classifier and deploy it. Furthermore, we encapsulated the AI engine in a REST server, thereby enabling seamless integration with the productive systems.
Thanks to the work of OpenWT, the product creation process of our client could be greatly facilitated, and a lot of tedious manual work could be replaced by the Artificial Intelligence engine. Using well-engineered features and cutting-edge technology, new products can be assigned to the correct product category and brand with a very high accuracy (more than 99% correct assignments on nine out of ten products). Once the category and brand have been assigned, more than 80% of the product properties, whose curation is essential for the quality of the product catalogue, can be created automatically by the AI engine. This allows our client to integrate new products into their catalogue more quickly, have more consistent product data due to the elimination of cumbersome manual processes, and increase their revenue.