August 9, 2022
Topsort's conversion rate
ROAS
Better CPC & CTR
"Thank you very much for all the effort. The implementation of the Topsort API was very good, thanks for all the support. It was actually a really simple implementation, Thank you!"
Erick Alanis
It’s not everyday we get to be compared side by side against our competitors in real action. Here’s how we stack up!
Linio is one of the most popular ecommerce marketplaces in Latin America, offering a wide variety of items from electronics to home goods to toys to clothing to perfume - over 4 million items in total. Early in 2021, Linio approached Topsort with an issue that they were facing. They were excited about auction-based sponsored listings and banners, and have been working with another SaaS company in the space (let’s call it Legacy Provider M) to operate on their platform, but were dissatisfied with the results. The take up among merchants was disappointing, and the monetization results were underwhelming.
Linio wanted to see whether Topsort could provide better results. Challenge accepted!
To test this, Linio's Product team conducted an A/B test, putting Topsort directly against the legacy solution. For Peru and Colombia, the Linio team picked a dozen categories across price ranges and conversion rates (including such popular categories as Appliances, Phone accessories, Shoes, Kitchen, Vitamins, Perfume, and others). For each session, the impression is randomly split into either a Topsort bucket or a Legacy solution bucket.
Given half of the customer traffic was sent to the legacy solution, while the other half was sent to Topsort, everything else was kept the same. Organic results shown to the customers were exactly the same in both cases, but sponsored listings results were different. Differences in the final user experience were drastic: Topsort's integration shows no delay in the sponsored listings rendering at the same time as the organic results, and showed up in the most prominent locations in the search results, instead of a glitchy delay where the sponsored listings would show up after the organic results. Immediately, user experience with Topsort is better.
Engineers from Linio and Topsort worked together to run and deploy the A/B test. On the Linio side, the engineers were responsible for “splitting" the traffic accurately. For the “A” part, they then called on the Topsort API to run the auction and retrieve the ads, while the “B” part continued to be sent to the legacy solution. On the Topsort side, the engineers helped answer Linio's questions, and made sure that everything was running smoothly. The whole deployment process took two Linio sprints with one engineer and during this process teams worked closely in Slack channels.
One tricky thing that Linio encountered was the sync of out-of-stock products. With legacy provider M, sync files were the only way to communicate the availability of SKUs. That of course can be limiting and causes delay in a large scale retail site like Linio. Hence the team was keen on exploring alternative solutions to keep the product catalogue more in sync. With Topsort's nerdy engineering power - here comes the catalogue service!
Catalog Service is an adapter and an API-first approach to keep the products availability completely in sync with the sponsored results.
After a short 2 weeks set up, Topsort's systems kicked into gear, receiving data from Linio queries, impressions, clicks, and purchases. More excitingly, a new array of technical features that were not previously available via the legacy solution now ran smoothly in production: ranging from Autobidding, Pacing, to Quality Score. The data model Topsort built to predict ad quality scores and conversion rates improved Linio's results on the fly, and improved continuously as time generated more session data. Topsort's data science team worked so hard on this project!
About a month into Linio's A/B test, Linio's team decided it had sufficient evidence to stop the test and take a look at the results!
Can we say it was a fully fair test? Mathematically, not really, as marketplace solutions get better with time and operate with less amount of context while being compared to a solution that had been in place for a year, required a level of resourcefulness. However, 4 weeks were enough to put us at a “steady state” and obtain statistically reliable comparisons.
The statistical results were unambiguously better on the Topsort's half. Topsort’s results outperformed those of the Legacy Solution in both countries and on all relevant metrics, including click-through rate, conversion, attributed sales. In one country, the click-through rate (CTR) on ads selected and shown by Topsort was more than double that of the Legacy Solution, and in the other Topsort exceeded that of the Legacy Solution M by over 90%.
Topsort’s conversion rate was 20% higher than the Legacy Solution's in one country, and more than 80% higher in the other country. During the second half of the experiment when Topsort had enough data to run Autobidding and Quality Score, the performance of CPC and CTR were 5x what the Legacy Solution showed.
The total clicks generated from sponsored listings powered by Topsort's half, combined with higher click-through and conversion rates, in the world of auction-based sponsored listings translate into much better monetization results. From the A/B test, one can say Topsort's technology can generate 5x the revenue for Linio than the Legacy Solution. Linio's business team was delighted with the outcome!
While the main goal of the test was to evaluate the performance of Topsort’s technology, it also gave a chance for Linio to experience the quality and responsiveness of Topsort’s engineering team, as well as the ease of integration that was a steep improvement from the previous Legacy Solution. Erick Alanis, a Linio engineering lead on this project, expressed his gratitude to Topsort, saying:
"Thank you very much for all the effort. The implementation of the Topsort API was very good, thanks for all the support. It was actually a really simple implementation, Thank you!"
Stay up to date with Topsort Cookiebites Newsletter
