A Global E-Commerce Company

a_global_e_commerce_company

Who is Our Customer?

Our customer is driven by a commitment to digital innovation across various industries, with a focus on helping businesses embrace new technologies and realize their goals through analytical and digital solutions. Operating as an R&D centre, the company specializes in e-commerce and retail technology solutions, supporting both internal brands and a range of external companies.

Key partners include a pioneering online supermarket in Türkiye, offering mobile solutions to simplify grocery shopping for a modern lifestyle, and a major retailer with an extensive network of stores and warehouses nationwide. This retailer is dedicated to providing affordable, quality products—from fresh produce to personal care essentials—delivered efficiently through both in-store experiences and a dedicated mobile app.

Problem

There were several challenges that the global e-commerce company faced regarding ensuring the reliability and performance of their mobile applications, particularly under heavy traffic and frequent updates. These key challenges were:

Ensuring consistent and comprehensive mobile test automation for their applications across diverse devices.
Efficiently managing dynamic app versions and builds within a CI/CD pipeline.
Optimizing regression and smoke test execution times without compromising quality.
Seamlessly integrating a robust mobile device farm into their existing test automation and CI/CD processes.

Client: A Global E-Commerce Company

Project type: Mobile Test Automation Transformation and SaaS Device Farm Integration

Solution

The company integrated the digital.ai Continuous Testing platform to address these challenges, enabling seamless mobile test automation and device farm integration. Key components of the solution included:

Mobile Test Automation: Utilizing Appium with Ruby to automate test scenarios for smoke, regression, production, and functional tests across iOS and Android platforms.
Device Farm Integration: Conducting multiple Proof of Concepts (POCs) to select digital.ai Continuous Testing, which offered key features like diverse device support, fast execution speed, easy debugging, and video recording.
CI/CD Integration: Automated app version management through Bitrise APIs, and fully integrated the device farm into the existing Jenkins CI/CD pipeline for continuous test execution.
Flexible Device Selection & Reservation: Implemented dynamic device queries for testing on specific device types and reserved devices for consistent and reliable test execution.
Enhanced Reporting & Debugging: Leveraged video recording, detailed logging, and integration with Allure for comprehensive error analysis and reporting.

Device Farm Integration

The two applications of the client need to work reliably under heavy traffic and frequent updates. To address this challenge, we developed a series of mobile automation projects to facilitate the daily execution of regression and smoke tests. For a thorough execution, we integrated a versatile device farm service encompassing a wide array of diverse devices.

To pick a device farm, we prepared several POCs with different real mobile Device Farm services on the market. Ultimately, our customer decided to use the digital.ai Continuous Testing (formerly known as Experitest) platform in light of the POCs we presented.

Features of digital.ai Continuous Testing that stand out from other device farm services:

Fast Execution Speed
Diverse OS support
Supported Tool Stack
Elaborative Reporting / Logging
Simple CI/CD Integration
Easy Debugging
Full Private Lab
Flexible Licensing Model
Unlimited Parallel Execution
Robust Community & Support
Real Device Diversity
Reasonable Pricing
Video Recording / Screenshot
Beta Version Support
Exhaustive API Documentation
User-friendly Interface

What Kloia Has Accomplished

Seamless Integration

Integrating digital.ai Continuous Testing with the project and other tools was easy, and there wasn’t much need to change the project's structure.

We used Ruby language with the Appium framework in our mobile test projects. By updating the access key, server URL, and app root in Appium capabilities according to the digital.ai configurations, we easily set up an integration to cloud devices and start running tests immediately. In addition, we were able to make our additional integrations quickly with detailed documentation and quick responses from the support team when we had problems.

Continuous Testing integrates easily with various CI/CD tools. Our customer already had an existing CI/CD pipeline created with Jenkins, so all we needed to do in the Jenkins pipeline script was to parametrize our runs using the cucumber run command. This quick change allowed us to configure and execute our Cucumber test scenarios with different parameters such as device version, device brand, and app version, based on the requirements of each run.

Powerful APIs and Versatile Features

Digital.ai Continuous Testing platform provides a large spectrum of features that we found particularly useful in our customer’s projects.

We had to scan barcodes for many of our test cases. With the Simulate Capture feature of the digital.ai Continuous Testing platform, we handled barcode reading cases on the cloud platform. Basically, we simulated the barcode reading behaviour by providing the URL of the barcode images we wanted to read with the phone's camera in the command line.
Thanks to the Appium capabilities of digital.ai Continuous Testing platform, we had more control over the device and application. With the device query feature, we made our device selections more flexible in the XPath format. For example, it is possible to specify your preference for a device with Android Version 12.0 but not manufactured by Google.

Another challenge of mobile test automation is dynamic app management. To integrate test automation into our CI/CD processes successfully, we need to automate all related processes.

In the projects of our customer, we faced the issue of receiving a new app build in the test environment every day, making it very difficult to manage this process manually. To address this, we implemented a fully automated structure to test with the appropriate app builds. The automation process downloaded the last successful build using Bitrise APIs and then uploaded this build to the cloud platform using digital.ai Continuous Testing APIs. As a result, before each test run, we ensured that the tests are executed on the latest successful version of the application. This approach has streamlined our testing process and improved the reliability of our test results.

Since we manage app build versions parametrically, it was also possible to upload old build versions to the cloud and perform test runs based on specific versions as needed.

Select and Reserve Your Devices

With the Device Reservation feature, another API of Digital.ai Continuous Testing, we scheduled and reserved devices for our test executions. This allowed us to perform our test runs consistently and reliably using a selected range of devices from our device matrix, which was created based on the devices commonly used by our users. The device reservation feature offered by Digital.ai Continuous Testing has been instrumental in ensuring stable and efficient testing practices.

Record Failures

Having a video recording feature with detailed step-by-step test execution logs for better analysis of errors in mobile testing is one of the critical features to consider when choosing a device farm. With the Appium log, device log, and video recording features specific to the Digital.ai Continuous Testing platform, we were able to manage report management from a single place by linking report URLs to Allure, our chosen reporting tool, using Digital.ai Continuous Testing APIs without additional integrations or additional tools.

Saving Time and Money

We have created a fully automated architecture with CI/CD integration for four different iOS and Android projects. We have set up over 500 test scenarios, including mobile smoke, regression, production, and functional tests, which are running in the cloud environment on a scheduled and triggered basis. As a result, our regression testing times have significantly decreased: from two days to just a few hours. Additionally, by running our production cases on a schedule throughout the day, we quickly detected and addressed any issues that occurred in the live environment.

results

Results

Reduced Regression Test Time: Testing time dropped from two days to just a few hours, dramatically improving efficiency.
Improved Test Coverage & Quality: Automated daily test execution and production monitoring across a wide range of devices ensuring higher test accuracy and faster issue detection.
Cost & Time Efficiency: Fully automated test architecture saved both time and resources, ensuring swift resolution of live environment issues.
Seamless Integration: Smooth integration into the existing CI/CD pipeline and ease of managing dynamic app versions streamlined the overall testing process.

The integration of these two projects with digital.ai Continuous Testing demonstrates a resolute commitment to enhancing dynamic mobile app testing. Automation projects, seamlessly merged with a versatile device farm, ensure daily regression and smoke tests. Through meticulous Proof of Concepts, digital.ai Continuous Testing emerged as a superior choice. This journey led to a fully automated architecture, significantly reducing regression testing times and elevating testing quality, reflecting our customer's dedication to innovation.