[Patterns] ABT

ABT SDK Design

The entire ABT SDK includes numerous APIs and tools to support application development, including:

• ABTCore: Provides the core functionalities of ABT at the lowest level, such as experiment information, traffic control, code pruning, data decision-making, and more.
• ABTServer: Offers middleware for server-side applications.
• ABT-Vue/ABT-React: Provides components, repositories, routing, and other tools tailored for the two major front-end frameworks.
• ABT-Webpack/ABT-Vite: Offers integrations for the two common front-end build tools, such as ESLint tools, PostCSS plugins, command-line tools, and more.

How to implement traffic splitting?

• Use Redis: Store the participation numbers for each control group in different experiments.

• Browser fingerprint + user identity: Ensure the same user only participates in one group for the same experiment.
  Two approaches:

  • Package the fingerprint + user identity + group into a JWT token and send it to the client (less accurate, low cost).
  • Use a database to save the mapping relationship (more accurate, high cost).

• Group assignment for new users: Assign groups based on the traffic splitting ratio defined in the rules.

• Deliver experiment data: Send all experiment IDs and the group numbers for each experiment to the client.

https://github.com/fingerprintjs/fingerprintjs

Frontend SDK Design

<ABTesting name="exp1">
  <template #default>
    <DefaultComp></DefaultComp>
  </template>
  <template #groupB>
    <GroupBComp></GroupBComp>
  </template>
  <template #groupC>
    <GroupCComp></GroupCComp>
  </template>
</ABTesting>

CSS SDK Design

Using CSS Csutom Directives and PostCSS plugin to handle differences in styles between experiment groups.

/* style.css */
@ab-testing exp1 {
  default {
    /* default styles */
    .a {
      /* styles for the 'default' group */
    }
  }
  groupB {
    /* groupB styles */
    .a {
      /* styles specific to groupB */
    }
  }
}

Transformed CSS Code via PostCSS plugin:

exp1-default-a {
  /* default styles for .a */
}

exp1-groupb-a {
  /* groupB styles for .a */
}

CSS Modules Transformation

By default, after enabling CSS Modules, the above code will be transformed into the following JS

export default {
  "exp1-default-a": "hash1",
  "exp1-groupB-a": "hash2",
}

The problem is that: defaultand groupBshould not be exposed to the client. Developer should not write if-elseblock against each target group.

To resolve the issue:

import { chooseValue } from "ABTCore";

export default (function () {
  return chooseValue("exp1", {
    default: {
      a: "hash1",
    },
    groupB: {
      a: "hash2",
    },
  });
})();

The chooseValue function from ABTCore dynamically selects styles based on the user’s group assignment (default or groupB) for the exp1 experiment. Instead of hardcoding styles, the experiment group is dynamically resolved at runtime using ABTCore. This improves maintainability and flexibility in testing.

How to handle the experiment after it is fully rolled out?

At this point, the question of how to prune the corresponding experiment's code arises.
Since the experiment SDK does not expose the experiment group that the current user belongs to, business developers must rely on the experiment SDK to handle code logic for different groups.

This provides a foundation for automated full rollout of experiments. Since all experimental code is handled using the SDK, a simple logic can achieve automated experiment rollouts:

1. The experiment SDK provides plugins for various build tools.
2. During packaging, the plugin analyzes the code (AST) to identify which files correspond to which experiments.
3. The plugin cross-references the latest experiment information to find experiments that have been fully rolled out.
4. The plugin locates all source code files related to the experiment.
5. The plugin prompts developers to confirm whether they want to prune the fully rolled-out experiments.
6. After developer confirmation, the plugin automatically modifies the AST to complete the pruning process.

Completing pruning logic through AST is very straightforward.
For example, pruning components

Before:

<ABTesting name="exp1">
  <template #default>
    <DefaultComp></DefaultComp>
  </template>
  <template #groupB>
    <GroupBComp></GroupBComp>
  </template>
  <template #groupC>
    <GroupCComp></GroupCComp>
  </template>
</ABTesting>

After:

<GroupBComp></GroupBComp>

Before:

ABTCore. call('elxp1', defaultMethod, groupBMethod, group(Method);

After:

groupBMethod()

Code Inspection Issues

Since code pruning during a full experiment rollout needs to happen at compile time, it relies on AST to inspect the code for references to the ABT-SDK. Most APIs in the ABT-SDK require the experiment name to be explicitly bound, for example:

ABTCore.call('exp1', defaultMethod, groupBMethod, groupCMethod);

If the experiment name is derived from a variable, expression, or anything that can only be determined at runtime, the pruning will fail.

To address this, we created an ESLint plugin to enforce that developers must use string literals or other values determinable at compile time for experiment names.

Summary

Introduce the A/B Testing SDK you developed

Background

This SDK was developed to support A/B experiment research and is part of our front-end infrastructure.

A/B testing is a methodology for user experience research.

When designing a product, product designers may want to test multiple design solutions simultaneously and then observe the data feedback from each option. This practice is known as A/B testing, which is used to gather data support through the grouping of users into different variants. In real-world scenarios, there may be more groupings.

Previously, the reason for developing this SDK was that before its existence, completing an A/B testing workflow was extremely costly. The high post-processing costs stemmed from the lack of standardization, as identified through my research and analysis.

Specifically, this is reflected in several aspects:

1. No standardized processes → Higher management costs
   Example: Experiment conflict issues

2. No standardized tools → Higher maintenance costs
   Example: Code pruning problems during full rollout of experiments

3. No standardized APIs → Higher development costs
   Example: A/B testing fully infiltrating business code, leading to excessive cognitive load

Solution Selection

The solution is very clear: standardization.

However, achieving standardization is challenging because the front-end field currently lacks a unified and mature solution. The reason for this is that A/B testing is deeply tied to specific business scenarios, and the variations in business needs make it difficult to establish technical standards in this domain.

Each experiment group in A/B testing can be viewed as a one-time business requirement proposed by the product team. These requirements are highly flexible, which makes it hard to standardize them in the technical field.

Although forming technical standards may be difficult, creating internal company standards is achievable.

Of course, these standards encompass multiple aspects. Here, I will focus on standardizing APIs from a technical perspective.

The primary goal of API standardization is to isolate all A/B testing-related logic from the front-end code from a semantic perspective.

Example 1: Logical Judgments

Example 2: High-Level Components

Such isolation not only improves development efficiency but also lays the foundation for automating future full experiment rollouts.

Of course, the API is just one part of the entire SDK. Our front-end code is scattered across different technical environments, such as BFF built on Node.js servers, activity pages primarily using jQuery, multi-page applications built with Webpack, component libraries built with Vite, and so on.

To adapt to various technical scenarios, we divided the SDK into two layers: core and upper layers. The core layer provides fundamental APIs, while the upper layer calls the core APIs to generate multiple libraries, tailored to fit different technical scenarios (see resume).

In summary, the entire SDK ensures that, regardless of the technical environment, you can always find the right tools for the job.

• For example, automated full experiment rollouts: The SDK includes corresponding command-line tools, as well as Webpack and Vite plugins that can automatically handle rollouts during the build process.

• For experiment traffic splitting, the SDK provides tools that determine the user's experiment group based on user identity, browser fingerprint, traffic-splitting rules, experiment status, and other information.

• For CSS code, the SDK includes a custom PostCSS plugin that recognizes our custom CSS directives. These directives are specifically designed for A/B testing, so a custom plugin is required to parse them.