Two-Stage Ensemble Framework For Bot Detection In Retail Web Traffic Using Isolation Forest And XGBoost

This article presents a two-stage ensemble framework for detecting bot activity in retail web traffic using Isolation Forest for anomaly detection, followed by a supervised classifier (XGBoost) for bot verification.

The approach is motivated by reducing false positives in unsupervised models and improving confidence in anomaly classification.

An initial anomaly score is generated by analyzing request rates and session-level behaviors using Isolation Forest.

Anomalous timestamps are then passed to a supervised model trained on aggregated session features to classify the proportion of bot activity.

The approach achieved an end-to-end accuracy of 93% and could reduce manual review significantly, illustrating its potential for operational deployment and offering a more interpretable and scalable pipeline for bot mitigation in dynamic retail environments.

1. Introduction

Online retail platforms face growing threats from automated bots that mimic human users to scrape prices, monitor stock, or rapidly purchase high-demand products.

These bots generate anomalous traffic patterns that often go undetected or result in high false positive rates when using conventional rule-based systems.

Isolation Forest (IF), an unsupervised anomaly detection algorithm, has proven effective for identifying statistically rare request behaviors, particularly request-per-minute bursts.

However, while IF is sensitive to anomalies, it lacks interpretability and may flag legitimate traffic under certain edge conditions.

To mitigate this, I propose a two-stage ensemble approach: the first stage uses Isolation Forest to detect anomalous timestamps; the second stage leverages supervised models (XGBoost) trained on aggregated session features from those timestamps to estimate the proportion of bots.

This architecture improves accuracy and introduces confidence scoring and interpretability, critical for operational bot detection teams.

I evaluated this system on simulated retail traffic and noticed significant gains in precision and reduced manual review load.

2. Related Work

Bot detection in web traffic has evolved from static, rule-based systems to more adaptive machine learning approaches.

Traditional heuristics, such as rate-limiting and IP blacklisting, have been widely used but are increasingly ineffective against bots that emulate human-like patterns (Cloudflare, 2023).

Isolation Forest (Liu, Ting, & Zhou, 2008) is a well-established unsupervised anomaly detection algorithm known for its efficiency and scalability. It has been applied in

fields such as credit card fraud (Zhao et al., 2019), intrusion detection (Kumar et al., 2021), and user behavior modeling.

However, its unsupervised nature often results in high false positive rates, particularly in noisy or bursty traffic typical of retail web environments.

Several researchers have proposed hybrid models that combine anomaly detectors with supervised classifiers.

For example, Alharbi et al. (2021) used Isolation Forest followed by Random Forest to classify IoT traffic anomalies with improved precision.

Similarly, Wang et al. (2022) applied a two-stage ensemble combining IF and XGBoost for fraud detection, demonstrating that this approach enhances confidence and interpretability.

Despite these advancements, literature applying such frameworks to retail bot detection particularly using session metadata like request-per-minute is limited.

Industry approaches (e.g., Akamai) often use proprietary behavioral fingerprinting, but these are not always transparent or reproducible. My work contributes a reproducible, interpretable, and scalable method tailored to high-volume retail environments.

3. Methodology

3.1 Feature Engineering

From raw web logs, I extracted and standardized features.

Create different features for both models; Isolation forest features include timestamps of start and end, requests per minute, number of unique IP addresses, number of distinct cookies, Ratio of various error codes, and seasonal features like day, night, weekend, etc.

XGBoost features include geolocation, hosting, and proxy features, as well as lookback features such as the number of requests per VTC in the last 60 seconds, the number of unique IP addresses per VTC in the last 60 seconds, and the number of unique user-agents per VTC in the last 60 seconds.

3.2 Stage 1: Isolation Forest

Isolation Forest is an unsupervised anomaly detection algorithm that isolates outliers by recursively partitioning data using random splits.

Anomalies require fewer splits to isolate and thus have shorter average path lengths in decision trees. It is fast, scalable, and effective for high-dimensional data.

Isolation Forest was trained on unlabeled traffic to detect statistical outliers. Retained the top 10% of sessions occurring based on the input features with timestamps.

3.3 Stage 2: Supervised Classifier

XGBoost (Extreme Gradient Boosting) is a supervised machine learning algorithm based on gradient-boosted decision trees. It builds models sequentially to correct the

errors of previous ones and is known for its high accuracy, efficiency, and ability to handle imbalanced datasets.

Session metadata corresponding to anomalous timestamps was aggregated and passed to a supervised classifier (XGBoost) trained on verified labels.

The classifier outputs the proportion of sessions predicted as bots within each time window, providing a confidence score.

3.4 System Architecture

4. Experimental Setup And Data

I used a labeled dataset of retail session logs, split into train/test sets. Labels were derived from expertise and known bot behaviors.

Metrics: Precision, Recall, F1-score, and end-to-end accuracy.

5. Results And Analysis

The approach helped identify the bot sessions with an end-to-end accuracy of 93%, precision of 93%, and Recall of 97%.

This approach makes the anomalies predicted by isolation forest effective, as the supervised model provides confidence in the prediction.

Helping to reduce the false positives of anomalies raised by Isolation Forest, leading to a significant reduction in the manual effort required to determine whether an anomaly tag session was a bot or not.

6. Limitations

While the ensemble model improves precision and scalability, it has limitations. First, labeling depends on expert knowledge and analyst review, which may not fully capture evolving bot behaviors.

Second, traffic seasonality (e.g., during flash sales) can introduce false positives, even after normalization. Lastly, the model’s effectiveness may degrade when facing sophisticated, low-volume bots that blend with human traffic.

7. Conclusion

This article introduced a two-stage ensemble method for bot detection in retail web traffic.

By combining Isolation Forest with XGBoost, the system achieved high detection precision while minimizing manual review.

Its interpretability and scalability make it a promising addition to layered defense strategies.

I plan to further explore how the framework adapts to increasingly sophisticated bot behaviors in future deployments.

References

· Alharbi, A., et al. (2021). Hybrid ensemble for anomaly detection in IoT. IEEE Access.

· Cloudflare. (2023). Monitoring ML models for bot detection. Retrieved from https://blog.cloudflare.com

· Kumar, R., et al. (2021). Isolation Forest for intrusion detection. Computers & Security.

· Liu, F. T., Ting, K. M., & Zhou, Z. H. (2008). Isolation Forest. ICDM.

· Wang, J., et al. (2022). XGBoost for fraud detection in anomaly-filtered traffic. Expert Systems.

· Zhao, Y., et al. (2019). Credit card fraud detection using Isolation Forest. Journal of Financial Crime.