In today’s world, where data-driven decision-making has become a core enterprise capability, the “watershed” of competitiveness increasingly lies in the data foundation: whether an organization can continuously provide real-time data capabilities that are highly reliable, low-latency, traceable, and governable in production environments characterized by high concurrency, strong volatility, and heavy dependencies. This directly impacts advertising efficiency, the speed of risk identification, and the pace of business iteration. As a result, real-time data systems are no longer merely about “getting the pipelines to run,” but about long-term operational engineering-systems must run fast, and also run stably, sustainably, and transparently.
Liu Chao, currently a Senior Big Data Engineer at SHEIN, has long focused on building integrated real-time and batch data platforms. He has continuously contributed to and led key data architecture design and implementation efforts, covering the full chain from event ingestion and real-time processing to data modeling, metric system construction, and service-oriented data delivery. Unlike many engineering paths that focus mainly on “point optimizations,” his strengths lie in breaking complex systems down into controllable modules and, with a platform-governance mindset, embedding quality, stability, observability, and recovery capabilities directly into the architecture-precisely the parts most likely to spiral out of control at scale, yet most decisive for success. Recently, he has systematized frontline production experience into academic outputs published in Advances in Computer and Autonomous Intelligence Research and Engineering Research , demonstrating his ability to abstract complex engineering challenges into reusable methodologies and validate them through practice.
In “Research on Building an ODS-Based Real-Time Data Pipeline and Fault-Tolerance Mechanisms,” Liu uses a large-scale streaming media advertising platform as the practical context. From an “end-to-end delivery” perspective, he systematically outlines the critical links of a real-time data pipeline and threads fault-tolerance principles throughout the entire lifecycle of “ingestion-processing-lake ingestion-serving.” He emphasizes that the true challenge of enterprise real-time systems is not whether individual components can run, but whether the system remains controllable, explainable, and repairable as data scale and business dependencies grow rapidly. Based on this, he proposes a dual-output design to isolate erroneous data while maintaining traceability, so that abnormal records neither contaminate the main pipeline nor become unrecoverable and can be replayed and repaired; he introduces multi-level reconciliation mechanisms to define data stability boundaries across different time scales, enabling the business side to form clear expectations regarding “when data is usable, when it becomes stable, and how reconciliation is performed”; and he combines end-to-end monitoring with automated recovery mechanisms to improve data quality, service availability, and eventual consistency. This research effectively moves real-time systems from “usable” to “sustainably operable.”
In the academic paper “Implementing a Real-Time Feature Lake for Large-Scale Risk Control: High-Concurrency Writes and Low-Latency Retrieval,” Liu further focuses on a core bottleneck in risk control systems-feature provisioning. Real-time requirements in financial and e-commerce risk control scenarios are often more stringent: systems must withstand massive event ingestion, support low-latency retrieval for online decisions, and maintain consistency and traceability. Otherwise, deviations can easily arise between model training, evaluation, and online decision-making, leading to severe reconciliation difficulties and sharply increased explanation costs. To address this pain point, Liu proposes and validates a real-time feature lake architecture based on Apache Paimon. By unifying streaming and batch pipelines and leveraging versioned capabilities, the system achieves high-concurrency writes and low-latency retrieval while ensuring consistency. The results show that it can support tens of billions to hundreds of billions of events processed per day, reduce feature update latency to the second level, and significantly improve query performance-providing the industry with a more replicable next-generation risk-control data foundation paradigm, where features evolve from “outputs of scattered script-based processing” into “governable, reusable, and traceable assets.”
Overall, Liu Chao’s practice-to-theory contributions take “reliability and low latency” as the central theme. They further distill production-grade best practices from two real-time-intensive scenarios-advertising and risk control-into transferable and verifiable engineering methodologies: on one hand, emphasizing fault tolerance, reconciliation, and an observability-driven closed loop across the end-to-end pipeline to address stability challenges in long-running real-time systems; on the other hand, enabling feature-lake construction through unified storage and consistent semantics, resolving the key tension in risk control-“data can be written, retrieved quickly, reconciled accurately, and traced back when needed.” For enterprises upgrading data platforms, evolving toward unified streaming-and-batch architectures, or building real-time risk-control systems, such outcomes-rooted in frontline production practice yet abstracted into reusable paradigms-offer a deployment-oriented reference point and a practical path toward high-performance, high-reliability data infrastructure.
Media Contact
Company Name: SHEIN Technology LLC
Contact Person: Chao Liu
Email:Send Email [https://www.abnewswire.com/email_contact_us.php?pr=liu-chao-a-practitioner-of-enterprisegrade-realtime-data-platform-delivery]
City: Irvine
State: California
Country: United States
Website: https://www.sheingroup.com
Legal Disclaimer: Information contained on this page is provided by an independent third-party content provider. ABNewswire makes no warranties or responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you are affiliated with this article or have any complaints or copyright issues related to this article and would like it to be removed, please contact retract@swscontact.com
This release was published on openPR.
