A document by Ts. Lee Chong Keat. Click on the document to view its contents.

The aim of this paper is to design the data pipeline for multinational e-commerce unified analytics platform by the Lambda data architecture then leveraging the Amazon Web Services (AWS) Cloud platform and services to implement the entire data pipeline. To support a large volume of transactions analysis, the data pipeline needs to consider load balancing and auto-scaling servers to ensure high availability and performance under varying workloads. In result, the data pipeline able to ingestion the data from various platforms and database then enabling advanced analytics and real-time actionable insight for example personalized recommendation, customer behavior analysis clickstream analysis, inventory monitoring, and real-time fraud detection.

This paper explores three distributed architecture-based big data storage Google Cloud Bigtable, Snowflake Storage, and Apache Cassandra. It focuses on analyzing the advantages and challenges of each solution across three sectors: Financial, Industrial Services, and Transportation, providing real-world applications. Through an indepth analysis of recent developments, the study compares these three solutions across the sectors. The results show that Bigtable is more suitable for the financial sector due to its high availability and scalability. Cassandra offers cost-efficiency in productivity for Industrial Services. For Transportation Services, Snowflake provides a data warehouse optimized for large-scale analytics, enabling real-time comprehensive analysis of transportation-related information to support informed decision-making.