loading page

Bidirectional Data Pipelines: An Industrial Case Study
  • +1
  • Aiswarya Raj Munappy,
  • Anas Dakkak,
  • Jan Bosch,
  • Helena Olsson
Aiswarya Raj Munappy
Chalmers tekniska hogskola AB

Corresponding Author:aiswarya@chalmers.se

Author Profile
Anas Dakkak
Telefonaktiebolaget LM Ericsson
Author Profile
Jan Bosch
Chalmers tekniska hogskola AB
Author Profile
Helena Olsson
Malmo universitet Institutionen for datavetenskap och medieteknik
Author Profile

Abstract

Background: Bidirectional data pipelines have emerged as a response to the evolving needs of modern data ecosystems. Traditionally, unidirectional pipelines allowed data to flow in a single direction, limiting interaction. The surge in demand for real-time bidirectional communication prompted the development of pipelines that enable two-way data flow, facilitating seamless and dynamic exchanges between source and destination. Objective: The research aims to delve into the role of bidirectional data pipelines within the companies producing and selling software-intensive embedded systems products. Further, the study endeavors to elucidate the fundamental differences between unidirectional and bidirectional data pipelines, shedding light on their unique characteristics. Through comprehensive exploration, it seeks to discern the benefits and challenges inherent in implementing and maintaining bidirectional data pipelines. Furthermore, a critical aspect of the research involves outlining the intricate steps and considerations essential for migrating from unidirectional to bidirectional data pipelines. This includes a focus on prerequisites, methodologies, and the potential benefits derived from such a transition. Method: This study employs a qualitative research approach centered around a multiple-interpretive case study to delve into the complexities of bidirectional data pipelines. Five distinct use cases have been meticulously selected to provide a comprehensive understanding of various aspects of bidirectional data pipelines. Through the in-depth analysis of these concrete use cases, this research aims to elucidate the intricacies, benefits, and challenges associated with bidirectional data pipelines in software-intensive embedded systems environment. Results: The study yielded insightful results on various aspects of bidirectional data pipelines, emphasizing their distinctions from unidirectional data pipelines without a shared data transmission channel. It uncovered the compelling need for bidirectional data pipelines in modern data-centric environments, where the dynamic exchange of information between source and destination is pivotal. The identified benefits ranged from enhanced real-time data synchronization to improved responsiveness in addressing evolving business requirements. Concurrently, the study elucidated inherent challenges, such as increased complexity in pipeline management and potential security considerations. Moreover, the research provided a nuanced understanding of the stepwise process involved in transitioning from unidirectional to bidirectional data pipelines. Conclusion: The study underscores the pivotal role of bidirectional data pipelines in meeting evolving data requirements, offering benefits like real-time synchronization and improved decision-making. Despite their importance, there is a lack of comprehensive research on bidirectional pipelines, prompting the need for further exploration. The transitional journey involves addressing challenges such as data consistency and security concerns while optimizing system design for compatibility. For researchers, the study suggests avenues for optimizing bidirectional pipeline performance, examining long-term impacts, scalability, and real-time anomaly detection. Practitioners can leverage insights for informed decision-making in transitioning to bidirectional data pipelines, aligning organizational needs with their benefits and challenges.
24 Feb 2024Submission Checks Completed
24 Feb 2024Assigned to Editor
24 Feb 2024Reviewer(s) Assigned