This paper explores the significance of Quality of Transmission (QoT) estimation in optical networks and highlights the increasing use of machine learning (ML) techniques to enhance QoT estimation accuracy. It presents a survey of literature in this area, categorizing studies into classification and regression algorithms. ML methods are shown to improve QoT estimation, mitigate nonlinearities, and optimize decisionmaking processes. Ultimately, these advancements reduce the reliance on conservative margins, maximize network capacity, and decrease infrastructure investment. Accurate and real-time QoT information is the foundation for efficient routing and spectral allocation (RSA) systems, it enables proactive failure management, facilitates network reconfiguration, provides inputs for optical line optimization and drives optical network automation.  This paper explores and provides insights into both traditional ML models techniques and large language models (LLMs) as applied to optical communications.

Conventional optical networks are limited by static operational methods, hindering scalability and effectiveness. As networks operate with reduced margins to maximize resource utilization, the risk of hard failures increases, necessitating efficient failure prediction systems and accurate quality of transmission (QoT) estimation. Effective management requires the detection of soft failures, accurate bit error rate (BER) predictions, and dynamic network operations to maintain minimal margins. Machine learning (ML) offers promising solutions for automating these tasks, significantly enhancing failure management and network reliability. This article provides an extensive overview of ML techniques applied to optical networks, specifically focusing on failure management. Key ML techniques discussed include network kriging (NK) for performance estimation and failure localization, support vector machine (SVM) for classification tasks, convolutional neural networks (CNNs) for signal analysis and soft failure identification, and generative adversarial networks (GANs) for synthetic data generation and soft failure detection. It also explores the application of artificial neural networks (ANNs), autoencoders (AEs), Gaussian process (GP), long shortterm memory (LSTM), and gated recurrent units (GRUs) in optical networks. The paper surveys ML techniques for earlywarning and failure prediction, failure detection, identification, localization, magnitude estimation, and soft failure detection and prediction. Emphasizing automations, it discusses how ML algorithms can streamline failure management processes, reducing manual intervention and service disruptions. The potential of large language models (LLMs) and digital twins (DTs) for further advancements in automating failure management, optimizing performance, and network optimization in optical networks is also examined. LLMs significantly advance network management by improving network design, diagnosis, security, and autonomous optimization through the integration comprehensive domain resources and intelligent agents. These advancements are paving the way towards achieving artificial general intelligence and fully automated optical network management.

Transformer-based Large Language Models (LLMs) are progressively entering specialized areas, including the management of optical networks. These models have the ability to automate critical tasks, optimize network performance, and interact with operators, transforming how decisions are made in real-time network environments. This article explores the transformative role of LLMs in optical networks, addressing their application in network design, failure prediction, reconfiguration, and quality of transmission (QoT) estimation. Furthermore, the integration of LLMs in operator-driven decision-making processes is examined, highlighting how these models can provide intelligent recommendations that enhance operational efficiency. Key challenges such as real-time processing, hallucination risks, and domain-specific model adaptation are also discussed, alongside the deployment strategies for LLMs in cloud, edge, and hybrid network architectures. The article concludes by exploring future directions in LLM-assisted optical network management, with insights on human-in-the-loop systems and continuous learning.

Large Language Models (LLMs), built on the transformer architecture, have gained widespread recognition for their capacity to handle complex tasks in natural language processing. However, their potential extends far beyond this domain, offering transformative solutions for optical network management. Optical networks are highly specialized and complex systems characterized by real-time performance requirements, multivendor equipment interoperability, intricate signal processing, and the need to manage diverse transmission impairments such as chromatic dispersion, polarization mode dispersion, and nonlinear effects. These challenges demand advanced automation and optimization techniques, which LLMs are well-suited to address. The integration of LLMs into optical networks provides a scalable approach to automating tasks like network configuration, fault diagnosis, and routing and spectral assignment (RSA). By leveraging LLMs, network operators can enhance quality of transmission (QoT) estimation, optimize amplifier gain control, and reduce operational costs. Additionally, LLMs offer user-friendly interfaces and the ability to insert human oversight through Human-in-the-Loop (HITL) systems, ensuring critical decisions are monitored and managed in real-time. Despite the promise of LLMs, challenges remain. LLMs can exhibit hallucination issues, producing semantically incorrect or fabricated outputs, especially in tasks involving numerical computation, comparison, and logical reasoning. Addressing these challenges requires strategies such as prompt engineering, retrieval-augmented generation (RAG), and fine-tuning with domain-specific data to improve accuracy and reduce errors. This paper explores the application of LLMs in optical networks, focusing on their advantages over traditional machine learning models and the unique challenges posed by the specialized nature of optical networks. The results demonstrate that with the proper adaptations, LLMs can offer significant advancements in automating and optimizing optical network performance.

Disaggregated networks allow operators to select components from different vendors, promoting vendor neutrality. This flexibility enables the selection of best-of-breed solutions for specific network elements. By decoupling hardware and software, disaggregated networks can potentially reduce costs. Operators can choose cost-effective devices and upgrade or replace them independently. Disaggregation also facilitates the adoption of new technologies and innovations and often adheres to open standards promoting interoperability between different equipment vendors. The Yet Another Next Generation (YANG) data modeling has been identified as the preferred language to interface the management and control system. The Network Configuration Protocol (NETCONF) is gaining prominence as a Software-Defined Networking (SDN) protocol standardized by the Internet Task Force (IETF). This paper provides an overview based on a survey of the best practices employed in designing, planning, and operating a disaggregated optical network. It presents the general system architecture including the open software tools SDN controller (based on the Open Operating Network System (ONOS)), optical line system controller (OLC), the QoT estimator based on the Gaussian Noise Simulation in Python (GNPy), and the orchestrator module.

Innovations in optical networks created new technological challenges as routing and spectrum allocation (RSA) problem, fragmented spectrum, the need for rapid and efficient channel restoration, and put pressure on the operation and maintenance. With a lot of lots of variables or knobs to adjust, it is crucial to improve the automation as traditional algorithms are not able to handle the network efficiently. So, this requires sensors, network abstraction, actuators and SDN (Software Defined Networking) in order to run algorithms on top, control, manage the network and make decisions. In addition to this, there are the requirements for low-margin systems and probabilistic shaping. So, machine learning (ML) provides a collection of techniques to adapt to this dynamic and flexible environment and provide a network that learn from experience, optimize and make the networks agile, robust, dynamic, and smarter. At the same time, network automation together with machine learning may cause an explosion in power consumption, making this solution costly, inefficient and not sustainable.