In traditional approaches to time series data augmentation, the focus has largely been on refining the architecture of Generative Adversarial Networks (GANs) to better align with the original data distribution while attempting to preserve the dynamic behavior inherent in the data. However, GANs inherently struggle to accurately retain the temporal dynamics of the original time series, primarily due to the absence of first-order difference information. To address this gap, we propose a novel framework, titled the GAN-MCMC approach, for generating multivariate time series data that integrates two key modules: (a) a GAN-based module for generating multivariate time series, and (b) an MCMC-based module designed to preserve the first-order difference distribution. This integrated approach ensures that the synthetic data not only replicates the original data distribution but also retains its dynamic properties.In this study, the multivariate time series data used were collected from Area X.O, which was employed to predict N2O emissions from farming. This dataset is ideal for our analysis because it originates from a complex dynamic system, and the equipment used to gather the data is prohibitively expensive to deploy on a wide scale. Therefore, data augmentation techniques to generate synthetic agricultural data are both necessary and valuable for improving the predictive models. A central aspect of the GAN-MCMC approach is adjusting the β factor in the modified Metropolis-Hastings algorithm, which is the core algorithm in the MCMC module. The β factor controls the extent to which information from the original time series is preserved. Our experiments demonstrate that small values of β effectively retain periodic information, and the joint distribution of the firstorder differences in the synthetic data remains consistent when the same β is used in the algorithm. Additionally, the memoryless property of the Markov Chain is preserved in the generated data, and we employ an exponential moving average (EMA) technique to simulate the long-term relationships present in the original time series.Finally, we use the synthetic time series data to train Long Short-Term Memory networks (LSTMs). Our results show that LSTMs trained on synthetic data generated by the GAN-MCMC framework outperform those trained on synthetic data produced by other GANs. The link to source code:  https://github.com/Developer2046/GAN-MCMC  

This study introduces the Subspace Rotation Algorithm (SRA), an innovative gradientfree method designed to discover the global optimal weight matrix. The SRA consists of two fundamental algorithms: the Left Subspace Rotation Algorithm (LSRA) and the Right Subspace Rotation Algorithm (RSRA). The combination of LSRA and RSRA, in two formats, LSRA-RSRA and RSRA-LSRA, can harness the advantages of both individual algorithms, resulting in enhanced performance. Our observations reveal that shallow and wide Multilayer Perceptrons (MLP), trained using RSRA-LSRA, achieve higher training accuracy compared to the Backpropagation (BP) algorithm. Moreover, when combining SRA with the BP algorithm, a remarkable impact on training MLP models is observed. Our experiments demonstrate that the BP algorithm may become trapped in local optima, while RSRA-LSRA, though capable of escaping local optima, may not fully realize its potential when the number of hidden nodes is limited. The synergy of RSRA-LSRA and the BP algorithm allows for the utilization of the advantages from both approaches, achieving optimal MLP performance with fewer hidden nodes.

This paper introduces Conjugate Neural Networks (CoNNs), a novel architecture designed to enhance robustness against adversarial attacks. Adversarial attacks pose significant security challenges to neural networks by manipulating input data to cause incorrect model predictions. The proposed CoNNs architecture leverages a dual-model framework where two complementary neural networks are trained in tandem, each using adversarial samples generated by the other, to cancel out adversarial effects and improve overall resilience. We evaluate CoNNs using various deep learning models, including MLP, ResNet, LSTM, and Agricultural-Informed Neural Networks (AINN), across multiple datasets such as Fashion-MNIST, CIFAR10, the data generated by Lorenz system, and agricultural N 2 O emissions data. The results demonstrate that CoNNs significantly outperform traditional single neural network in resisting both Fast Gradient Sign Method (FGSM) and Basic Iterative Method (BIM) attacks. CoNNs maintain higher accuracy and exhibit greater stability under adversarial conditions, making them a promising approach for improving the robustness of neural network-based systems.

Both the Restricted Hopfield Network (RHN) and Dense Associative Memory (DAM) are derived from the classical Hopfield Neural Network (HNN), but they improve upon HNN using different mechanisms. This paper presents a comparative analysis of RHN and DAM with respect to storage capacity, time complexity, training efficiency, and retrieval from incomplete or noisy patterns. The analysis reveals that RHN and DAM use different mechanisms for storing patterns. RHN, trained using the Subspace Rotation Algorithm (SRA), exhibits better time complexity than DAM, which is trained using the Energy-based Algorithm (EBA). Furthermore, in practice, RHN requires significantly less time than DAM to memorize the same number of patterns, as DAM needs more epochs to converge. The experimental results demonstrate that when storing 1,000 character patterns (825 bits), RHN with two hidden layers outperforms DAM in terms of retrieval from noisy and incomplete patterns. When memorizing 10, 50, and 100 patterns from both the MNIST and Fashion MNIST datasets, RHN shows a slight advantage over DAM in retrieval from incomplete and noisy patterns. However, the performance of both models degrades as the number of hidden nodes or stored patterns increases.  The link to the source code: https://github.com/Developer2046/rhn_dam.

This paper introduces the QLSTM-DLEM model, a quantum model combining Quantum Long Short Term Memory (QLSTM) and Dynamic Land Ecosystem Model (DLEM), for predicting nitrous oxide (N2O) emissions in agricultural fields. Given the time series data collected from different bifurcation branches, integrating all the data into a single model becomes challenging. To address this, time series data, accompanied by signatures, is utilized to distinguish data from different branches. The auto-correlation function (ACF), partial auto-correlation function (PACF), data distribution, and joint distribution of the first-order difference illustrate that time series data collected under different initial conditions and parameters exhibit slight differences but share several similar characteristics. Performance evaluation metrics, including mean absolute error (MAE), mean absolute percentage error (MAPE), root mean square error (RMSE), and coefficient of determination (R 2), are employed to assess the model. The results indicate that the QLSTM-DLEM model outperforms the classical LSTM-DLEM model in terms of generalization and stability, possibly attributed to the parallelism and superposition of quantum computing. Overall, our simulation results demonstrate that the proposed QLSTM-DLEM hybrid model can effectively predict N2O emissions in agricultural fields, with potential applications in environmental monitoring and management.

This paper introduces the Subspace Rotation Algorithm (SRA) for training the Restricted Hopfield Network (RHN) as an auto-associative memory. SRA is a gradient-free subspace tracking method based on Singular Value Decomposition (SVD) to update the weight matrix. Despite having slightly worse time complexity than Back-propagation (BP) theoretically, in practice, SRA completes training faster since it requires fewer iterations to converge. Comparative analysis with BP for training RHN reveals that SRA consistently reaches the optimal solution, whereas BP fails to achieve comparable performance if the weight initialization is not within the appropriate basin of attraction. Experiments involving the memorization of 10, 50, and 100 patterns from the MNIST dataset show that RHN trained with SRA exhibits better robustness to noisy and corrupted patterns compared to RHN trained with BP. These findings suggest that SRA offers a more reliable and effective method for training RHNs in applications needing high tolerance to input distortions.