Fixed structure controllers (such as proportional-integral-derivative controllers) are used extensively in industry. Finding a practical and versatile method to tune these controllers, particularly with imprecise process models and limited online computational resources, is an industrially relevant problem which could improve the efficiency of many plants. In this paper, we present two flexible neural network-based approaches capable of tuning any fixed structure controller for any control objective and process model and compare their advantages and disadvantages. The first approach is derived from supervised learning and classical optimization techniques, while the second approach applies techniques used in deep reinforcement learning. Both approaches incorporate model uncertainties when selecting controller parameters, reducing the need for costly experiments to precisely estimate model parameters in a plant. Both methods are also computationally efficient online, enabling their widespread usage.