The first part of this study showed that the triglyceride composition of purified oils has little impact on sterol/sterol ester oleogels. Hence, changes in the gels’ properties observed in previous studies must arise from minor polar components, particularly by changing the interactions within the fibrillar network. Selected molecules (oleic acid, tocopheryl acetate, monoglycerides, and water) were added to three purified oils to unravel the individual contributions introduced by different functional groups. While all additives retarded the molecular self-assembly of sitosterol with oryzanol, distinct effects were found for gel hardness, transition temperatures and enthalpies, strain sweep responses, and microstructure. It was discovered that the maximum storage modulus in the linear viscoelastic region does not necessarily relate to the gels’ compression firmness. In samples comprising oleic acid and tocopheryl acetate, discrete interaction mechanisms with the scaffolding elements were suggested since results between the two additives developed differently and were dose-dependent. A network supporting effect was suggested at low concentrations, in line with previous results for oils comprising low levels of thermal deterioration products. The microstructure of oleogels was considerably modified with additives. Unfortunately, effects are difficult to quantify due to the packed surface observed in AFM micrographs.