With the proliferation of edge computing, efficient AI inference on edge devices has become essential for intelligent applications such as autonomous vehicles and VR/AR. In this context, we address the problem of efficient remote object recognition by optimizing feature transmission between mobile devices and edge servers. We propose an optimization framework to address the challenge of dynamic channel conditions and device mobility in end-to-end communication systems. Our approach builds upon existing methods by leveraging a semantic knowledge base to drive multi-level feature transmission, accounting for temporal factors, state transition, and dynamic elements throughout the transmission process. Additionally, we enhance the multi-level feature transmission policy by introducing an additional 5th-level edge-assisted semantic communication, maximizing recognition performance by leveraging a large semantic knowledge base on the edge server. Formulated as an online optimization problem, it aims to simultaneously minimize semantic loss and adhere to specified transmission latency thresholds. To address this, we design a soft actor-critic-based deep reinforcement learning system with a carefully designed reward for real-time decision-making, overcoming the optimization difficulty of the NP-hard problem and achieving the optimization objectives. Numerical results showcase the superiority of our approach compared to traditional greedy methods under various system setups using open-source datasets.