The emerging technologies leveraging Data Center Networks (DCN) and their consequent traffic patterns impose more necessity for improving Quality of Service (QoS). In this paper, we propose Sieve, a new distributed SDN framework that efficiently schedules flows based on the available bandwidth to improve Flow Completion Time (FCT) of mice flows. In addition, we propose a lightweight sampling mechanism to sample a portion of flows. In particular, Sieve schedules the sampled flows, and it reschedules only elephant flows upon threshold hits. Furthermore, our framework allocates a portion of the flows to ECMP, so that the associated overhead can be mitigated in the control plane and ECMP-related packet collisions are fewer as well. Mininet has been used to evaluate the proposed solution, and Sieve provides better FCT up to 50% in comparison to the existing solutions like ECMP and Hedera.