The COVID-19 pandemic poses a serious threat to global health, and one of the key epidemiological factors that shape the transmission of COVID-19 is its serial interval (SI). Although SI is commonly considered following a probability distribution at a population scale, slight discrepancies in SI across different transmission generations are observed from the aggregated statistics in recent studies. To explore the change in SI across transmission generations, we develop a likelihood-based statistical inference framework to examine and quantify the change in SI. The COVID-19 contact tracing surveillance data in Hong Kong are used for exemplification. We find that the individual SI of COVID-19 is likely to shrink with a rate of 0.72 per generation and 95%CI: (0.54, 0.96) as the transmission generation increases. We speculate that the shrinkage in SI is an outcome of competition among multiple candidate infectors within a cluster of cases. The shrinkage in SI may speed up the transmission process, and thus the nonpharmaceutical interventive strategies are crucially important to mitigate the COVID-19 epidemic.