The improved fractional order derivative empirical power function model was put forward to describe the creep property of asphalt mortar. According to the results of creep recovery experiments of asphalt sands at different stress levels and creep experimental results of asphalt mastics at different temperatures, the parameters of the improved model were determined by simulating the experimental results. The physical meaning and change rule of the parameters of the improved model were analyzed and the evaluation index of high temperature performance of asphalt mortar was proposed. The results show that the improved model can well describe the creep characteristics of asphalt mortar and the effect of temperature on the creep deformation. The improved model can be widely used to study the creep property of asphalt mortar.