Two different asphalt mortars were studied with common evaluation methods and rheological testing approach. Analysis shows that, the changes of intensity and viscoelasticity depend on the composition differences in asphalt mortars, different stress response of common asphalt and modified asphalt is due to different viscoelasticity composition and structure characteristics. The mortar system with high filler bitumen ratio is easy to be destroyed at high temperature and high shear frequency conditions. And in the cognition of performance mechanism of mortar, its structure and dissipation properties should be considered, especially the micro physical meaning of phase angle. The maximum filler bitumen ratio should be determined by non recoverable compliance difference by multiple stress creep recovery test.