The ash specific surface area, specific adhesion work of asphalt mortar and the rutting factor at high temperature in mineral powders and Trinidad Lake asphalt(TLA) were tested. Thus the impact of the ash content and earth asphalt in TLA modified asphalt on the rutting factor at high temperature of TLA modified asphalt was studied and the correction coefficient for the high temperature rutting factor of TLA modified asphalt was obtained. Furthermore, it is found that the base asphalt is modified 15.08% by the adhesion between the ash in TLA and the base asphalt and 84.92% by TLA itself. From the quantitative analysis of the modification of the base asphalt by ash content and earth asphalt in TLA, impact of ash content and earth asphalt of different amounts at different temperature on the rutting factor of asphalt mortar was clarified. Finally, it is concluded that, in order to eliminate the impact of the ash content as the asphalt filler, the high temperature rutting factor of TLA modified asphalt should be divided by a coefficient of 0.9253.