Spherical shale haydite was selected as main coarse aggregate, while 30% ordinary sand was replaced by grade I fly ash of the same volume to prepare fly ash lightweight aggregate concrete. 10%40%(by mass) or 5%15% cement of fly ash lightweight aggregate concrete were equivalently replaced by ultra fine fly ash(UFA) or silica fume(SF) respectively, the mechanical property evolution law and change of microstructure of fly ash lightweight aggregate concrete under sulfate attack were investigated in terms of corrosion resistance coefficient, porosity and dynamic elastic modulus of concrete, in which compressive strength test, drying wetting cycle test, industrial computed tomography test, mercury injection test and dynamic elastic modulus test were employed. As the research result shows, after a period of drying wetting cycle of 20 times with the Na2SO4 solution concentration(by mass) ranging from 0% to 10%, the compressive strength reaches the maximum value and then decreases fast; the corrosion resistance coefficients of fly ash lightweight aggregate concrete reach the maximum value when the replacement rate of UFA and SF are 20% and 10% respectively with drying wetting cycles of 50 times in 10% Na2SO4 solution. Meanwhile, the volume fractions of pores with size between 50nm and 100nm in fly ash lightweight aggregate concrete with UFA or SF added increase significantly. The volume fractions of pores with size less than 100nm in fly ash lightweight aggregate concrete with 20%UFA or 10%SF added are 139 times and 146 times that of ordinary concrete. Furthermore, the interfacial transition zone and the bond property between lightweight aggregate and mortar is improved, so that the further erosion by sulfate to the structure of fly ash lightweight aggregate concrete is delayed.