The fast fractal face image compression algorithm is proposed based on the facial symmetry. In the encoding, domain blocks are restricted in the symmetrical candidate region for the range block. If no match blocks can be found, the candidate region is expanded. We analyze the time complexity of the variance based method and the proposed method. The result shows that the encoding time of the proposed method is half of the variance based method even under the worst circumstance. The experiments on the ORL and YALE face databases show that the average encoding time is 1/3 of the variance based method when using the proposed method. The relationship among the candidate region, threshold, hit range blocks,encoding time and PSNR is discussed by the experiments.