Due to the bit correlation of high-order modulation,polarization code can not effectively polarize the channel in optical communication system.In this paper,a polarizat ion interleaver for polarization multiplexing optical communication system is proposed.The proposed interleaver uses a unified encoder to encode the information bits of the system,and then interle aves the coded sequence,which breaks the bit correlation of high-order modulation,reduces th e impact on the channel polarization of the polarization code,and improves the performance of t he communication system.At the same time,the 2D and 4D modulation and coding schemes are compar ed and analyzed,and the scaling factor is added in the decoding symbol metric calculat ion to improve the bit error rate (BER) performance of the system.Simulation results show that compared with the tr aditional bit interleaved coded modulation (BICM), the BER performance of the polarization interleaver is improved by about one ord er of magnitude when the channel is additive white Gaussian noise (AWGN),and the BER performance of the system is improved by a bout 0.5dB when the scaling factor is added.In DP-16QAM coherent optical communication sy stem,the performance of 4D polarization interleaving scheme is improved by about 1.3dB c ompared with the traditional BICM scheme when the BER is 10－4.Moreover,the tolerance of 4D mod ulation scheme to polarization mode dispersion (PMD) is better than that of 2D modulation scheme under differen t PMD.