The performance of nonbinary low-density parity-check (LDPC) codes is obviousl y better than that of binary LDPC codes with the same bit length,but nonbinary LDPC codes have relatively high im plementation complexity.In order to reduce the implementation complexity of nonbinary LDPC codes and improv e the coding gain of the system,this paper adopts permutation polynomial method in order to impr ove the general nonbinary quasi-cyclic LDPC (QC-LDPC) codes.And the improved nonbinary QC-LDPC codes a re then applied to coherent optical orthogonal frequency division multiplexing system with polarization division mul tiplexing (PDM-CO-OFDM) to study the transmission performance of PDM-CO-OFDM system in detail. According to the simulation results,we can find that the system based on GF (4) QC-LDPC encoding is obviously better than that employing corresponding binary QC-LDPC codes.And compar ed with the system encoded by general GF(4) QC-LDPC codes,the system encoded by improved GF(4) QC-LDPC code s has error performance improvement over 0.65dB,spectral efficiency improvement of 2.16bit/s/Hz and ha s the improvement obviously in inhibition of the channel dispersion ability and implementation complexity.T herefore,nonbinary QC-LDPC codes constructed by the method based on permutation polynomial have broad application prospects in optical fiber communication syst ems.