Because of low iterative efficiency and dropping into local optimum ea sily of the standard particle swarm optimization (PSO) algorithm the varied local particle swarm optimization (VLPSO) is proposed based on variation mechanism.In 100Gbit/s optical transmission system,the second-order polarization mode dispersion (PMD) return-to-zero (RZ) coding adaptive compensation experiment is performed based on VLPSO algorithm for logic control.The results show that after searching compensation,the openin g of eye diagrams is enhanced obviously and the average searching compensation t ime is 1.650ms.The VLPSO algorith m can follow and compensate the small change of PMD,whose tracking comp ensation time is less than 0.431ms.