LAUSR

Abstract. The geometrically shaped 32-ary quadrature amplitude modulation (GS-32QAM) signal enabled by swarm intelligence algorithm is proposed in high-speed and long-haul coherent optical communication system. The geometric shaping (GS) is obtained by constructing a generic constellation optimization scheme applying particle swarm algorithm (PSO), wolf pack algorithm (WPA), and marine predator algorithm (MPA), respectively. The complexity of three algorithms applied to GS-32QAM is systematically analyzed, where MPA can obtain better search results with slightly higher complexity than PSO. The results show that the optimized constellation is significantly better than the uniform signal in terms of reducing bit error rate and increasing generalized mutual information (GMI). The maximum GMI gain of GS signal with PSO and WPA are both 0.25  bit  /  sym, and the maximum GMI gain of GS signal with MPA is up to 0.3  bit  /  sym. At the hard-decision forward error correlation limitation of 3.8  ×  10  −  3, compared with uniform signal, the GS signal with PSO, WPA, and MPA can provide the optical signal-to-noise ratio gain of 1, 1.1, and 1.6 dB tolerance, respectively. In terms of data rate, compared with uniform signal, three GS signals can increase by 33, 36, and 43  Gbit  /  s, respectively.