LAUSR

In this paper, we study the Helmholtz transmission eigenvalue problem for inhomogeneous anisotropic media with the index of refraction n(x)≡1 in two and three dimensions. Starting with the nonlinear fourth-order formulation established by Cakoni et al 2009 J. Integral Equ. Appl. 21 203–27, we present an equivalent mixed formulation for this problem with auxiliary variables, followed by finite element discretization. Using the proposed scheme, we rigorously show that the optimal convergence rate for the transmission eigenvalues on both convex and nonconvex domains can be expected. With this scheme, we obtain a sparse generalized eigenvalue problem whose size is too demanding, even with a coarse mesh that its smallest few real eigenvalues fail to be solved by the shift and invert method. We partially overcome this critical issue by deflating nearly all of the ∞ eigenvalues with huge multiplicity, resulting in a marked reduction in the matrix size without deteriorating the sparsity. Extensive numerical examples are reported to demonstrate the effectiveness and efficiency of the proposed scheme.