## (a) The fidelity susceptibility for the ground state as a function of the atomic transition strength *J* for different particle numbers *N*

**Figure 5.** (a) The fidelity susceptibility for the ground state as a function of the atomic transition strength *J* for different particle numbers *N*. (b) The finite-size scaling behaviours of \Delta J^\prime =J^\prime _N-J_\mathrm{c} with δ_{F} = 2/3 and (c) χ_{max} with γ = 1.305 for different atomic interaction \widetilde{U}, where the discrete points denote the numerical results for finite particle numbers *N* = [130, 270] and the solid lines are the fitting functions. (d) The finite-size scaling analysis of the fidelity susceptibility χ with \widetilde{U}=2g for different system sizes *N* = 130, 150, 170, 190, 210, where ν = 2/3, J^{\prime }_N\sim (3-\sqrt{2})/2+2.511 N^{-2/3} and χ_{max} ~ 0.2383*N*^{1.305}. The parameter is δ_{b} = −2*g*.

**Abstract**

The quantum phase transition in an atom–molecule conversion system with atomic transition between two hyperfine states is studied. In the mean-field approximation, we give the phase diagram in which the phase boundary only depends on the atomic transition strength and the atom–molecule energy detuning, but not on the atomic interactions. Such a phase boundary is further confirmed by calculating the fidelity of the ground state and the energy gap between the first excited state and the ground state. As a comparison to the mean-field results, we also study the quantum phase transition with the full quantum method where the phase boundary depends on the particle number. Analysing the finite-size scaling behaviours of the energy gap, the fidelity susceptibility and the first-order derivative of entanglement entropy with respect to the atomic transition strength, we obtain their critical exponents by numerical calculation and show that in the thermodynamic limit, one can obtain the same phase boundary as in the mean-field approximation. Our results show a new way to manipulate the quantum phase transition by regulating the atomic transition strength with the intensity of the laser.