This study proposes a verifiable multi-party quantum key agreement protocol which based on X operation and four-qubit cluster states. The protocol allows two participants to perform X operations on two particles of each four-qubit cluster states using their own key at a time, and to perform delay measurements on the converted cluster state, which ensures that each participant contributes equally to the agreement key. The proposed protocol uses mutually unbiased bases particles as decoy particles and performs unitary operations on these decoy particles using a pair of function values of symmetric binary polynomial, which can not only perform eavesdropping checks, but also achieved the identity authentication between participants. The scheme can be applied to any situation with more than 2 participants. The security analysis shows that the proposed protocol can resist external attacks and participant attacks. Compared with the existing multi-party QKA, the proposed protocol not only has advantages in the use of decoy particles, but also has high quantum bit efficiency.