It is not sitting right, try depressing the trigger before you put in the bumper. I've had the same issue when I try to put back together the OG Xbox One controller
open