An off-duty police officer and others at the church subdued Capener and held him down until police arrived.
Some of those who were stabbed were members of the choir.
The choir's pianist, Brenda Baca King, told KRQE-TV that the attacker was looking at the lead soloist. "I just remember seeing him hurdle over the pews, hurdle over people and run towards us and I thought, 'Oh my God, this is not good,'" Baca King said.
Police described the stabbing scene as chaotic as parishioners screamed as the attack unfolded.
Gibbs says Capener was interviewed by police and was expected to face felony charges. It's not yet known whether Capener has an attorney.
Gibbs says investigators don't yet know the motive for the stabbings, whether Capener had ties to the victims or whether he regularly attended the church.
The stabbings occurred as the choir had just begun its closing hymns.
Archbishop of Santa Fe Michael Sheehan released a statement saying he was saddened by the attack.
"I pray for all who have been harmed, their families, the parishioners and that nothing like this will ever happen again," Sheehan said.
The church didn't immediately return calls seeking comment on Sunday afternoon.