"Americans have a right to be angry about it and I'm angry about it," the president said the day after an inspector general report detailed the abuses. "We are going to hold the responsible parties accountable."
He called removing Miller the "first step."
In his resignation letter, Miller wrote: "There is a strong and immediate need to restore public trust in the nation's tax agency. I believe the service will benefit from having a new acting commissioner in place during this challenging period."
The FBI has launched a criminal probe and at least three congressional committees are beginning their own investigations. One of those panels is the Senate Finance Committee, where Hatch is the top Republican. The Finance Committee will hold its first hearing on the matter on Tuesday.
Clean slate • Hatch said Miller's removal is a "positive and important step" for the tarnished agency.
"A clean slate at the IRS with new leadership is imperative to fix this egregious encroachment on the lives of honest, hard-working Americans whose only sin was that they want to express their beliefs," he said in a statement shortly after Obama's announcement.
Hatch along with Senate GOP leader Mitch McConnell, R-Ky., and Sen. Marco Rubio, R-Fla., had called for Miller's ouster.
The Utah senator believes Miller intentionally deceived him about the IRS's review of politically active 501(c)(4) groups in September 2012 by couching his reply to a letter from Hatch in highly technical terms, while not divulging that he knew the inappropriate screens took place.
"This was either one of the greatest cases of incompetence that I've ever seen or it was the IRS willfully not telling Congress the truth," Hatch said in a Capitol Hill news conference Tuesday. "The American people deserve the truth. There is no single agency of the federal government that instills more fear than the IRS."
Miller took over for former Commissioner Doug Shulman whose five-term ended in November.
Both Shulman and Miller learned last May that IRS employees in Cincinnati had inappropriately stalled the tax-exempt applications of groups with "tea party," "patriot" and "9/12" in their names and sent intrusive questions about their donors and the political activity of board members. Yet neither of them admitted to the activity when pressed by GOP lawmakers such as Hatch.
Hatch sent Obama a letter Tuesday, signed by all 45 Republicans, that asked the president to require all IRS employees involved in the scandal to work with congressional investigators.
"It is imperative," the letter reads, "that the administration be fully forthcoming to ensure that we begin to restore the confidence of our fellow citizens after this blatant violation of their trust."
Cooperation • In his brief statement Wednesday, Obama promised to work with Congress to investigate what happened and offer changes to ensure that the tax agency doesn't target groups based on their political beliefs.
The inappropriate activity didn't come to light until an IRS official admitted to it last Friday. An inspector general report, released late Tuesday, says front-line employees in Cincinnati targeted solely because of their names about 100 tea party groups seeking to organize as social welfare organizations. IRS employees also sent inappropriate questionnaires to tea party groups and other conservative organizations throughout an 18-month period. The report places much of the blame on the agency's top officials for not setting a politically neutral standard or taking swifter corrective action once informed of the political targeting.
Sen. Mike Lee, R-Utah, said the IRS actions are indicative of a government that is overstepping its bounds, and he warned against focusing narrowly on who was responsible when wider changes are needed.
"While it is important we get answers about how and why these groups were targeted by the IRS, firing a few employees will not solve the problem," he said. "The long-term solution is for the American people to demand that government be less involved, less intrusive, and therefore less able to carry out these kinds of abuses."
Lee linked the IRS scandal to the Justice Department seizing the phone records of Associated Press reporters and the Health Department requesting that companies donate to help implement the Affordable Care Act, saying that when such actions happen "we are no longer citizens but subjects."
So far, only one Utah group claims to have been inappropriately singled out by the IRS.
Freedom Path, created in January 2011 to support Hatch's re-election efforts, spent $570,000 backing Hatch, while keeping its donors secret. It is still waiting for an IRS ruling on its application, but claims to have received intrusive questionnaires. The IRS also inappropriately gave a copy of Freedom Path's application to ProPublica. Hatch's office said the senator was unaware of Freedom Path's interaction with the IRS until informed by The Salt Lake Tribune.