"We felt that we needed to go in a different direction at this time," USSF President Sunil Gulati said.
He said the federation will begin looking for a new coach immediately, with the team looking toward qualifying for the 2015 Women's World Cup.
On Monday, a message on goalkeeper Hope Solo's Twitter account addressed the firing: "It's a somber day for us all today. Tom is simply a classy man in every way, and he will certainly be missed."
Jill Ellis, the USSF's director of development, will serve as interim coach. She went 5-0-2 in that role in 2012. The women's team faces China again on Thursday in San Diego.
"I was completely blindsided," Sermanni told SI.com on Sunday night. "Maybe I'm losing my intuitive and perception skills, but I didn't sense a real unease in the team. But I could be wrong in that regard."
Sermanni said he assumes there was "some degree of dissatisfaction among the players, otherwise we wouldn't have gotten to this point so quickly now." He added that the players did what he asked and he has "no qualms with them in the sense of having poor relations with them."
He was hired in October 2012, but Ellis served as interim coach until he took over that January. Sermanni made his debut in a 4-1 win over his native Scotland on Feb. 10, 2013.
He spent the previous eight years coaching Australia's women's team, leading the Matildas to the quarterfinals of the last two World Cups.
Sermanni helped the U.S. to a 13-0-3 record last year, but the Americans went 1-2-1 at the Algarve Cup, the last major tournament for the U.S. before qualifying for next year's World Cup. The seventh-place finish included a 1-0 defeat to Sweden and former U.S. coach Pia Sundhage, ending a two-year, 43-game unbeaten streak. That was the first loss following a 16-0-4 start under Sermanni.
"To put it in a nutshell, they just felt that the way I was managing the team wasn't working," Sermanni told SI.com. "It could be the U.S. team is a unique team that has certain demands that perhaps my management style or my philosophy didn't quite jell with."
His firing came about six months before North and Central American and Caribbean World Cup qualifying, which takes place from Oct. 16-26 at Cancun and Playa del Carmen, Mexico.
On Sunday, Sermanni experimented with a 4-3-3 alignment (four defenders, three midfielders and three forwards) and talked about the team making the most of its offensive opportunities.
"I thought our movement was good, thought our passing was quite incisive," Sermanni said. "What we should've done better is finish the game off a lot earlier. We really need to be finishing these games off when we're creating chances."
The U.S. outshot China 23-1 and ran its unbeaten streak on home turf to 81 games (71-0-10).
Holiday was back in the lineup after missing the Algarve Cup because of a family commitment. She scored in the 39th minute the U.S. finally solving goalkeeper Zhang Yue after a flurry of shots. Holiday nearly had another goal early in the second half, only to have it deflected near the goal line.
The defense allowed only one scoring chance to China, which played defensively even after falling behind.
"The most important thing is coming together now and figuring out what works and what doesn't," forward Sydney Leroux said. "We're getting better every single day."
Any further improvement will come under a new coach.