Ultimately, the shoppers themselves could determine what happens next. Zimmer, 64, who founded the company in 1973, has been one of advertising's most recognizable pitchmen, immediately recognizable for his slogan: "You're going to like the way you look. I guarantee it."
"I am kind of sad," said Gerard McLean, 51 a Web developer from Dayton, Ohio. "George Zimmer is a trusted face and voice. Men need to hear that their fashion choices are OK."
McLean said he'll continue to shop there. But he fears the stores' no-nonsense fashion choices may fade and drive him elsewhere. "My concern is that [Men's Wearhouse] will start listening to advice dispensed by the GQ crowd."
Since Men's Wearhouse's terse announcement Wednesday of Zimmer's firing as executive chairman, it had remained tight-lipped about the reasons.
But on Tuesday, Men's Wearhouse said Zimmer, who owns just 3 ½ percent of the company's stock, pushed for "significant changes that would enable him to regain control." The chain said Zimmer had refused to support its CEO Doug Ewert and other senior managers unless they gave in to his demands.
The retailer also said Zimmer expected veto power over certain corporate decisions, such as executive compensation, even though it has an independent board committee that sets such policies.
Analysts had speculated that the rift was caused by a power struggle. Last week, Zimmer said in a written statement that over the past several months he and the board have disagreed about the direction of Men's Wearhouse. At the time, Zimmer said that the board chose to silence his concerns. On Monday, Zimmer sent a letter to the company saying he was quitting the board.
In addition, Men's Wearhouse said Zimmer also began arguing for a sale of the business to an investment group, reversing course on his previous position.
Men's Wearhouse said its board unanimously agrees that now is not the time to sell the company. The chain said a deal to go private could load the company down with debt.
Zimmer declined to comment for this story.
Investors may have seized on the hint of a potential buyout in Tuesday's revelations and sent the company's stock up more than 5 percent. The surge came after four straight days of declines. Analysts also said investors may have felt some closure from the company's clarification of the reasons for the firing.
However, Richard Jaffe, a Stifel Nicolaus analyst, said that there's not much Zimmer, with such a tiny stake in the company, can do because mounting a leveraged buyout would be expensive. The stock price is hovering near its post-recession high point, and investors would want a premium to that, as much as $50 a share, he said.
Branding experts said the company's moves threaten to alienate its fans, who could vote with their dollars. It also could create poor morale among workers.
David Johnson, CEO of Strategic Visions LLC, a branding company, said that the board should have come clean with the story last week instead of coming out with it days later. It smacks of bad taste, he said.
"[George Zimmer] has won the PR war already. They look cold and heartless," he added.
Robert Passikoff, president of Brand Keys Inc., a New York customer research firm, agrees.
"Airing boardroom dirty laundry doesn't work for consumers," said Passikoff. "It matters internally, not externally. The reason people are reacting emotionally is because they bonded to the brand via the man."
Founders who have wanted to gain back some control have found mixed success. Last year, Best Buy's co-founder and former Chairman Richard Schulze considered making a buyout bid for the electronics retailer but never made a formal offer. In March, he returned to the Best Buy fold as chairman emeritus.
A key difference from Zimmer's situation: Schulze owned 20 percent of Best Buy. But his quest to regain control also didn't register with shoppers because he wasn't the public face of the brand.
On Tuesday, Men's Wearhouse's Facebook page was covered with hundreds of comments from shoppers who lashed out at the company for ousting the founder. Many said they would stop shopping there.
Men's Wearhouse said Tuesday that it didn't want a total breakdown of its relationship with Zimmer and that it wasn't trying to hurt him. The company said it made "considerable efforts" to find a solution that kept Zimmer involved in the business, but that he wouldn't accept anything other than full control.
Men's Wearhouse also said Zimmer, who had initially supported looking at strategic options for the company's K&G stores, did an about-face and began objecting to the review process.
The retailer announced a strategic review of K&G three months ago. The division accounts for about 15 percent of the retailer's total revenue. It operates stores in largely urban markets that cater to low-income shoppers, who have faced more pressures recently due to the tough economy. K&G's business has declined.
Under Zimmer's stewardship, Men's Wearhouse Inc. grew from one small Texas store using a cigar box as a cash register to one of North America's largest men's clothing sellers with 1,143 locations.
Like many clothing retailers, Men's Wearhouse saw its sales and profits battered during the Great Recession, but over the past two years, business has been recovering. For the latest year ended Feb. 2, revenue rose more than 4 percent to $2.48 billion. Net income rose over 9 percent to $131.7 million.
Shares of Men's Wearhouse climbed $2, or 5.7 percent, to close at $37.13. The stock has traded between $25.97 and $38.59 over the past year.
Men's Wearhouse should be hoping for more shoppers like Johnathon Fitzpatrick, who doesn't plan to boycott the chain.
"I have this attachment from growing up seeing him," said Fitzpatrick, 29, from Seattle. "But it's not going to affect my shopping. Price will and service."
Retailer hanging in there
Like many clothiers, Men's Wearhouse saw its sales and profits battered during the Great Recession, but over the past two years, business has recovered. For the year ended Feb. 2, revenue rose 4 percent, to $2.48 billion. Net income rose 9 percent.