"The Street was looking for him to say some magic buzzwords about avoiding the 'fiscal cliff,' about cooperation," Sal Arnuk of Themis trading, a New Jersey brokerage. Instead, Arnuk said, the remarks were "unyielding."
The "cliff" is a package of tax increases and government spending cuts that will take effect Jan. 1 unless Obama and Congress reach a deal first. They would total about $700 billion for 2013 and could send the country back into recession.
Stocks have fallen steadily since voters returned Obama and a divided Congress to power Nov. 6. The Dow has fallen 675 points, including three single-day drops of more than 100 points.
The Standard & Poor's 500 index has dropped 5 percent in that time, returning to where it stood in late July.
"Investors' hopes that the election would end uncertainty remain unfulfilled," said Lawrence Creatura, a portfolio manager at Federated Investors in Rochester, N.Y. "It's very tough to determine what happens next."
Obama said Wednesday that he would not to cave to Republicans who have pressed for tax cuts first passed by President George W. Bush to be extended for all income earners.
Obama has long opposed extending the cuts for families making more than $250,000 a year, but he gave in to GOP demands in 2010, when the cuts were up for renewal.
The president also met with business leaders Wednesday to discuss the economy.
Obama is to meet Friday with Republican House Speaker John Boehner and the top Republican in the Senate, Mitch McConnell. They are expected to designate aides to begin the search for a compromise on the budget.
Boehner and McConnell have said they won't agree to raise tax rates for the wealthy. There are ways to increase tax revenue from the wealthy without rates, including limiting tax deductions, but the path to compromise is unclear.
On Wednesday, the Dow dropped 185.23 points to 12,570.95. The S&P slipped 19.04 points, or 1.4 percent, to 1,355.49, and the Nasdaq composite was down 37.08 points, or 1.29 percent, at 2,846.81.
Stocks are still up on the year, though they have pared gains since peaking in September. The S&P has fallen 7.5 percent since its Sept. 14 peak, and the Dow has fallen 7.6 percent, more than 1,000 points, since its Oct. 5 peak.
Among individual stocks, Abercrombie & Fitch, which sells teen clothes, bucked the trend of a declining market and was by far the best-performing stock in the S&P 500.
Abercrombie jumped $10.54 to $41.92 after reporting that its international business was thriving and that its net income soared 40 percent in the most recent quarter, more than financial analysts were expecting.
The strong results from Abercrombie were tempered by a report from the Commerce Department saying that Americans cut back on spending in October, suggesting that many are still cautious about the economy.
Sales dropped 0.3 percent last month after three months of gains. That was worse than analysts had been expecting, according to FactSet, a provider of financial data.
The government also said auto sales fell 1.5 percent, the most in more than a year. Sales may have been hurt by Superstorm Sandy.
The Federal Reserve released minutes of its October meeting and suggested that it may start a new program of buying U.S. government bonds to try to spur job growth.
The purchases are intended to lower long-term borrowing rates to encourage spending and strengthen the economy. The hope is that more hiring would follow. The yield on the 10-year Treasury note was unchanged at 1.59 percent.
Among other stocks making big moves:
Cisco Systems, the world's largest maker of computer networking equipment, gained 81 cents, or 4.8 percent, to $17.66. Cisco said late Tuesday its earnings rose 18 percent in the latest quarter and that U.S. companies are starting to spend again.
Mosaic, a company that mines for potash, a key ingredient in fertilizers, slipped $1.65, or 3.3 percent, to $49.10 after saying that international demand for its product had weakened and that it was lowering its sales forecasts.
Advanced Micro Devices, a chipmaker, slumped 16 cents, or 7.7 percent, to $1.93 after the company denied a report that said it was considering a sale. The stock has lost about 77 percent since March amid dwindling sales.
Facebook jumped $2.50 to $22.36. Wednesday was the expiration of a so-called lockup period that prevents insiders from selling stock immediately after a company goes public. About 850 million shares are being freed for sale.
Analysts had feared the stock would fall because of all the additional shares on the market. Youssef Squali, an analyst at brokerage Cantor Fitzgerald, said investors should take advantage of any "market dislocation" in the stock to buy more of it.