The accident occurred when the workers were preparing for a shift change, officials said, which likely raised the casualty toll because there were more miners inside than usual.
Yildiz said the deaths were caused by carbon monoxide poisoning and feared the toll could end up much higher than the latest count of 166 workers.
He said rescue operations were hindered because the mine had not been cleared of gas.
Authorities say the disaster followed an explosion and fire caused by a power distribution unit.
Nurettin Akcul, a mining trade union leader, told HaberTurk television that Turkey was likely facing its worst mining accident ever.
"Time is working against us. We fear that the numbers could rise further," Yildiz said. "We have to finish this (rescue operation) by dawn. I have to say that our pain, our trouble could increase."
Earlier Yildiz said some of the workers were 420 meters (460 yards) deep inside the mine. News reports said the workers could not use lifts to get out of the mine because the explosion had caused a power cut.
Television footage showed people cheering and applauding as some trapped workers emerged out of the mine, helped by rescuers, their faces and hard-hats covered in soot. One wiped away tears on his jacket, another smiled, waved and flashed a "thumbs up" sign at onlookers.
Authorities had earlier said that the blast left between 200 to 300 miners underground and made arrangements to set up a cold storage facility to hold the corpses of miners recovered from the site.
Prime Minister Recep Tayyip Erdogan postponed a one-day visit to Albania scheduled for Wednesday and planned to visit Soma instead.
Hundreds of people gathered outside the mine and the hospital in Soma anxiously seeking news of family members. NTV television said people broke into applause as rescued workers arrived in ambulances. Interviewed by Dogan news agency, some complained about the lack of information from state and company officials about the situation of the trapped workers.
One woman threw herself on the ground, crying after hearing about the death of a loved one, HaberTurk television showed. There were tears of joy for another who told the station she had just spoken by telephone to a missing relative.
Police set up fences and stood guard around Soma state hospital to keep the crowds away.
SOMA Komur Isletmeleri A.S., which owns the mine, said the accident occurred despite the "highest safety measures and constant controls" and added that an investigation was being launched.
"Our main priority is to get our workers out so that they may be reunited with their loved ones," the company said in a statement.
Mining accidents are common in Turkey, which is plagued by poor safety conditions.
Turkey's worst mining disaster was a 1992 gas explosion that killed 263 workers near the Black Sea port of Zonguldak.
Fraser reported from Ankara, Turkey.