"The IOC has adopted a more hands-on role, it is unprecedented for the IOC, but there is no plan B. We are going to Rio," Coates said. "We have become very concerned. They are not ready in many, many ways."
Brazil has also come under fire from football's world governing body, FIFA, for long delays in construction of stadiums and other infrastructure and the overdue delivery of venues for the World Cup, which kicks off in June. Two years out from the Olympics, the situation on the construction front is just as bleak.
"And this is against a city that's got social issues that also have to be addressed; a country that's also trying to deal with the FIFA World Cup coming up in a few months," Coates said.
Work hasn't even begun at Deodoro, a complex for eight Olympics sports venues.
Earlier this month, in a rare display of criticism against an Olympic host, 18 sports federations publicly aired widespread concerns over Rio's preparations, with some sports asking about "Plan B" contingencies for their venues.
The IOC responded by sending a senior troubleshooter, executive director Gilbert Felli, to Brazil as part of a series of emergency measures to tackle the delays threatening the games.
Rio de Janeiro mayor Eduardo Paes last week responded to the high number of complaints by saying sports federations were making too many unnecessary demands and he expected them to continue complaining until the start of the games.
He said the federations were asking for too many "large things" that won't be used by the city after the Olympics, so he will keep fighting to provide only what will be absolutely necessary to guarantee successful games in two years.
Coates said dealing with three levels of government in Brazil made it harder for local organizers than it was for the heavily criticized organizers of Athens 2004.
"I think this is a worse situation than Athens," he said. "In Athens, we were dealing with one government and some city responsibilities. Here, there's three.
"There is bureaucracy, there is little coordination between the federal, the state government and the city which is responsible for a lot of the construction. The flow of funds from the federal government is not happening quickly enough. We think we need to help facilitate that."
Coates, who was involved in the organization of the Sydney 2000 Games as head of the Australian Olympic Committee and is chairman of the IOC's coordination commission for Tokyo 2020, said he thinks the IOC has got the message across in Brazil.
"We have to make it (the Olympics) happen and that is the IOC approach, you can't walk away from this," he said. "If it comes off the first (Olympic) Games on the South American continent, in a magical city in so many ways - it'll be a wonderful experience for the athletes."