Worshippers have been using a back room at vacant shop Tip Top TV to pray five times a day during the week, after turning it into a prayer space.
But the council says using the premises, in Old Lodge Lane, Purley, as a place of worship breaches planning laws, and that it will soon clamp down on "unauthorised activity".
The makeshift "mosque", with shoe rack, prayer mats and a 99 Names Of Allah wall chart installed, has also proved controversial with local residents, who say the location is inappropriate.
Diane Hearne, chairman of Hartley and District Residents' Association, said: "It is an unusual location for a mosque, and the objection is that it is on a road where there is not enough parking.
"We have had a number of phone calls from people concerned about it because they hadn't been told anything and wanted to be informed. There should be a planning application so people can have their say.
"It is not about it being a mosque, it would be the same for any place of worship, because there is not enough parking on that road and so we feel it is not an appropriate location."
The Advertiser visited the premises on Monday where around seven men, three of whom said they were locals, are using the building for prayers between 6.45am and 8pm.
Proprietor Kamran Hussain, who also runs the nearby mini cab office, said: "I suppose it was out of desperation as there were no facilities available to my staff that I opened the space.
"I have never openly advertised it as a mosque, I have not employed a priest or Imam and am sorry if it has caused any distress.
"If the public feel that we should not allow Muslims to privately pray in a private room then with sadness I will explain to my staff that they have to go somewhere which allows them to pray."
A worshipper, who wished to remain anonymous, said: "We just come peacefully here and go home peacefully. We come here to pray, nothing more."
Steve Hollands, borough councillor for Kenley, said planning permission would be needed for the men to continue using the shop as a place of worship.
Monir Mohammed, 40, a trustee of Purley Islamic Community Centre (PICC), said: "I think it reinforces the need for an Islamic centre in the community, one that is established through the correct channels."
PICC is currently in talks to buy an empty building near the Purley War Memorial Hospital to create such a centre.
A council spokesman said: "The council is aware of concerns regarding this address, has investigated the matter and has contacted the occupier of the premises with a view to ceasing any unauthorised activity."