Come on, I'm pretty sure there a people you yourself know personally people that don't pray 5x a day, some will pray 2 or 3 some will not pray at all. I myself know countless of 'Muslims' who don't pray. On whether neglecting prayer constitutes kufr there is no debate, as this is what your Prophet said on the matter.
It was narrated that Buraydah ibn al-Husayb (may Allaah be pleased with him) said: “I heard the Messenger of Allaah (peace and blessings of Allaah be upon him) say: ‘The covenant that distinguishes between us and them is the prayer, and whoever neglects it has disbelieved (become a kaafir).’” (It was narrated by Ahmad, Abu Dawood, al-Tirmidhi, al-Nisaa’i and Ibn Maajah).