Janša, popular with conservative voters in the euro zone country, was elected to parliament in a snap election in July while he was serving a two-year prison term for bribery.
After that, the two-time premier and leader of the conservative Slovenian Democratic Party was allowed to come to work in parliament and return to a prison near the capital Ljubljana on a daily basis.
Rival politicians complained the arrangement was illegal and his party called a vote to settle the matter. That vote was lost on Wednesday.
"We establish that a member of parliament Janez Janša has been sentenced to a prison sentence which is longer than six months so his mandate is terminated as of today," parliamentary speaker Milan Brglez told the assembly.
About 500 of Janša's supporters gathered peacefully outside the parliament during the vote.
His party - the second biggest in parliament - had said he would remain leader and it would appeal against the decision to the Administrative Court of Slovenia.
Janša was sentenced to two years in prison in June 2013 for bribery in a 2006 arms deal. He had denied taking money in the planned purchase of 135 armoured vehicles from Finnish defence group Patria in 2006 while he was the prime minister. The deal was eventually cancelled.
Jansš's centre-right government fell in February last year after an anti-corruption commission said Janša was unable to explain the origins of a significant part of his income over several years.