If there's someone in your living room and you don't want him there, you just ask him to leave; first politely, then firmly, then violently. You don't have to give a reason, or allow him to appeal your decision to another house further down the street. You want him gone, he must go.
It is, or at least should be, the same at the national level. If a person who is not a British citizen is in the country and we, ie our government, do not want him here then he must go. We shouldn't have to give a reason, nor should there be any judicial process, if asked to leave a foreigner should just leave.
I'm thinking of course of the case of Abu Qatada who it seems is wanted in most countries in the world but not in the UK. He is not a British national, he entered the country on a false passport, there should be no question of him remaining.
We need a short enabling Act of Parliament to put the above principle into law, then any undesirable we want rid of should just be told: you have seven days to leave the country, after which, if you're still here, we will forcibly put you on a plane and send you to any other country that will take you - including delivering you to your worst enemy if that's most expedient for us. We should not even give a reason. If someone doesn't know why we find them undesirable it's not our job to educate them.
The only possible appeal against such an order would be on the grounds that the deportee is actually a British citizen. If he claims that, then an actual judicial process must take place, otherwise - sayonara!