People want to keep Jesus a baby because it keeps him out of power.
It is connected to the "Jesus on the cross" imagery that is so popular.
If Christ is always crucified and dying, or a baby in a manger, then he is relatively powerless and unable to hold you accountable.
That is why the world doesn't mind Jesus when He is portrayed in either of those positions. The idea of Jesus Christ returning in power, is offensive to the person who doesn't want to be told he is wrong about anything... and doesn't want anyone, not even God, to be a boss over him.