God uses all sorts, whether within His grace or outside of His grace, to profit His true children in His Glory.
It is possible that a pastor is not saved. But God can still use him nonetheless. And nontheless, the annointment of the pastor is not to be taken lightly, whether he is saved or not saved, ethical or unethical, moral or immoral.
A visible church may fall or not with or without a pastor.
God may even uphold His annointment on an unsaved, unethical or immoral pastor to a point where He allows evil to enter some members of the church so that with all their hatred at heart and justification excuses they will kick this pastor out and be used somewhere else. For it is definitely better than the pastor (saved or unsaved) staying in a dying church and ending God's use for him there. And that those members are prevented to affect other sheep in other pastures with their faulty ideals since with the pastor gone, their will is fulfilled, they are sure to stay in their own pasture, at least for a while.
If that is the case, my prayer shall be that some learn their lessons and some be saved in God's mercy, and that I see all these through in all righteousness and holiness with God's mercy upon me.