I Have No Reason to Call the Pope as "Holy Father"

I find it irritating how often Roman Catholics come up with whatever poor excuse they have to call the Pope as the "Holy Father". Whether they like it or not Peter (who they assume as the first Pope) was not called "Holy Father" even once in Scripture. Not even the Good News Translation calls Peter as Holy Father. Guess who the GNT calls as "Holy Father" and it ain't Peter!

John 17:11 

And now I am coming to you; I am no longer in the world, but they are in the world. Holy Father! Keep them safe by the power of your name, the name you gave me, so that they may be one just as you and I are one.

Yup, when Jesus called God the Father as Holy Father in the religious sense then there's only one Holy Father or Most Holy Father. The wrong interpretation is that they say since 1 Peter 1:16 says that God tells people to be holy for He is holy as an excuse. Holiness for people is imputed but the problem of calling the Pope as "Holy Father" is blasphemy. Matthew 23:8-9 says that no religious leader is to be called Father in the spiritual sense. To be a father in the faith means to be a father figure but not in the same way as the Roman Catholic priest or the Pope.

Peter wasn't even called as Holy Father even once in Scripture nor is there anything to suggest about it. If Jesus didn't even call Peter as Holy Father nor did the other apostles then why should I even assume that the Pope should be called as such? If he truly succeeded Peter in an "unbroken succession" then what makes people think the Pope should be called as Holy Father?