We know that the "gods" don't grant kingdoms because of the rise and fall of other pagan kingdoms (Assyria et al) which were no less devout than the nations which conquered them, and yet fell anyway. And of course, they cannot blame Christians either since 1) Christ had not yet come when they fell and 2) they would have known nothing of Christianity in any case, being so far to the East.
Besides, which gods would we give credit to even if we did think it was the pagan gods? The pagans have multiple gods for everything, and picking one out to worship is always at best a crap shoot.
But let's not deal with these lesser deities, let's engage the chief 'god' of all--Jupiter ('Jove'). If the Romans owe their Empire to any single god, surely it must be the king of the gods whose temple is at the top of the Capitol Hill ("Jupiter Optimus Maximus", or "Jupiter the highest and greatest"). Yet, if this is true, why are the Romans allowed to get away with abusing him--namely with representing him as an image? "Why," asks Augustine, "has he [Jupiter] been so badly used at Rome (and indeed by other nations too), that an image of him should be made?" As even the great pagan writer Varro notes, making an image of your god results in both taking away the proper fear of the gods (by humanizing them) and adding error (by creating a likeness that is by definition inaccurate, to say nothing of the sin of idolatry!).
Even the pagans in their more thoughtful moments understand that idolatry and the worship of images are sins, and never to replace the worship of the one true God who alone gives and takes away power from the nations of the world.