The pagans know that Jupiter isn't the one true God since they surround him with a host of other deities--gods of earth, air, fire, water, and time. In challenging these conceptions, Augustine is not only going after the myths, as he points out he is also taking aim at the philosophers. Specifically in this chapter he is targeting the pre-Socratic philosophers, who had various theories about the composition of the universe. While there were enough of them that a survey is perhaps more trouble than it would be worth here, their project can be summarized as a search for a "theory of everything", by means of which all that we would ever need to know about God, man, and the universe can be expressed in a simple and understandable axiom. Socrates (and his student Plato) would break away from these philosophers and approach philosophy from a different direction--and Augustine will have much to say about that in a later section. Here Augustine refutes both these theory of everything philosophers and the gods that go along with them by contrasting them with Christ: "all this nonsense ought to be completely abolished and extinguished by Him who is born of a virgin."
But what about the argument that there's one all-encompassing god (Jupiter, in this case), aspects of which may be seen in all the various other gods? Surely when pagans (I suppose today we would have a different list of false religions than 'pagan') worship these other gods, they're worshiping the truth of the one true god that they see on display there?
Augustine clearly has little patience with such religious pluralism--being sincere about your false religion, even worshiping in a false god a characteristic that is found in the true God of the Bible is absolutely not okay with Augustine. We cannot divide God and worship Him piecemeal--not even Jupiter would accept such devotion (as a moment's reflection reveals)!