The phrase "City of God" is not one that Augustine made up on his own to explain some arbitrary or abstract concept, it is a real place spoken of in Scripture repeatedly (Psalm 87:3, for example). "[In Scripture] we learn of the existence of a City of God whose Founder has inspired us with a love and longing to become its citizens." This is opposed to that other city, the "city of man", which contains the bulk of the world bent on worshiping false gods and ignorant of the Creator of all. Book XI is dedicated to explaining where these cities came from.
God does not reveal Himself to man by physical means--He does not speak to us as you and I speak to each other, through the vibration of air from my mouth to your ear. Instead, He speaks as a Divine Being to a creature made in His image "by means of the truth itself, and to all who can hear with the mind rather than with the body."
Yet, because of sin even communication via the mind is closed off. In order to have our relationship with God restored, we need regeneration. This is why the Son of God took on flesh. "It was in order to make the mind able to advance more confidently toward the truth that Truth itself, the divine Son of God, put on humanity without putting off His divinity, and built this firm path of faith so that man, by means of the God-man, could find his way to man's God."
Because of the Gospel salvation is now available to man, but only in one way: "Now, there is one way and one way alone that can save us from all aberrations, the Way which is both God and man--God as the goal and man as the means to reach it." It is in Christ alone through His work as Mediator that the regeneration necessary for a restored relationship with God is possible.
In a sense, this Mediatorial work of Jesus is itself mediated. This came "first through the Prophets, then by His own lips, afterwards through the Apostles." This is not to say that God has left us dependent on second-hand knowledge per se; since "He also inspired the Scripture, which is regarded as canonical and of supreme authority and to which we give credence concerning all those truths we ought to know and yet, of ourselves, are unable to learn."
When you and I want to know something, there are two ways we can know it: we can see it for ourselves, or we can trust someone else. So too with the knowledge of salvation and reconciliation with God that comes not through the senses, but through (as discussed in chapter 2) a regenerate soul. Where we no longer witness the events of salvation in person, and we no longer have Jesus immediately before us, we have the testimony of Scripture and the witness of those who have believed it:
If our perceptions are of invisible things remote from our own interior sense, we ought to believe either those who have learned these truths as revealed in the Incorporeal Light or those who contemplate these truths in an abiding Vision of God.So we may not (indeed probably should not) expect a direct vision of Jesus to necessarily burst Saul-like into our lives and touch our souls, but we should see the display of the Gospel in Scripture ("those who have learned these truths as revealed") and in the lives of those who themselves believe, the Christians ("those who contemplate these truths in an abiding Vision of God"). There is in a sense a chain--not so much of authority, but of cause and effect--that runs from God through the Mediatorial work of Christ, which we can no longer see directly but which is described for us in Scripture and visible in its effects in the lives of Christians.