The Jews, living under Rome, have now been scattered across the world not necessarily as punishment for killing Jesus (though they certainly had a hand in that), but for so consistently refusing to believe the words of their own Scriptures. While it would be a stretch to say that Augustine is anti-Semitic (he quite clearly knows the importance of the Jews in this history of the church, and obviously longs for their salvation), we can say that there is language here that will later be historically abused that we can wish might have been clarified a bit.
When we ask whether there have ever been any Jews brought into the City of God, we have to say of course-- even the Jews in their most isolationist periods have never claimed that they were the only people to come to know God. The Old Testament itself disproves this claim!
The OT prophecies about the restoration and coming glory of the new temple are about Christ and the church, not about the building put up in Ezra.