Would Adam and Eve have died if they hadn't sinned? Augustine gives a resounding "no", since death is the penalty for sin and you can't have a penalty without a crime.
The resurrection does not pull our spiritual selves down to the level of our sinful flesh, it rather pulls our sinful flesh up to the level of our regenerate souls. This will be a more glorious existence even than Adam and Eve experienced in the Garden, even accounting for their immortality. This is because Adam and Eve, for all the glory and perfection of their nature, still relied for life on the fruits of the earth. Christians, on the other hand, currently do and will more perfectly in the resurrection bodies rely on God directly for our life.
Some people want to allegorize the Garden of Eden:
The trees and fruit-bearing shrubs are turned into symbols of virtues and ways of living, as though they had no visible and material reality and as if Scripture had no purpose but to express meanings for our minds. The assumption here is that the possibility of a spiritual meaning rules out the reality of a physical Paradise.The problem is that this is one assumption too far. It's true that there is spiritual meaning (as we see Paul doing with the story of Sarah and Hagar in Galatians), but this does not preclude the possibility of a physical truth to the story as well.
But neither does the physical reality mean that there is no spiritual value. The Garden, for example, is a picture of the church, and "the four rivers are the four Gospels, the fruit-bearing trees are the saints... and the tree of life is, of course, the Saint of saints, Christ." [Here I think Augustine is getting a bit silly in his pushing the allegory, but still.]
The point is, such speculations are fine "so long as we believe in the historical truth manifest in the faithful narrative of these events." Which is a wonderful balance for a pastor to strike in his teaching.