How does the title of "By the Waters of Babylon" add to the understanding of Benet's story?

Excellent question! Always look to the title of a piece when you are doing your analysis. Here, the biblical reference to Babylon brings an image of this great, beautiful, blessed, city which is of course completely destroyed. Depending on which angle you approach it, Babylon is either a great city or a terrible and wicked city, either apply to the short story (New York is referred to as the Babylon of the western world).