In the heading for Chapter Seven, it says that Nephi and his brothers "return to Jerusalem." However, Nephi specifically wrote that they returned to "the land of Jerusalem." It would have been unwise, if not unsafe, for them to go into the city itself. Like Lehi, it is unlikely that Ishmael lived in downtown Jerusalem. So, what was the "land of Jerusalem?" This might reference the entire Judah tribal area, which extended well beyond Jerusalem’s city walls.
Why was Ishmael’s family chosen to join Lehi and his family in the wilderness? When did one marry in Lehi’s culture? Often when the children were very young, fathers put a contract together for the marriage of their children. In the case of Lehi’s children marrying Ishmael’s children, there may have been a previously established understanding. If not, Ishmael may have supported or had an interest in Lehi’s calling as a prophet and trusted Lehi’s sons when they came for his family. When Nephi and his brothers went back to get Ishmael’s daughters, Ishmael seemed to know that something important had transpired, because they were all willing to go—the whole family.
Laman, Lemuel, and the sons of Ishmael didn’t make it far outside of the land of Jerusalem before they wanted to go back to the lives they had left behind. Nephi argued with them, "How could you forget? After all you have seen and experienced, you still lack faith?" And then, in his final argument he says,
"Now behold, I say unto you that if ye will return unto Jerusalem ye shall also perish with them. And now, if ye have choice, go up to the land, and remember the words which I speak unto you, that if ye go ye will also perish; for thus the Spirit of the Lord constraineth me that I should speak." (1 Nephi 7:15)
Did they have a choice? Even if they went back to Ishmael’s estate, which is probably where the sons of Ishmael would have wanted to go, it wouldn’t have been very safe. Jerusalem was becoming more and more dangerous. The people had animosity toward Laman and Lemuel’s father, not to mention the fact that they were connected to Laban’s death. Nephi essentially said, if you think you have a choice, then go. And they didn’t go, so they must not have thought it was much of a choice either.