Initially, it may appear that letting go is a physical activity. In reality, it is a mental one.
Yes, the Buddha began his life as a wandering seeker by leaving his princely existence. He traded his royal garments for the clothes of a seeker and had his jeweled necklace and sword taken back to his father.
But he went much further by letting go of his attachments to personal opinions and accepted ideas. He could only find a new way by letting go of the old as it was not providing the answers to his questions.
Fortunately, we can rely on what he learned and work from there.
We too need to let go of our attachments to personal preferences and viewpoints. As we focus less on what we want and more on what we will receive, we will relinquish our hold on what we thought would make us happy.