I found this article on downsizing in retirement, and I like it because it shares some fairly balanced viewpoints on downsizing. One of the downsides is leaving friends and neighbors behind as you move. Typically, when downsizing, you end up leaving your neighborhood and relocate to a different section of the city or to another city entirely.
The article also touches on some of the financial aspects, although the completely left out renting as an option, and instead focused on downsizing from a house to a condo. They also failed to mention the decreased living expenses of a smaller space, leaving out things like lower utility costs.
I have one tip when moving, especially when downsizing. After you move into your new home, find some activity to get you out and among other people. When we moved we found a local roller skating rink, and spent a lot of time there initially. Once our daughter started making friends, we decreased our roller skating time. It was a great transitional activity. You might also look into clubs or groups in your new area to help you meet people and make new friends.