I totally agree with you, and I think this is huge.
For me, there were two major factors, one of which broke down into two related factors.
The standalone one was money. Recovering from the crappy relationship I was in took a lot of money. I don't mean paying for therapy ; I mean buying all new furniture (because he took most of the old stuff), moving back to my home town, changing careers, and so on. Most of my remaining "fertility window" coincided with those tasks.
The two sub-factors both had to do with potentially being a single parent. Dating was going nowhere fast, so choosing to be a single parent was a consideration. Since one of my parents died young, the prospect of starting a family with the minimum number of parents right off the bat did not appeal. Plus, all the successful deliberately single parents I knew had supportive extended families, and I knew from what my relatives had to say about deliberately single parents we were acquainted with meant I would not enjoy that support should I choose to go that route.
The way I see it, what I wanted was twisted and soured by my situation. I'm not bitter about it -- I assessed the situation and came to a conclusion I've agreed to live with -- but like I said above, I do get annoyed when people make assumptions.
The line between choice and privilege is a very narrow one.