How can secondary infertility be worse than childlessness?

It just can't.

It would be impossible for anyone to know either so that's the first thing that bothers me about this. People, parents, say it hurts more TTC#2 without success as they know how wonderful it can be to be a parent and so they know what they are missing a second time around. Of course it's painful but in response to a rant of mine on this subject last year a wise woman who had one child via ICSI wrote (from memory) 'of course it hurts when you can't have another child, people never stop asking, your child never stops asking....when?....but at the end of the day you look at your life before when you were going through treatment and your study was a study not a nursery...my house is full of baby paraphernalia and baby noise and I'm accepted in a part of society I felt excluded from'.
So surely, while having one child not by choice is painful it is the difference between being a parent or not being a parent and that is huge! I wonder if people with secondary infertility would like to swap places with these childless people that it's 'easier' for?