Ever since I left the farm, my experience of manual labor has been mostly at my discretion. That's key. Even when it isn't, as when we get a big snowstorm and I have to shovel out it's not too bad. I can tell myself I'm fighting the elements, along with some (not many, the softies) of my neighbors.
Gardening is a hobby with a reward of vegetables, so the work involved is just a prerequisite to a payoff.
There have been times when I really got into using my hands. The family farm was populated with my father's constructions and improvements: hen houses, brooder houses, range shelters, milk house, cow shed, equipment shed, 3 stall garage, etc. My attempts were more domestic, dining room table and chairs, light box, book shelves, etc. That was all after I bought my house and before marriage.
This last week or so has brought me back to manual work, not building anything, but repairs around the house. It's been a change from my usual winter routine of reading and TV, but enough to remind me of the satisfaction that can come when you fix something, not as well or nearly as quickly as the expert who does the work for her living, but "good enough fo,r government work", as we used to say.