wanting time away from your loved ones doesn't seem the least bit weird to me.

Like sure you love your wife and kids, but being with them every second of every day is probably too much for most sane people. Especially if you have hobbies that demand a lot of focus time.