Since your point seems to be that not all the founding fathers parent was referring to were actually slave owners do you have a claim for a rough ratio? I think that would be interesting and would be a more informative thing regardless of where on the scale it lands from "everybody but Adams" all the way up to "only a big names like Washington, Jefferson".
To a first order approximation half the founders were from New England (no slavery) and the other half were from Virginia (no realistic chance of being important/rich enough to be a signatory without owning slaves). So call it 50-50
There's a reason the founding fathers all had slaves; they weren't the common folk.