IRS says you are “required to file”, but in reality the only penalties for non-filing are as a percentage of the amount of tax owed. If no tax is owed (and in fact you are owed a refund), then there are no penalties for not filing.

OTOH it would be a pretty dumb move since the chances that the amount taken out of your checks was exactly right is very small, and you’d be leaving hundreds or thousands of dollars in refunds unclaimed.