This is like suggesting the policy of the United States is set by "just a panel of less than 10 judges" and not the Federal government. Not only is SCOTUS part of the US government, it may actually be the most powerful part of the Federal government