Here's what I think the answer is: a "golden key" will not exist IF those who initialize the trusted setup don't have compromised systems AND at least one of the initializers is honest in their execution of the setup.
That's also how I understood it. This is why you can trust a trusted setup, if you are part of the setup !
For that, you have to have:
1) a medical certificate testifying that you are not schizophrenic and that you can trust yourself
2) being sure that you are part of the setup
3) be sure that you didn't give out your secret part
In other words, you have to have a way to verify that the published setup has actually used your contribution. I don't know if this can be tested independently, but there is one thing that does work: if you have the public contributions of every contributor, then you can run the "compilation" yourself, and verify that it generates indeed, the published public setup.
It is indeed true that it is sufficient that ONE contributor has not given his part of the golden key (his random number/secret key) to the others to be sure that the golden key has not been generated. So if YOU are part of it, and you can trust yourself that you didn't give your key, then you know that the trusted setup can be trusted.
The subtle thing is probably that you are to be sure that that little piece of data has not left your computer. So you have in some way to be totally master of the software that did the generation. Best is to have the source code, and verify that it doesn't do any sending, but just a calculation on a console, and do this on an airgapped computer which is physically destroyed (or kept switched off in a safe, the day you want to be part of the conspiration that will have the golden key) once you copied the public results on a piece of paper.
Even if that answer is correct, it's still not a good system. Why take a chance? People are probably sick of me saying this, but it's true: we shouldn't have to trust people in crypto, just math.
If you are part of the trusted setup, you can trust it.
And if thousands of people are part of it, in the end, that comes down to "the market". In crypto, you have to trust the market. If ALL bitcoin holders except you decide that bitcoin has no value any more, then it doesn't matter that the protocol is well implemented. It is worth zero. So at a certain point, using a cryptoCURRENCY, you have to trust a market. If the trusted setup is set up with so many people that they are a serious part of the market, and if you could have been potentially part of it, then I think you can trust it enough.
But best is to be part of it.