There are set rules governing what an address can look like. These rules are different for different address types. For example, legacy addresses must begin with 1, must use Base58Check encoding, must have a valid checksum, and so on. Native segwit addresses must begin with bc1q, must use Bech32 encoding, must have a valid checksum, and so on. If you break any of these rules, then this address is invalid. If you try to look up an invalid address on most block explorers they will return an error, and if you try to send funds to an invalid address then almost certainly your transaction will either be rejected by your wallet or rejected by the node you try to broadcast it to.
If, on the other hand, an address passes all these rules, then the address is valid, even if it has never received any coins. Such addresses are simply unused, rather than invalid.
If you look again at the two address examples I gave in my earlier post in this thread - the first one is invalid, while the second is valid but unused.