We need to understand that there is no such global mempool or as such. Each node contains its own mempool and they store the recent transactions which they have listened from other peers. As far i have understood, we can configure our own node to accept certain set of transactions, but if you are a miner this will put you into trouble since you are mining those invalid transactions. Full node operators don't follow rules of the network to store the transactions in the mempool. Since each of the node contains their own mempool, they can configure as they wish to contain any such transactions if they are not accepted as valid in the consensus rules.
Let us consider a situation that you are a miner here. Your node accepts invalid transactions, includes that in a block and broadcast them to the network. Once the other nodes listens to the new block found, they will validate that block along with the transactions present inside them. If the block is valid, they will be added to the chain and simultaneously all other valid transactions that has been mined are removed from the mempool of the node. OTOH, if the block is invalid the node doesn't add them to the chain and further blacklists the node which has broadcasted the new invalid block.
Whenever you start your node, you see that the wallet loads the 'Banlist' actually. The Banlist consists of certain nodes which has incurred certain amount of Banscore over the period of time. Banscores are coded in the source code of the bitcoin, and if a particular node reaches certain score they are updated in the Banlist probably.
In basic terms, when a node finds a reason the banscore increases and they are added to the Misbehaving function present in the net_processing.cpp#L943
void Misbehaving(NodeId pnode, int howmuch, const std::string& message) EXCLUSIVE_LOCKS_REQUIRED(cs_main)
{
if (howmuch == 0)
return;
CNodeState *state = State(pnode);
if (state == nullptr)
return;
state->nMisbehavior += howmuch;
int banscore = gArgs.GetArg("-banscore", DEFAULT_BANSCORE_THRESHOLD);
std::string message_prefixed = message.empty() ? "" : (": " + message);
if (state->nMisbehavior >= banscore && state->nMisbehavior - howmuch < banscore)
{
LogPrint(BCLog::NET, "%s: %s peer=%d (%d -> %d) BAN THRESHOLD EXCEEDED%s\n", __func__, state->name, pnode, state->nMisbehavior-howmuch, state->nMisbehavior, message_prefixed);
state->fShouldBan = true;
} else
LogPrint(BCLog::NET, "%s: %s peer=%d (%d -> %d)%s\n", __func__, state->name, pnode, state->nMisbehavior-howmuch, state->nMisbehavior, message_prefixed);
}
You can find such reasons in the same code base by searching for 'Misbehaving' keyword.