Bitcoin transactions are just inputs and outputs. Whenever you make a bitcoin transaction you reference old unspent outputs which form your inputs and create new outputs for the recepient to spend.
Bitcoin transaction fees are denominated in satoshis per byte, which is the size of the transaction in bytes.
In a transaction inputs make up most of the transaction: in a standard transaction with one input and 2 outputs inputs take up about 65% of the transaction, and in the input the signature takes up 60% of the space.
So to recap so far in a bitcoin transaction we have inputs and outputs: inputs reference where the bitcoin is coming from, and outputs state where they're going to.
Inputs have ~70% of a transaction, and signatures in inputs take up ~60% of the input.
Signatures are only needed for validation yet they take up most of the space and you can't remove signatures because they are needed to prove said person has permission to move the bitcoin, but you can move them elsewhere, which is what segwit does.
It moves the signature to "witness" portion which doesn't count in transaction size so the transaction size becomes smaller and thus the fees are smaller.
Like I said above its the inputs that dictate the size of the transaction so no matter the type of address you're sending TO, it won't affect the size of the transaction and hence the fees.
However sending "FROM" a segwit address incurs less fees because transactions are smaller.