After reading your post, i came to know that even if we receive small amounts on the SAME bitcoin address, when we withdraw from that wallet, it will detuct a lot of fee. Don't know the reason for this as all the bitcoin are in the same wallet address.
Bitcoin transactions are based on outputs, not on balances. Every time you receive some bitcoin, that creates a new output, even if you receive to the same address multiple times. Each of those outputs remains separate until you combine them in a transaction as you just did, and when you do you must pay a fee for each individual output, regardless of how much bitcoin is on that output. I can see you had outputs ranging from around 10,000 sats up to 0.02 BTC.
Think of it like receiving physical coins and putting them in your pocket. Even if you receive hundreds of coins, they don't magically turn in to a $20 bill in your pocket. Only by spending them altogether and getting back a $20 bill in change can you combine them in to a single output.
Another thing was that i only send around 5$ to some address, why it used a change address in the transaction and move all my funds to a new address in my wallet. Maybe this was the reason of high fee
Yes, this was the reason for the high fee. You had plenty of outputs which could have made the 20,000 sat payment on their own. Instead your wallet chose to consolidate every output you had in the same address and return the rest to you in a single output as change. This will benefit you in the future by saving you fees in the long run, but as I said above, you could have performed this consolidation at a better time to pay less fees now as well.
Electrum has a feature known as coin control which allows you to spend just a single output when making such a transaction rather than consolidating all your outputs, if you wish.
How to send to certain wallet without using the change address
There will always be a change address unless you spend an output completely. Bitcoin cannot "leave behind" some part of an output. If you have an output of 20,000 sats and you send it all to the recipient (minus the fee) there will be no change. If you have an output of 100,000 sats and you pay 20,000 sats, the rest will be sent back to a fresh change address you control. You cannot skim off 20,000 and leave the other 80,000 behind.
Another question is that since the fee these days is always above 6Sat/vbyte usually, when can we perform that consolidations as you mentioned ?
If you don't need to spend those other outputs now, then just wait. At some point in the coming weeks or months the fee will be likely to reduce somewhat.