The --cpu-affinity parameter uses a hex-based mask to specify which combination of available threads should be used. Instead of specifying just the number of threads you want to use, you have to figure out a bit representation of all the threads you want to enable and then convert that to hex. This allows you to skip virtual or hyper-threaded cores and limit mining to just the physical cores. For example, on an Intel i7 with four physical and four virtual cores, I use 0x55 to indicate that only the four physical cores should be used. Basically each 5 represents enabling only the first thread in a pair. However, that value is based on the Windows convention of numbering physical cores starting at zero. I have seen somewhere that with Linux the first physical core starts at one instead - so you would need to reverse which threads are enabled by each bit. Finally, I would also suggest using both the --cpu-affinity and --threads parameters together just so the software can better understand your intent.
This is the best comment i can find anywhere online for --cpu-affinity after searching for days in the past and even years later today.
For my 2 physical cores to run 2 threads which is the best setting for yepower algorithm coins that need physical cores like veco coin and butkoin i use --cpu-affinity 0x5 --threads 2 since each 5 represents the first thread in a pair. it works with higher hash rate now. thanks