Topic: Assessing the impact of TLB trashing on memory hard algorhitms - page 4. (Read 7704 times)

Do you have a binary for the variable chunk size? I wonder if the future ethminer can also let user choose the chuck size for optimization.

I've modified the sourcecode a bit to allocate in 256MB chunks. Now it should be possible for AMD cards to get to use more RAM. On my GTX780, the hashrate curve is just about the same (tiny bit slower) when using 256MB chunks.   

I'm still trying to get my 7850 2GB do above 1280MB , but getting the out of memory error.  
Even with
set GPU_MAX_ALLOC_PERCENT=100 / GPU_MAX_ALLOC_PERCENT=95
set GPU_MAX_HEAP_SIZE=100
set GPU_USE_SYNC_OBJECTS=1

But it does seem to drop hard. from 1,024->1,280

Chunked (512) version below:

Chunked (256) version below:

What different access patterns? The ones in Eth are pseudorandom over the whole DAG file, IIRC.

This thread on the CUDA forums is most relevant:
https://devtalk.nvidia.com/default/topic/878455/cuda-programming-and-performance/gtx750ti-and-buffers-gt-1gb-on-win7/
Somebody over there (@allnamac) wrote a completely independent test that verified my findings.

This is not so interesting but shows the problems affect both NVidia and AMD:
http://gathering.tweakers.net/forum/list_messages/1659186

Interesting.
Please, can you share links to all discussions?

Considering OpenCL is possibly higher level than GL ever was, I'm quite surprised one pinpointed an hardware construct issue especially as GPUs are traditionally managed and there's a huge gap between different OS which in my experience should not be there for HW constructs... odd.

I have a 1GiB card so there's little I can do. I will try to take a look in the next few days if I can set apart some time. Initial analysis in CodeXL gave me inconsistent results.

Have you investigated different access patterns?

During the development of the CUDA miner for Ethereum, I ran into an issue where the hashrate on GTX750Ti dramatically drops when the size of the memory buffer the miner operates on exceeds a certain threshold (1GB on Win7/Linux, 512MB on Win8/10). After a long discussion on the CUDA forums, one of the designers of CUDA weighed in and identified the issue as TLB trashing. I'm currently conducting a bit of research on the subject and have created a simple test program that measures these effects. It simulates the 'dagger' part of the Ethereum algorithm at different memory buffer (DAG) sizes and writes the results to a CSV file. So far, I have concluded that it is not an Nvidia-only issue, but manifests on AMD hardware as well. And apparently this is not an ETH-only issue, I've got some reports from srcypt-jane miners in as well.

I'm currently looking for as many as possible hardware/OS combinations to come to a recommendation for miners as well as designers of new algo's. Below is an example for ETH hashrate on GTX780 on Windows with increasing buffer size (in MB):



The test program can be dowloaded from https://github.com/Genoil/dagSimCL. Win-64 binaries are in the x64/Release folder. You can also build it yourself, but only have supporting MSVC files targetted at Nvidia OpenCL. On AMD hardware you may want to run


first. By default, the program tries to use all of your GPU's RAM up until 4096MB. If you have less system RAM, you may add a cmd line param to test up until a lower maximum:


If you have multiple GPU's, you need to add a second param:


If you have multiple OpenCL platform installed:


I would be very grateful if you could participate in this bit of research and possible discuss any workarounds. Thanks!

p.s. note that achieved hashrates with the test program can be significantly higher than what you actaully get with ethminer. This is because it only simautes the Dagge stages, not the Keccak stages.