AES encryption is being employed as the hash function and assumed to be a random oracle with perfect distribution in order to provide the randomized memory access. Problem is that AES is not suitable as a hash (certainly not when employed as encryption) for it has too small of a output space (repeating patterns will be over a few number of bits), thus it will be possible to attack this with an algorithm to reduce the scatchpad size size significantly from the 2MB.
In the memory hard phase, and it uses 256-bit key sizes. This is followed by a number of SHA3 candidates at the bottom. Even if you broke the memory hard AES phase, you'd still have to contend with those.
So, whoever breaks 256-bit AES keys in the memory hard section is awarded most of the hash rate for the network. Good for them, and good luck to them.
I have no real concern with keeping it "CPU only". Whoever innovates the first GPU miner or ASIC miner or whatever should be rewarded accordingly for their efforts.
I think you've misunderstood my point. From ocular inspection of the code, the current 16 word value in the 2MB array is 'hashed' by applying AES encryption and this produces a new value and index into the array to store. Thus the uniform, random oracle, and thus non-patterned distribution of indices is assumed, otherwise an algorithm similar to a birthday attack can be applied to reduce the storage requirements in order to fun it faster on for example a GPU because more instances could be run simultaneously.
In short, AES encryption is not a cryptographic hash function and shouldn't be employed as one.
Thus I am not talking about breaking CryptoNote's slowhash function, rather I am pointing out that by misusing AES encryption, you are breaking the memory hard assumption.
If you are not concerned with keeping it CPU-only, then why call it "CPU only"? There are so many altcoins which have deceived on this point.