The maximum value length is in theory 1024 bytes (or 1023, not sure), as it always was - but yes, in practice 520 bytes is the longest you can do, otherwise you are not able to renew the name. (And the client will not easily allow you to register names with longer values, even if possible in theory).
As for the OP_RETURN size, I'm not 100% sure right now what the actual value is - you can check the code, but I can confirm that it should be the same as what upstream Bitcoin currently has (i. e., I did not change that).
I was looking into something other than a dns entry.
I have been working on this concept and was thinking that instead of a Bitcoin OP_RETURN, this might be best route might be using Namecoin's ledger as it is about as secure as Bitcoin but is built more for such concepts:
https://bitcointalksearch.org/topic/concept-opreturn-with-torrent-file-to-an-html-file-hostless-website-1593635
I see. If you are fine with registering a name and then having to let it expire, you can store up to 1023 bytes in it - but for your usecase, it is probably also enough to have much less data, as you only need enough information to find your stuff in a DHT. With Namecoin, you could actually store the DHT key (e. g., Torrent file) in a d/ name for your website, and even update it when your data changes. Then you can directly use the Namecoin name as the address for your site.