Your bitcoin client will check all the blocks and if they are nonvalid, it won't accept them. There is no good way to fake the block chain, the whole "hardcode-first-block-hash-and-then-include-hash-of-previous-block-in-each-next-block" thing is what makes bitcoin so secure. So unless there is some 0-day bug in the bitcoin client I wouldn't worry too much about downloading block chain from unknown source.
That sounds like it makes sense. I can understand the position of being careful when one doesn't understand the entire technology/security involved though, and downloading from a more well-known url won't hurt.
If this is the solution then we need to bake this into the client, and use proper signatures and such. However, last time I did download the block chain using the client which is probably just a few months ago, it went much faster. Is this the result of a higher percentage of clients not accepting connections?
Hardcoding a download from a central source goes against the concept of bitcoin, and creates a weak point in the system. Downloading from peers is the only real thing to bake into the client.
Speed depends on connection speed, and number of connected peers and such. For me it took a few hours when I re-did it yesterday, IIRC.
Speed increasing is actually being worked on; the idea is to only download the block headers at first, making the client usable much faster. I believe the rest of the chain is then downloaded in the background, but don't hold me to that.