Miner share finding is as capricious as block finding on the Bitcoin network. It has even more variance if the variable difficulty algorithm is setting your miner to a higher difficulty than 1 given your relatively low hashrate. Pools generally display hashrate over a small period of time, such as the last 10 minutes - the rate is better used just to see that miners are working.
You should analyze for yourself if the number of shares submitted over a period of time truly decreases, or if not decreasing, if the pool is properly crediting you for all submitted shares:
cgminer --sharelog sharelog.txt
The first entry in a log line is the epoch time in seconds. Inspect the number of shares submitted in a 10000 second period after miner startup vs a 10000 second period when the pool displays a lower rate than expected. Repeat for a different time interval.
If hashrate indeed decreases, many things might be going on, from OS entering a low power state, USB driver problems, overheating/bad hashing in the ASIC, etc.