If that's the case the person who finds your phone cannot access your wallet but also make sure you save your keys from separate device and don't left any hint on your phone so that you will be totally secured. But since it's mobile phone much better if you wouldn't trust so much since anything from it is not secured that's why it's advisable not to store huge volume of crypto's to avoid any risk in future and if you are a big holder then much better to grab a hardware waller for safe storing.