I agree with you in the first place that you need to work hard to implement a passive income, in the beginning is always hard. But, in the long run you harvest the benefits and money come in your bank account.
You should continue to work on your website to get more traffic and more money for your future, don't give up
This is just like other blogger out there they are making too much work in the first time thinking about new blog post ideas and Doing Seo for organic search traffic.. building trust can help for your site and how long your site are alive that can get more subscribers until you can make even 1k CPM to your site that can make you passive income if you monitize your site properly..