Topic: Are automated accounts allowed? (Read 299 times)
I use "recent" indeed.
I'm working on this project right now, and I've almost finished the backend and the frontend, so the scraping is the last thing left. I've found that there's this interesting page - https://bitcointalk.org/index.php?action=recent
Is this all I need to setup my scraping? @LoyceV, is that how you save your posts? This page doesn't even require registration, so I don't have to create account for my app.
Is this all I need to setup my scraping? @LoyceV, is that how you save your posts? This page doesn't even require registration, so I don't have to create account for my app.
To do this I want to utilize the forum search function, and to do that I need to use an account.
This will not work very well. The forum search function is a) not very good, and b) very resource intensive on the forum's server. A superior way of doing this would be to use the See unread posts since last visit page, review the last pages of threads on this page until your bot gets to a post it has already reviewed, and check for keywords in-between the following:
Code:
[quote=
AND
]
Code:
[quote=
That would put me at constant 6 thread requests per minute, while the search can be configured with any rate you want, with a tradeoff of creating delays, potentially very long if there's too much subscribers.
If 6 threads per minute is not too much, then I'll stick to it, otherwise I'll use my original idea.
This is an old comment from theymos, but it could be useful in your research.
The limitation applies to the forum inbuilt search engine and not the Google search option.
The contextual search in the upper right searches the board you're on and all subboards. If this results in more than 5 boards, but less than all boards, then you get that error (recently-added for performance reasons).
If you want to search the entire forum, you can go to the search page or to the front page. The limitation I added is that you can either search all boards, or limit it to 1-5 boards. But you can't limit it to 10 boards, for example.
If you want to search the entire forum, you can go to the search page or to the front page. The limitation I added is that you can either search all boards, or limit it to 1-5 boards. But you can't limit it to 10 boards, for example.
The limitation applies to the forum inbuilt search engine and not the Google search option.
If 6 threads per minute is not too much, then I'll stick to it, otherwise I'll use my original idea.
My scraper (see Viewing unedited posts and deleted posts is currently not uploading (I need some time to test it), but when it works, I could provide you with a daily snapshot of all new posts. To do this I want to utilize the forum search function, and to do that I need to use an account.
This will not work very well. The forum search function is a) not very good, and b) very resource intensive on the forum's server. A superior way of doing this would be to use the See unread posts since last visit page, review the last pages of threads on this page until your bot gets to a post it has already reviewed, and check for keywords in-between the following:
Code:
[quote=
AND
]
Code:
[quote=
That would put me at constant 6 thread requests per minute, while the search can be configured with any rate you want, with a tradeoff of creating delays, potentially very long if there's too much subscribers.
If 6 threads per minute is not too much, then I'll stick to it, otherwise I'll use my original idea.
To do this I want to utilize the forum search function, and to do that I need to use an account.
This will not work very well. The forum search function is a) not very good, and b) very resource intensive on the forum's server. A superior way of doing this would be to use the See unread posts since last visit page, review the last pages of threads on this page until your bot gets to a post it has already reviewed, and check for keywords in-between the following:
Code:
[quote=
AND
]
Code:
[quote=
Thanks, I plan to do things a bit differently, instead of requiring messages to have specific pattern for mentioning, it will just look for posts that include quotes from subscribed users. I already do this by searching for my name with day limit set to 1, to see if someone has quoted my posts, now I want to automate it, add push notifications and release it to the public.
Great! Where you want the user/subscriber to receive the notification alert. Is it a thread, email, or a telegram account as well. Yes, some of them have their own bot for notification purposes you can ask @LoyceV and @Piggy for that. This telegram bot can help you as well, credits to @Piggy
Thanks, I plan to do things a bit differently, instead of requiring messages to have specific pattern for mentioning, it will just look for posts that include quotes from subscribed users. I already do this by searching for my name with day limit set to 1, to see if someone has quoted my posts, now I want to automate it, add push notifications and release it to the public.
Yes, some of them have their own bot for notification purposes you can ask @LoyceV and @Piggy for that. This telegram bot can help you as well, credits to @Piggy
I want to make a bot that notifies people when their posts are quoted. To do this I want to utilize the forum search function, and to do that I need to use an account. Are these types of accounts allowed, or will they get banned? And if they are allowed, are there any rate limits for searches? Thanks.
Jump to: