I guess we are facing same issue once again?
No solution yet?
Topic: Forum's Cloudflare turned to always on? (S.O.S bots) (Read 464 times)
Tor Browser is getting 403s on all Bitcointalk images, I'm guessing this is again CF doing something.
Never mind, you're not talking about images posted by users.
Scraping seems to be ok now, but Tor Browser is getting 403s on all Bitcointalk images, I'm guessing this is again CF doing something.
Has CF patched that WAF vulnerability yet? Seems it was enabled around the same time as this exploit was announced, I'm pretty sure it's been around a little longer though as it was attempted on 1splitkey last month.
A bunch of other sites that I deal with, mostly non crypto related are also dealing with the same issue. I'm guessing CF did not so patch it as do a quick and dirty work around and will fix it later.
If this is the final fix, there are going to be a lot of people with a lot of apps that are going to be really annoyed since there are a bunch of API calls and other things that are not working at the moment.
You wind up with the question of 'is a broken CF better then nothing"
-Dave
Today I woke up and saw this post on the BitcoinTalk SuperNotifier thread:
I started debugging and it looks like every request fails due to Cloudflare. I tried bypassing it but have had no success so far.
@theymos, is this here to stay?
loyce's and bpip's scraper also seems to be down? (paging @LoyceV, @suchmoon, @ibminer).
Debug on should help (theymos), CF caching fully on breaks a lot of stuff as update times can be up to 20 mins.Mayday mayday mayday…. Bot is down… I repeat Telegram bot is down!
Edit:
Oh I see now I konw why it's down... cloudflare thank you theymos... looks like its updated to not only for login page but on every first or new request to bitcointalk.
Edit:
Oh I see now I konw why it's down... cloudflare thank you theymos... looks like its updated to not only for login page but on every first or new request to bitcointalk.
I started debugging and it looks like every request fails due to Cloudflare. I tried bypassing it but have had no success so far.
@theymos, is this here to stay?
loyce's and bpip's scraper also seems to be down? (paging @LoyceV, @suchmoon, @ibminer).
Has CF patched that WAF vulnerability yet? Seems it was enabled around the same time as this exploit was announced, I'm pretty sure it's been around a little longer though as it was attempted on 1splitkey last month.
Are things back to normal today?
I’ve tried out my scraping processes for a short while, and none of them seemed to have encountered any 503 error nor any others of the kind. Scraping speed seems the same as before constant errors, as opposed to the experience over the last couple of days where it was at least twice as slow. I'd probably have to try it out at different time intervals, but right now it looks promising.
I’ve tried out my scraping processes for a short while, and none of them seemed to have encountered any 503 error nor any others of the kind. Scraping speed seems the same as before constant errors, as opposed to the experience over the last couple of days where it was at least twice as slow. I'd probably have to try it out at different time intervals, but right now it looks promising.
You are right it looks like everything is working OK for now, except the stats page it's still giving 503
https://bitcointalk.org/index.php?action=stats Error 503
Are things back to normal today?
I’ve tried out my scraping processes for a short while, and none of them seemed to have encountered any 503 error nor any others of the kind. Scraping speed seems the same as before constant errors, as opposed to the experience over the last couple of days where it was at least twice as slow. I'd probably have to try it out at different time intervals, but right now it looks promising.
I’ve tried out my scraping processes for a short while, and none of them seemed to have encountered any 503 error nor any others of the kind. Scraping speed seems the same as before constant errors, as opposed to the experience over the last couple of days where it was at least twice as slow. I'd probably have to try it out at different time intervals, but right now it looks promising.
<...> except the stats page it's still giving 503 <...>
It loads on my end on multiple non-tor browsers, albeit showing the "Checking if the site connection is secure" message on most of them.Unfortunately I'm getting a 503 on https://bitcointalk.org/index.php?action=login2;ccode=... which makes it impossible for the parser to log in. But at least it's something I can experiment with and see if tweaking some parameters would help.
I log in using https://bitcointalk.org/index.php?action=login;ccode= and it works well, is there a difference between login and login2?login2 happens when you click the "Login" button. I guess it redirects to index.php and throws the error, but at that point the login is completed and cookies should be good so I'll try to ignore that error and see if it can load any other pages afterwards. It just sucks having to code all these sketchy workarounds that will probably break the next time Cloudflare tweaks something.
Edit: even now, i still see CloudFlare security check page on few occasion.
I don't know what the problem is, I also experienced it. It's frequent and I've experienced CloudFlare security check 2 times in the last hour. Indeed, at this time my internet is not smooth where several times I have to turn off the wifi device because the internet is disconnected, but I'm sure this is not an internet problem.
I hope there is a solution to this problem quickly, I will be bother if the frequency is getting shorter even this CloudFlare security check doesn't take much time ''only 3-5 seccond".
Unfortunately I'm getting a 503 on https://bitcointalk.org/index.php?action=login2;ccode=... which makes it impossible for the parser to log in. But at least it's something I can experiment with and see if tweaking some parameters would help.
I log in using https://bitcointalk.org/index.php?action=login;ccode= and it works well, is there a difference between login and login2?-
Regards,
PrivacyG
Strange it may seem but this is what I figured out for now if you navigate to https://bitcointalk.org/index.php you will still get 503 error. However if you navigate to to any other page or only https://bitcointalk.org/ it bypasses Cloudflare
Unfortunately I'm getting a 503 on https://bitcointalk.org/index.php?action=login2;ccode=... which makes it impossible for the parser to log in. But at least it's something I can experiment with and see if tweaking some parameters would help.
shahzadafzal is correct. I am only running my browser with Java Script off and I can log in to Bitcoin Talk using ccode but Home page does not load. The rest seems to work properly. I am lucky I know 'index.php?action=profile' means viewing my own profile and that is how I skip the error page.
-
Regards,
PrivacyG
-
Regards,
PrivacyG
Perhaps it's the bots that were most affected.
I use Tor Browser and yesterday i was also seriously affected with CloudFlare security check page. I managed to reduce frequency of check page by only opening 1 tab, change Tor circuit occasionally and save post text/draft on text editor. But it wasn't pleasant experience.
Edit: even now, i still see CloudFlare security check page on few occasion.
loyce's and bpip's scraper also seems to be down? (paging @LoyceV, @suchmoon, @ibminer).
True, I can't scrape anything - getting 503 errors.
Strange it may seem but this is what I figured out for now if you navigate to https://bitcointalk.org/index.php you will still get 503 error. However if you navigate to to any other page or only https://bitcointalk.org/ it bypasses Cloudflare
For example:
https://bitcointalk.org/index.php Error 503
https://bitcointalk.org/index.php?action=stats Error 503
https://bitcointalksearch.org/user/shahzadafzal-1634314 200 OK
https://bitcointalksearch.org/topic/voting-2022-bitcointalk-community-awards-5422131 200 OK
https://bitcointalk.org/index.php?action=merit;stats=recent 200 OK
I think only theymos can guide here more.
Edit: Strangely enough you can bypass Cloudflare for the home page just by adding any parameter e.g. https://bitcointalk.org/index.php?s=1
loyce's and bpip's scraper also seems to be down? (paging @LoyceV, @suchmoon, @ibminer).
True, I can't scrape anything - getting 503 errors.
1. If Cloudflare pops up, calls a function that invokes a puppeteer page (an instance of chrome) to the forum and waits for CF to clear out.
I have a prototype of a parser that works entirely via Playwright/Firefox and even solves captchas but it'd be quite absurd to use it. Not only because it needs an ungodly amount of RAM to work, but also because it looks like we're going against the policy (unwritten as it may be) of Cloudflare and/or Bitcointalk. This needs to be solved properly, i.e. by implementing some sort of whitelist/key/whatever. Not by trying to hack the damn thing.
Edit: my other parser that I use for plagiarism checks is still running albeit intermittently, not sure what the difference is that makes this one kinda sorta work.
I don't feed cookies to my scraper, so it doesn't index the Investigations board. I use cookies for other tasks, but none of them were running so I can't tell if it would have worked.
Bpip fed cookies under Vod's code (it probably still does) to get user info so we probably already know the issue still persisted during that time.
I wonder if this was resolved by whitelisting everything or just the bots, we probably won't find out similarly to if this was something automated by CF, an accidental kill switch triggered or was implemented for a reason (all are quite likely).
Based on the up time of this forum, there has to be multiple people with access to all connection information (there's probably more that could do defensive operations too - such as a controlled rerouting or changing certain cloudlfare settings to improve performance or limit some more intense pingers).
Happened to me earlier on, though, I was just accessing my account that was already logged in. The unusual check appeared, and I had initially thought it was just me. Nonetheless, it was not slow, so I was ok with everything. Perhaps it's the bots that were most affected.
After 10 minutes of waiting, I logged in this time. Something happened to the server, or it's just because of the user load?
But now I confirmed it is not because of my connection.
But now I confirmed it is not because of my connection.
I was on an endless checking if your connection is secure loop, closed the site, came back 10 min later to see I'm logged in.
I am still having issues with Cloud Flare. I still get the page asking me to enable Java Script every now and then. Got it for about 3 times in the last hour while browsing the forum. Weirdly enough, it typically appears if I go to the Home page of the forum.
-
Regards,
PrivacyG
-
Regards,
PrivacyG
Is there way to bypass it or somehow configure it to use our existing (live) session or pass cookies for the scripts?
I already pass the bot's session (cookies) to my scraper, but that didn't help. I may have found a possible workaround, but I didn't have the time to test it.1. If Cloudflare pops up, calls a function that invokes a puppeteer page (an instance of chrome) to the forum and waits for CF to clear out.
2. Grabs the cookie from CF that lets you use the forum normally and passes it to all my other headless requests.
There is a chance that CF could ask me to solve a captcha though, so that would be a dead end.
Yes looks like it's working now, but at first I thought that 1 sec time has cost me a ban to my IP 
But later saw Telegram Bot and ChartBuddy also down then realized its not only me.
So.... you're saying you broke it for everyone But later saw Telegram Bot and ChartBuddy also down then realized its not only me.
Quote
Is there way to bypass it or somehow configure it to use our existing (live) session or pass cookies for the scripts?
I don't feed cookies to my scraper, so it doesn't index the Investigations board. I use cookies for other tasks, but none of them were running so I can't tell if it would have worked. loyce's and bpip's scraper also seems to be down? (paging @LoyceV, @suchmoon, @ibminer).
I hadn't noticed yet, but indeed, I miss all posts between posts 61426048 and 61427300. That's 1251 missing posts in 5 hours and 40 minutes.Yes looks like it's working now, but at first I thought that 1 sec time has cost me a ban to my IP
But later saw Telegram Bot and ChartBuddy also down then realized its not only me.
Theymos hasn't been online.
That means cloudflare itself did this or some rule triggered it... It's quite possible in future it might be turned on again or may be permanently.
Is there way to bypass it or somehow configure it to use our existing (live) session or pass cookies for the scripts?
Looks like we are back! Requests aren't failing anymore.
Thanks, theymos?
Thanks, theymos?
I was on my way tryna post this, I have switched between Firefox and Chrome all day trying to figure out may be its my connection or the browser. I have seen some error messages logining before, but I haven't encounter this since signing up here in the forum. I just hope it's a temporary error and it's fixed quick
Today I woke up and saw this post on the BitcoinTalk SuperNotifier thread:
I started debugging and it looks like every request fails due to Cloudflare. I tried bypassing it but have had no success so far.
@theymos, is this here to stay?
loyce's and bpip's scraper also seems to be down? (paging @LoyceV, @suchmoon, @ibminer).
Mayday mayday mayday…. Bot is down… I repeat Telegram bot is down!
Edit:
Oh I see now I konw why it's down... cloudflare thank you theymos... looks like its updated to not only for login page but on every first or new request to bitcointalk.
Edit:
Oh I see now I konw why it's down... cloudflare thank you theymos... looks like its updated to not only for login page but on every first or new request to bitcointalk.
I started debugging and it looks like every request fails due to Cloudflare. I tried bypassing it but have had no success so far.
@theymos, is this here to stay?
loyce's and bpip's scraper also seems to be down? (paging @LoyceV, @suchmoon, @ibminer).
Jump to: