I'm sure it has less flaws than if anyone else were to have attempted it, it's way over my head!
Keep up the good work, I hope it is used effectively to reduce scamming/cheating.
Topic: Viewing unedited posts and deleted posts, view per post, per user or per topic (Read 8626 times)
February 16, 2024, 10:00:09 AM
February 16, 2024, 09:46:56 AM
It's probs because I am very VERY old!
Or you could read the post right above yours that gives the real reason:my "page detection" had some flaws
February 16, 2024, 09:00:18 AM
Just stumbled on this topic whilst doing some catch-up.
It's probs because I am very VERY old!
Appreciate you doing due-diligence whilst we were discussing my account revival.
Question:
Ratters has made a total number of 507 posts, but according to archive, There are only 472 archived posts. Could this be a case of some posts not getting archived?
Ratters has made a total number of 507 posts, but according to archive, There are only 472 archived posts. Could this be a case of some posts not getting archived?
It's probs because I am very VERY old!
Appreciate you doing due-diligence whilst we were discussing my account revival.
January 05, 2024, 10:45:52 AM
Ratters has made a total number of 507 posts, but according to archive, There are only 472 archived posts. Could this be a case of some posts not getting archived?
Good find! I don't have this post for example. Even worse: I only have the first 20 posts from this topic with 6448 posts. I guess my scraping didn't catch the second page, my "page detection" had some flaws sometimes.Considering this shitcoin topic was abandoned in 2014, I won't update my scraping.
I think the archival only archives posts after a given period of time. Those none archived posts were from before the archived date.
I also scraped older posts. 
January 05, 2024, 10:23:11 AM
I think the archival only archives posts after a given period of time. Those none archived posts were from before the archived date.
January 05, 2024, 10:03:54 AM
September 22, 2023, 01:20:56 PM
Could it be because of the API, which automatically updates the balance at source of the data, and that's why the post does not appear as edited on the forum?
Correct:Code:
[img]https://bitcoindata.science/api/addressbalance.php?address=bc1qfqd0mqhws95d5q2rv6cgeqyr7cyxd4slltg7ze07wt42x0xg7q5q9l55a0[/img]
The image needs to be generated when Bitcointalk's image proxy asks for it, and that doesn't always work (or only after a while). 
September 22, 2023, 01:02:16 PM
@Don Pedro Dinero, you are dealing with API calls :-D
legendary
Activity: 1288
Merit: 1491
The first decentralized crypto betting platform
September 22, 2023, 11:28:30 AM
Ever wanted to see who's lying when a post has been edited or deleted? I may be able to help!
I have a question about this. I am sure this post has been edited, and does not appear as such on either bitcointalk.org or loyce.club:
<...>
Week#8 payment sent: fecf5189175f804322440a38e72e1ad4d3050704d4568d076986a2ca03a92c2a
Current balance: 0.11616218 BTC (The balance is enough to cover more than 1 week)
Once there are refill, I will update.
<...>
Week#8 payment sent: fecf5189175f804322440a38e72e1ad4d3050704d4568d076986a2ca03a92c2a
Current balance: 0.11616218 BTC (The balance is enough to cover more than 1 week)
Once there are refill, I will update.
<...>
I'm not saying that my campaign manager lied or anything like that, I'm just curious because I was checking to see if the escrow had been filled in and when I didn't see it in later posts, I see that the balance has been updated in that post but without it being edited. At first I thought about asking it in the campaign thread but it would seem a bit off topic there. Could it be because of the API, which automatically updates the balance at source of the data, and that's why the post does not appear as edited on the forum?
After publishing the post I see that the image in question does appear as invalid but in the preview appeared correctly:
And now after posting this I see that the image in the quote has changed and appears as correct. Crazy.
July 08, 2023, 11:45:53 PM
Cloudflare on Bitcointalk is now blocking me from scraping (again). Unfortunately, I can't do anything about that.
I noticed it this morning, when my browser gave me a Cloudflare page before showing the forum.
I noticed it this morning, when my browser gave me a Cloudflare page before showing the forum.
July 03, 2023, 12:03:23 PM
The DDOS got worse before migigating it, and I missed more posts. Once in a while, my server managed to get some data again, and I'll only list the bigger gaps. I haven't archived any posts between the following msgIDs:
62490527-62490570
62490579-62490594
62490606-62490612
62490641-62490690
62490699-62490726
62490748-62490777
62490802-62490831
62490964-62491018
62491027-62491089
62491098-62491150
62491159-62491277
62491286-62491552
62491561-62494873
In total, I missed 4172 posts in the 62490000-category.
Long-term, I want to setup a backup scraper so I won't lose posts again. That's going to be complicated to process for members and topics, considering I still don't use a database. So that's why it's a long-term plan
62490527-62490570
62490579-62490594
62490606-62490612
62490641-62490690
62490699-62490726
62490748-62490777
62490802-62490831
62490964-62491018
62491027-62491089
62491098-62491150
62491159-62491277
62491286-62491552
62491561-62494873
In total, I missed 4172 posts in the 62490000-category.
Long-term, I want to setup a backup scraper so I won't lose posts again. That's going to be complicated to process for members and topics, considering I still don't use a database. So that's why it's a long-term plan
June 30, 2023, 01:23:59 AM
DDOS 
It looks like there's a DDOS going on, which started yesterday. I started DROPping ports, but I forgot to exclude Bitcointalk (it was quite late), so I broke my scraper.
For 7 hours and 2 minutes, I didn't download any posts between posts 62480523 and 62481467. That's 944 missing posts, which also mean my notifications failed for those.
It looks like there's a DDOS going on, which started yesterday. I started DROPping ports, but I forgot to exclude Bitcointalk (it was quite late), so I broke my scraper.
For 7 hours and 2 minutes, I didn't download any posts between posts 62480523 and 62481467. That's 944 missing posts, which also mean my notifications failed for those.
December 31, 2022, 11:11:58 AM
My bot is whitelisted as well, but when CloudFlare goes to its highest security level, the whitelist doesn't matter, as it will always show a Javascript challenge.
I was afraid of that:In this case it may not have mattered. Cloudflare was having a real bad morning today. Got the calls from a few clients that they could not get to their own sites from the own offices that are in the IP whitelist.
I'm just flabbergasted that Cloudflare doesn't allow a real whitelist. Makes you wonder: are they trying to force Tor users to use a less secure connection? What could they possibly gain from that? Wait, never mind:Cloudflare is very probably an NSA honeypot
December 31, 2022, 10:20:03 AM
My scraper got whitelisted, but don't know yet if that solves all problems.
My bot is whitelisted as well, but when CloudFlare goes to its highest security level, the whitelist doesn't matter, as it will always show a Javascript challenge. December 31, 2022, 10:17:35 AM
Yeah, CloudFlare is a bit of a bitch, but not much you can do.
My scraper got whitelisted, but don't know yet if that solves all problems. December 31, 2022, 03:37:17 AM
I've reduced my scraping frequency for new posts (to reduce unnecessary server load). I created my posts archive when there were many more posts per day, and the recent page only shows the last 10 so I have to be quick.
If I start missing more posts, that's either because it's spam and MindlessElectron deleted it before I scraped them, or because there were many posts in a short interval. I'll keep an eye on this.
If I start missing more posts, that's either because it's spam and MindlessElectron deleted it before I scraped them, or because there were many posts in a short interval. I'll keep an eye on this.
June 21, 2022, 04:01:10 AM
April 23, 2022, 04:01:59 AM
I've censored the following posts in my archive because of doxing (as requested here):
The first post has been removed from Bitcointalk, the other 3 posts have been edited by theymos. I didn't bother highlighting the changed, I just wiped them.
The first post has been removed from Bitcointalk, the other 3 posts have been edited by theymos. I didn't bother highlighting the changed, I just wiped them.
Jump to: