Viewing unedited posts and deleted posts, view per post, per user or per topic - page 5.

TheQuin

hero member

Activity: 2576

Merit: 882

Freebitco.in Support https://bit.ly/2I9BVS2

Thanks for providing this service. It saved me a lot of time rewriting a post in a thread that got trashed yesterday.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Yesterday, I created LoyceV's Topic Details: highlight deleted and edited posts (forum wide):

Quote from: LoyceV on June 20, 2020, 03:42:25 AM

Short version:
Get a topicID you want to see, for instance 5145594.
Insert the topicID into the following link and post it on any public board on Bitcointalk: http://loyce.club/archive/details/topic_5145594.html
Wait a bit, then click the link!

(click the link for the full version)

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: LoyceV on July 21, 2019, 01:08:59 PM

Viewing unedited/deleted posts

See http://loyce.club/archive/posts/ for all posts (Working!)
New posts are archived within seconds after being created, and instantly available.
See http://loyce.club/archive/members/ for posts made by a certain user (Working!)
Updated every 5 minutes.
See http://loyce.club/archive/topics/ for posts made in a certain topic (Working!)
Updated every 5 minutes.

Question: would it be useful to add links from all archived posts to the other categories (and the other way around)? I can quite easily update the "members" and "topics" category, but updating the "posts" will be more work.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: Vod on June 05, 2020, 05:48:44 PM

There are only 100,000 active users. Don't need to recheck posts made by inactive or banned users.

That could actually work, but it's still a lot of scraping to do. I won't do it, even if the average user has only 200 posts, that means scraping a million pages on a regular basis. With 5 seconds delay, it takes 2 months to find changed posts.
Considering how much I scrape already, I don't think this is worth it.

Vod

legendary

Activity: 3668

Merit: 3010

Licking my boob since 1970

Quote from: LoyceV on June 03, 2020, 05:55:03 AM

There's no way for me to know which posts have been edited. I'd have to check all 50 million posts again.

There are only 100,000 active users. Don't need to recheck posts made by inactive or banned users.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Millions more posts added:
I have now archived the first 35.5 million posts, all available online. This included posts made in topics created until April 24, 2018 and currently fills 43 GB.
Example: my first post!

See this quote on how to use it:

Quote from: LoyceV on February 02, 2020, 02:05:42 PM

Sneak preview: http://loyce.club/archive/oldposts/
How to use:

Find the msgID you need. Let's use 28228
Remove the last 5 digits from the msgID to get the directory name (if there are less than 5 digits, use 0): 0
Replace the last 2 digits of the msgID by xx, and add .html (if there are less than 5 digits, use 0xx): 282xx.html
Add "#msg" and the msgID: #msg28228
Put everything together and go to http://loyce.club/archive/oldposts/0/282xx.html#msg28228

Limitations

Currently, the first 2.1 million posts are available.
I'll scrape the first 5.21 million topics and all posts in there.
That means I'll archive 53.36 million posts, this partially overlaps with my scraper for new posts.
This is a one-time thing, I won't update it with newer posts (I scrape unedited versions for those).
The time "scraped on" is Amsterdam time.

If no username is mentioned, it's either "Anonymous" or "random". I forgot those exist when I started scraping, and it's not important enough to start over.

This bug is not fixed yet:

Quote from: LoyceV on April 23, 2020, 08:10:39 AM

I found a bug (which I'm posting here as a reminder to myself): Posts on the עברי (Hebrew) board don't show up. Example: this post is missing, while it exists.
I'll see if I can add them later. I think it has something to do with the right-to-left writing, even selecting text on that board doesn't work as expected.
Update: عربية (Arabic) has the same problem.

I'll re-scrape these boards after finishing scraping all posts.

Todo:

Quote from: LoyceV on June 03, 2020, 06:13:11 AM

When I have the time, I'll create something to classify all posts in a requested topic as "unedited", "deleted and archived", "edited within 10 minutes" or "edited after 10 minutes". But that will only be for one topic at a time, you can't easily check all posts.

Another Todo: I should create this per user, that could prove very useful. Deleting a post would make that post stand out more!

ChuckBuck

hero member

Activity: 1372

Merit: 783

better everyday ♥

Quote from: LoyceV on June 03, 2020, 05:55:03 AM

I only archive the original unedited post.

Ahaa got it, it means you start browsing through all the new posts posted on this forum, then archive it.

Quote from: LoyceV on June 03, 2020, 05:55:03 AM

There's no way for me to know which posts have been edited. I'd have to check all 50 million posts again.

An impossible idea if I want to browse 50 million posts. There is no way to browse through them within seconds Cheesy

I had a wrong thought here. Yesterday, I was on another forum, when I edited the post, the post ID was changed. But I just checked, it doesn't happen on this forum Cheesy

We can skip my idea now Cheesy

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ChuckBuck on June 03, 2020, 05:47:52 AM

I have a small question here, hope you can answer it Cheesy

I have a habit of publishing my post early, then making necessary edits, such as updating information and data. This process can be 1 time or 2 times, it can be 3 times or more. So, are they all stored? Or just the original post is archived?

I only archive the original unedited post.

Quote

I think it would be interesting if all the edits were saved Cheesy

Have you thought about it yet?

There's no way for me to know which posts have been edited. I'd have to check all 50 million posts again.

ChuckBuck

hero member

Activity: 1372

Merit: 783

better everyday ♥

I have a small question here, hope you can answer it Cheesy

I have a habit of publishing my post early, then making necessary edits, such as updating information and data. This process can be 1 time or 2 times, it can be 3 times or more Cheesy

So, are they all stored? Or just the original post is archived? I think it would be interesting if all the edits were saved Cheesy

Have you thought about it yet?

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Update: you can now use TryNinja's site to search most posts created since June 2019:

Quote from: TryNinja on May 16, 2020, 06:43:40 AM

- Since I'm already scraping and saving posts, I'm also providing a website that lets you easily search for posts with filters (author, text, in X topic and date range) and share the scraped post so you can send to someone (this original post, for example) or even archive it. I cleaned the database to make some changes and release the bot, so posts just go as far as May 14th. But I have enough hosting and space for at least 1 year of posts (and then I can renew), so you may be able to use it better in the future. Edit: now with +2,5 million old posts from loyce.club

You can use it here: https://posts.ninjastic.space

ChuckBuck

hero member

Activity: 1372

Merit: 783

better everyday ♥

Quote from: Rikafip on May 17, 2020, 02:47:24 AM

Post is in the link I just gave you, and there were no edits of that post, that I can see. Maybe you meant some other post and wrote wrong message ID?

I also thought I was wrong somewhere Cheesy

Quote from: LoyceV on May 17, 2020, 02:50:01 AM

Just follow the instructions, it should get you all unedited (or deleted) posts made by Zwei and all unedited (or deleted) posts in [OPEN] WOLF.BET Signature Campaign - Hero/Legendary - Up to 0.008 BTC/week!. The post should be in there.

I have found it following your instructions. From the list you gave above, I tried reading each post behind my post. But there are no post that contain content that I need Cheesy

The bot announced, it always works right whenever someone quotes my post or mentions me. But now I can not find. Maybe it's from my side Roll Eyes

Anyway, just a curiosity, I think I should ignore it Cheesy

Thank you very much!

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ChuckBuck on May 17, 2020, 02:45:02 AM

Can you help me find the deleted post?

Just follow the instructions, it should get you all unedited (or deleted) posts made by Zwei and all unedited (or deleted) posts in [OPEN] WOLF.BET Signature Campaign - Hero/Legendary - Up to 0.008 BTC/week!. The post should be in there.

Rikafip

legendary

Activity: 1722

Merit: 5937

Quote from: ChuckBuck on May 17, 2020, 02:45:02 AM

Quote from: Rikafip on May 17, 2020, 02:37:22 AM

Hey Chuck, are you sure that's the correct topic ID? If I am not mistaken, that should be Wolfbet signature topic and this post
http://loyce.club/archive/posts/5443/54439447.html

The ID of topic is exactly. That means the article I need to read has been deleted Cheesy

I haven't read that post, but I know it mentioned me because I received a notification from the bot. Some first characters were quoted by the bot and I found that it mentioned me. I also know that it is made by Zwei. Can you help me find the deleted post?

Post is in the link I just gave you, and there were no edits of that post, that I can see. Maybe you meant some other post and wrote wrong message ID?

ChuckBuck

hero member

Activity: 1372

Merit: 783

better everyday ♥

Quote from: Rikafip on May 17, 2020, 02:37:22 AM

Hey Chuck, are you sure that's the correct topic ID? If I am not mistaken, that should be Wolfbet signature topic and this post
http://loyce.club/archive/posts/5443/54439447.html

The ID of topic is exactly. That means the article I need to read has been deleted Cheesy

I haven't read that post, but I know it mentioned me because I received a notification from the bot. Some first characters were quoted by the bot and I found that it mentioned me. I also know that it is made by Zwei. Can you help me find the deleted post?

Rikafip

legendary

Activity: 1722

Merit: 5937

Quote from: ChuckBuck on May 17, 2020, 02:07:13 AM

Hi LoyceV,
It's me again Cheesy

There was an edited post that mentioned me in it. I have not read it to the fullest yet. So I am very curious. Can you help me to search it?
If I'm not mistaken, it is in the subject whose ID is 5244041 and #msg54439447.

Hey Chuck, are you sure that's the correct topic ID? If I am not mistaken, that should be Wolfbet signature topic and from what I can see, that post hasn't been edited.
http://loyce.club/archive/posts/5443/54439447.html

ChuckBuck

hero member

Activity: 1372

Merit: 783

better everyday ♥

Hi LoyceV,
It's me again Cheesy

There was an edited post that mentioned me in it. I have not read it to the fullest yet. So I am very curious. Can you help me to search it?
If I'm not mistaken, it is in the subject whose ID is 5244041 and #msg54439447.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: LoyceV on April 08, 2020, 08:12:49 AM

Well, that's a first: I censored data

The credit card info sale is not good. pointed me at this post: http://loyce.club/archive/posts/5417/54178004.html
I didn't check if my webhost allows it, but censored it anyway.

Update (April 23, 2020): I received a PM (I'll keep the sender private, so sorry, no credits) about http://loyce.club/archive/posts/5428/54281329.html, which is now censored too.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: OmegaStarScream on April 27, 2020, 11:16:54 AM

I could be wrong but I think that's only valid for the posts that have both latin and arabic/hebrew alphabets.

Maybe. I currently have a few months of scraping to go, after that I'll re-scrape those boards and replace all posts in my archive by the freshly scraped ones. I still have to figure out what exactly I have to change for this.

Quote

A suggestion... Have you thought about making a browser extension? I think it would be nice if we could access the archived posts directly from the forum's posts instead of having to manually check every post.

I'll take your suggestion to [BETA] BPIP Extension - user info add-on / extension for Firefox, Chrome, et al. I'm not making any browser extensions myself.

OmegaStarScream

staff

Activity: 3402

Merit: 6065

Quote from: LoyceV on April 23, 2020, 08:10:39 AM

I found a bug (which I'm posting here as a reminder to myself): Posts on the עברי (Hebrew) board don't show up. Example: this post is missing, while it exists.
I'll see if I can add them later. I think it has something to do with the right-to-left writing, even selecting text on that board doesn't work as expected.
Update: عربية (Arabic) has the same problem.

I could be wrong but I think that's only valid for the posts that have both latin and arabic/hebrew alphabets.

A suggestion... Have you thought about making a browser extension? I think it would be nice if we could access the archived posts directly from the forum's posts instead of having to manually check every post.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: LoyceV on February 02, 2020, 02:05:42 PM

Sneak preview: http://loyce.club/archive/oldposts/
How to use:

Find the msgID you need. Let's use 28228
Remove the last 5 digits from the msgID to get the directory name (if there are less than 5 digits, use 0): 0
Replace the last 2 digits of the msgID by xx, and add .html (if there are less than 5 digits, use 0xx): 282xx.html
Add "#msg" and the msgID: #msg28228
Put everything together and go to http://loyce.club/archive/oldposts/0/282xx.html#msg28228

I found a bug (which I'm posting here as a reminder to myself): Posts on the עברי (Hebrew) board don't show up. Example: this post is missing, while it exists.
I'll see if I can add them later. I think it has something to do with the right-to-left writing, even selecting text on that board doesn't work as expected.
Update: عربية (Arabic) has the same problem.

The problem doesn't occur with my real-time post scraper. this post for instance is archived just fine.

Topic: Viewing unedited posts and deleted posts, view per post, per user or per topic - page 5. (Read 8635 times)