Ninjastic.space - BitcoinTalk Post/Address archive + API - page 13.

PrimeNumber7

copper member

Activity: 1652

Merit: 1901

Amazon Prime Member #7

Quote from: TryNinja on April 21, 2022, 08:48:58 AM

Quote from: PrimeNumber7 on April 21, 2022, 08:46:35 AM

Okay, fair enough. Although my suggestion that the sub a thread is located in when scraping new posts could easily be implemented.

I think this is doable, so I'll add it to my TODO list. But yeah, it would only detect topics that had recent activity.

I don't think it would be terribly difficult to implement, but it really depends on your current code base.

I am not 100% sure, but I think it is reasonable to believe that most threads that get moved will get moved not long after they are created, which means most threads that get moved will have at least one post made after they are moved. This is probably a good compromise between having a high accuracy rate of the location of a thread and the resources consumed.

Pmalek

legendary

Activity: 2730

Merit: 7065

Quote from: dkbit98 on April 21, 2022, 08:10:14 AM

I tried doing that in the first place before asking my question in forum, and even after refreshing page I was not getting the same results like before.

I assume you didn't try it on Google Chrome or Firefox. Knowing you, you probably avoid Chrome like the plague. Grin

I just tried it again to test the new feature where you can undo the ignored threads and it works and appears just fine both on Firefox and Chrome.

You use an alternative browser, forked off of Firefox right? Did you get it fixed? If not, try it with one of the regular ones if you haven't already.

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: Pmalek on April 21, 2022, 04:29:07 AM

@TryNinja if you can make modifications to the results, it would be a good idea to remove the sensitive data from those couple of posts.

Anyone can DM me which posts they think should be censored for privacy/security reasons. I have received a few requests in the past.

Quote from: dkbit98 on April 18, 2022, 09:17:45 AM

Thanks for updating website with cool new features that can improve search function a lot, but I have some questions.
Is there a way I could unignore specific topic that I ignored before with click on the button?

I added a small list with the ignored threads (so you can view which ones and even undo it) below the search params card.

Quote from: dkbit98 on April 21, 2022, 08:10:14 AM

I tried doing that in the first place before asking my question in forum, and even after refreshing page I was not getting the same results like before.
Even clicking on F5 didn't fix this so maybe all history with cookies need to be deleted in some cases.
I would like to hear what TryNinja has to say about this.

The ignored list isn't stored, so this shouldn't be the case. As soon as you refresh the page, it should be cleared.

The only thing that is stored anywhere is the color scheme you're using (light or dark) on your browser's localStorage.

Quote from: PrimeNumber7 on April 21, 2022, 08:46:35 AM

Okay, fair enough. Although my suggestion that the sub a thread is located in when scraping new posts could easily be implemented.

I think this is doable, so I'll add it to my TODO list. But yeah, it would only detect topics that had recent activity.

PrimeNumber7

copper member

Activity: 1652

Merit: 1901

Amazon Prime Member #7

Quote from: LoyceV on April 18, 2022, 02:12:18 AM

Quote from: PrimeNumber7 on April 17, 2022, 04:41:20 PM

TryNinja could theoretically add every thread to his watchlist and monitor for the thread's location being changed, although this is probably less feasible than the above.

The Watchlist gets slower when there are more topics to watch, and I've seen theymos recommend to clean it up. So I doubt this will work with 5 million topics.

Okay, fair enough. Although my suggestion that the sub a thread is located in when scraping new posts could easily be implemented.

hosseinimr93

legendary

Activity: 2380

Merit: 5213

Quote from: dkbit98 on April 21, 2022, 08:10:14 AM

I tried doing that in the first place before asking my question in forum, and even after refreshing page I was not getting the same results like before.

I also tried that to see how it works on my side.
When I refresh the page (a normal refresh, no need to a hard refresh), the result is back to normal and ignored topics are shown again.
When I do a new search"without refreshing the page, the ignored topics are still hidden.

dkbit98

legendary

Activity: 2212

Merit: 7064

Quote from: Pmalek on April 21, 2022, 04:29:07 AM

If you refresh the page, all changes will be disregarded and you will see the normal search results.

I tried doing that in the first place before asking my question in forum, and even after refreshing page I was not getting the same results like before.
Even clicking on F5 didn't fix this so maybe all history with cookies need to be deleted in some cases.
I would like to hear what TryNinja has to say about this.

Pmalek

legendary

Activity: 2730

Merit: 7065

Quote from: LoyceV on April 21, 2022, 06:13:46 AM

It's better to PM requests like this. Pointing it out here makes it more likely someone sees it, or even archives the archives.

I agree, but in this particular case, the data has already been in Meta for 2 days and originally posted in the Russian section on 11 April. Plenty of time to do "stuff" with it.

Quote from: LoyceV on April 21, 2022, 06:13:46 AM

I found 4 posts (one of them in a different topic). Did I catch all of them?

Yeah, I think you got them all. 1 in the Russian board and the 3 in Meta. Unless it was shared elsewhere as well. Thanks!

Not that it matters, but who is the guy and why was his personal data posted here? A Russian soldier?
isn't doxxing a serious violation that results in a ban or a temporary one?

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: Pmalek on April 21, 2022, 04:29:07 AM

Some sensitive data has been posted on the 17th page in this thread > https://bitcointalk.org/index.php?topic=5231446.320. theymos has removed the things that shouldn't be posted on the forum, but with scrappers such as Ninjastic.space, the original posts can still be seen.

It's better to PM requests like this. Pointing it out here makes it more likely someone sees it, or even archives the archives. Once it's gone I always post anything I've censored in my topic (for transparency).

Quote

@LoyceV everything can be seen on loyce.club as well!

For future reference: postIDs are much easier than topicIDs

I found 4 posts (one of them in a different topic). Did I catch all of them?

Pmalek

legendary

Activity: 2730

Merit: 7065

Quote from: dkbit98 on April 18, 2022, 09:17:45 AM

...as well as option to reset and remove everything back to default settings.

If you refresh the page, all changes will be disregarded and you will see the normal search results.

Some sensitive data has been posted on the 17th page in this thread > https://bitcointalk.org/index.php?topic=5231446.320. theymos has removed the things that shouldn't be posted on the forum, but with scrappers such as Ninjastic.space, the original posts can still be seen. @TryNinja if you can make modifications to the results, it would be a good idea to remove the sensitive data from those couple of posts.

@LoyceV everything can be seen on loyce.club as well!

dkbit98

legendary

Activity: 2212

Merit: 7064

Quote from: TryNinja on April 16, 2022, 10:51:48 AM

As suggested in this post, I've added a button on the search page to hide results from specific topics.

Thanks for updating website with cool new features that can improve search function a lot, but I have some questions.
Is there a way I could unignore specific topic that I ignored before with click on the button?
I did some testing and ignored few topics, but now I can't find what exactly I ignored, so it could be useful to add list of ignored topics, as well as option to reset and remove everything back to default settings.

hosseinimr93

legendary

Activity: 2380

Merit: 5213

Quote from: LoyceV on April 18, 2022, 02:12:18 AM

You can't scrape a 1 day old post without knowing the topicID already.

Even if a topic is moved to a different board, the topic ID doesn't change and you can still use the same link to see the post.

TryNinja can scrap the location of every topic one more time after a certain time to check if it has been moved. Of course, I doubt it's worth it to do that.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: PrimeNumber7 on April 17, 2022, 04:41:20 PM

TryNinja could theoretically add every thread to his watchlist and monitor for the thread's location being changed, although this is probably less feasible than the above.

The Watchlist gets slower when there are more topics to watch, and I've seen theymos recommend to clean it up. So I doubt this will work with 5 million topics.

Quote from: logfiles on April 17, 2022, 05:21:44 PM

How about scrape the location of only new posts made in the last n days or n weeks?

That's only possible for the last few seconds. You can't scrape a 1 day old post without knowing the topicID already.

logfiles

copper member

Activity: 2170

Merit: 1822

Top Crypto Casino

Quote from: hosseinimr93 on April 17, 2022, 01:26:46 PM

It's almost impossible to do that.
In order to have information about the new location of the posts, TryNinja has to scrap the location of ~~all the posts made in the forum~~ again and again forever.

How about scrape the location of only new posts made in the last n days or n weeks?

Quote from: Stalker22 on April 17, 2022, 02:16:59 PM

Yes, exactly. I do not think TryNinja will be able to do that.

As there is no log that indicates which topics have been moved and when, finding out this information would require scraping all the data over and over again. I think this would be a waste of time and resources.

There's no log that indicates that a post has been edited as well, but data is scrapped anyway to show changes in the post from the archived version. Does this mean waste of resources too?

The updating could be made weekly on the new posts, since I believe new posts that are made in wrong boards could have been moved within a week by the mods.

PrimeNumber7

copper member

Activity: 1652

Merit: 1901

Amazon Prime Member #7

Quote from: hosseinimr93 on April 17, 2022, 01:26:46 PM

Quote from: logfiles on April 17, 2022, 01:18:35 PM

The topics that were created in one board and then moved into another board do not have any updated information about the new board they have been moved to.

It's almost impossible to do that.
In order to have information about the new location of the posts, TryNinja has to scrap the location of all the posts made in the forum again and again forever.

There are two ways this could be done.

1 - When a new post is made, the post will show which section the subject thread is located in. If a new post is made in an existing thread, the location of the thread can be updated.

2 - When a thread on your watchlist is moved, it will show up on your watchlist again, even if there are no new posts. TryNinja could theoretically add every thread to his watchlist and monitor for the thread's location being changed, although this is probably less feasible than the above.

Stalker22

legendary

Activity: 1526

Merit: 1359

Quote from: hosseinimr93 on April 17, 2022, 01:26:46 PM

Quote from: logfiles on April 17, 2022, 01:18:35 PM

The topics that were created in one board and then moved into another board do not have any updated information about the new board they have been moved to.

It's almost impossible to do that.
In order to have information about the new location of the posts, TryNinja has to scrap the location of all the posts made in the forum again and again forever.

Yes, exactly. I do not think TryNinja will be able to do that.

As there is no log that indicates which topics have been moved and when, finding out this information would require scraping all the data over and over again. I think this would be a waste of time and resources.

hosseinimr93

legendary

Activity: 2380

Merit: 5213

Quote from: logfiles on April 17, 2022, 01:18:35 PM

The topics that were created in one board and then moved into another board do not have any updated information about the new board they have been moved to.

It's almost impossible to do that.
In order to have information about the new location of the posts, TryNinja has to scrap the location of all the posts made in the forum again and again forever.

logfiles

copper member

Activity: 2170

Merit: 1822

Top Crypto Casino

The topics that were created in one board and then moved into another board do not have any updated information about the new board they have been moved to.

For example, this post was originally in Bitcoin Discussion, and then it was moved to Marketplace altcoin, but when you specifically add Marketplace altcoin as one of the search options, the post won't appear in the search results. According to ninjastic space, it's as though the post is still in Bitcoin discussion.

TryNinja

legendary

Activity: 2758

Merit: 6830

As suggested in this post, I've added a button on the search page to hide results from specific topics.

For example:

1. You get 3 results for your search. Post 1 and 2 were made on topic A, while post 3 was made on topic B;
2. You click "Ignore Topic" on the post 1 or 2, ignoring their parent topic (A);
3. Both post 1 and 2 will be hidden, leaving only the post 3 in the page.

This will be the case for all searches until you refresh the page.

@Pmalek

suchmoon

legendary

Activity: 3654

Merit: 8909

https://bpip.org

Quote from: dkbit98 on February 01, 2022, 08:46:50 AM

Quote from: TryNinja on February 01, 2022, 04:00:35 AM

Looks like my certificate got caught in the middle of the last Let's Encrypt bug.

First it was BPIP website, now ninjastic.space affected by that let's encrypt bug, but in my case everything worked just fine.
Not sure why some users are affected and some not, but it could be because if specific browser settings that are not the same for everyone.

It's interesting that my issue happened before the letsencrypt revocation was supposed to start and I think the error message was different. Not sure if they had another problem earlier or it's part of the same screwup.

But yeah some browsers are handling the revocations differently. There are privacy concerns with that too (the server your browser asks about revocations will know which sites you're browsing).

dkbit98

legendary

Activity: 2212

Merit: 7064

Quote from: TryNinja on February 01, 2022, 04:00:35 AM

Looks like my certificate got caught in the middle of the last Let's Encrypt bug.

First it was BPIP website, now ninjastic.space affected by that let's encrypt bug, but in my case everything worked just fine.
Not sure why some users are affected and some not, but it could be because if specific browser settings that are not the same for everyone.

Topic: Ninjastic.space - BitcoinTalk Post/Address archive + API - page 13. (Read 16648 times)