Ninjastic.space - BitcoinTalk Post/Address archive + API - page 24.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: TryNinja on October 01, 2020, 07:03:40 PM

Would you consider also scraping the post's title and last board id for the next posts? That's the only thing that is always missing for my archive.

I don't want to change my storage format, it might break searches that cover old and new posts. I should have done this from the start though.

Bumping my own post with a quote from another topic:

Quote from: TryNinja on October 03, 2020, 08:24:55 AM

https://bitcointalksearch.org/topic/p2pb2b-scam-5190513 returns 1.
https://bitcointalksearch.org/topic/p2pb2b-scam-5190513.0 returns 306.

Request: can you add the option to omit the post from the results? I'd like to see a list of headers only so I can quickly see who posted all this. It looks like a few users posted the same thing a couple hundred times, but it takes a lot of scrolling to check all posts.

I'm impressed by the performance! I know nothing about databases, but searching this much data in a few seconds is impressive.

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: LoyceMobile on October 01, 2020, 05:52:43 PM

If it's just 400, you can easily get them from my online post archive.

All good now. It had missed 394 posts to be exact.

Would you consider also scraping the post's title and last board id for the next posts? That's the only thing that is always missing for my archive.

TheBeardedBaby

legendary

Activity: 2184

Merit: 3134

₿uy / $ell

Quote from: TryNinja on October 01, 2020, 04:58:04 PM

Thanks. I think I fixed it.

Confirm, it's fixed now, and it looks nice. Thank you

LoyceMobile

hero member

Activity: 1643

Merit: 683

LoyceV on the road. Or couch.

Quote from: TryNinja on October 01, 2020, 04:58:04 PM

I learned a few new things about how databases work, but not before missing around ~400 posts from the last hour because of 'table locks'. This means that the bot may have missed a few notifications. I'll try to somehow scrape them later for the archive. Tongue

If it's just 400, you can easily get them from my online post archive.

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: LoyceV on September 30, 2020, 05:05:40 PM

You're right, I fixed the total count

Thank you! I just added most of the missing posts from this year. Now I have more archived posts then my current post count. (8,362 vs 8,314)

To compensate the good news, I learned a few new things about how databases work, but not before missing around ~400 posts from the last hour because of 'table locks'. This means that the bot may have missed a few notifications. I'll try to somehow scrape them later for the archive. Tongue

Quote from: TheBeardedBaby on October 01, 2020, 02:28:23 AM

Just a note,
The arrangement of the results of the address search in the mobile version of the website is OK expect for the users.

Thanks. I think I fixed it.

TheBeardedBaby

legendary

Activity: 2184

Merit: 3134

₿uy / $ell

Just a note,
The arrangement of the results of the address search in the mobile version of the website is OK expect for the users. See for yourself and search the address from my profile page.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: TryNinja on September 30, 2020, 02:41:31 PM

It looks like they are already in this file. Thanks.

You're right, I fixed the total count

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: LoyceV on September 30, 2020, 01:57:16 PM

I got you posts 54060000-55049998 (this file will self-destruct in 7 days). Does this work?

~~Could you get me a few more (+1917 to be exact)?~~

~~54060000-55051915 (You can also round it up to whatever makes it easier for you)~~

It looks like they are already in this file. Thanks.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: TryNinja on September 30, 2020, 07:43:22 AM

I think I need the posts from around March 22th until that date.

I got you posts 54060000-55069999 (this file will self-destruct in 7 days). Does this work?

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: LoyceV on September 30, 2020, 04:28:41 AM

I thought you had the more recent posts already

Nope. I only have your 'old posts' archive and the very most recent ones since the bot v2 was released around August 24th. I think I need the posts from around March 22th until that date.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

I'm taking this quote from another topic here:

Quote from: TryNinja on September 29, 2020, 11:02:36 PM

I know my database is very incomplete and many posts from this year are missing

I thought you had the more recent posts already, but if you want I can quite easily get you a compressed version of my posts archive.

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: LoyceV on September 29, 2020, 02:43:30 PM

Can you increase maximum string size in the Search ("Content") field?

Done. The limit was there because the old search engine worked very poorly with big strings. Also added a "total results" count (without having to scroll down until there are no results), since I guess that's the info you want.

But I don't think you need the html tag. IIRC, the search index stripes them, so you can just search for the full URL.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Can you increase maximum string size in the Search ("Content") field? I now can't search for this:

Quote from: Vod on September 28, 2020, 02:55:24 PM

Code:

Stalker22

legendary

Activity: 1484

Merit: 1355

I agree with TryNinja. It is better to show all the posts where the address appears than to miss important information.
Maybe, in the future, when reporting ban evaders and bounty abusers, we can put their related addresses in quotes and that way they won't show up in the search?

Quote from: TryNinja on September 17, 2020, 07:34:33 AM

Maybe what I can do is put a mark next to the posts in those boards so you can distinguish them.

This is actually a very good idea.

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: FatFork on September 16, 2020, 02:24:28 PM

How about excluding certain boards, for example: Scam Accusations and Reputation? That way it won't connect us to the scammer addresses we reported.

I prefer to keep them.

People can read the posts where you mentioned it and see if you posted the address as your own or if you are just talking about it.

Also, I can see people searching for an address and wanting to know if it was previously mentioned in a Scam Accusation, the Known Alts topic, etc... the more cases the better. You can always exclude the false-positives, but can't easily include the ones that didn't show.

Maybe what I can do is put a mark next to the posts in those boards so you can distinguish them.

Added:
- Graph of posts in the last 24 hours.
- Edited post difference (HTML). E.g: https://ninjastic.space/post/55206900 (click "check diff")

SiNeReiNZzz

legendary

Activity: 1022

Merit: 1043

αLPʜα αɴd ΩMeGa

Finely made, nice!

Really a good idea and it makes "patrols" much easier.
You can include almost all parameters in your investigations.
Most important for me are links to Twitter or Facebook accounts and payout addresses... All this is available!
The website is clearly arranged and above all really helpful...

Thank you and best Regards

SiNeReiNZzz

FatFork

legendary

Activity: 1568

Merit: 2581

Top Crypto Casino

Quote from: TryNinja on September 16, 2020, 11:33:57 AM

Quote from: PrimeNumber7 on September 16, 2020, 11:31:43 AM

I would argue that the majority of the time, if you post an address in a quote, it won’t belong to you. I would suggest that any address posted inside a quote be excluded from being displayed if it does not also appear outside of a quote. To say this another way, the person must post the address outside of a quote for it to count to being in this list.

It does exclude addresses insides quotes. The only addresses that show up there are the ones you mentioned in the body of your post.

It's much better now. But, I have another suggestion. Wink

How about excluding certain boards, for example: Scam Accusations and Reputation? That way it won't connect us to the scammer addresses we reported.

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: PrimeNumber7 on September 16, 2020, 11:31:43 AM

I would argue that the majority of the time, if you post an address in a quote, it won’t belong to you. I would suggest that any address posted inside a quote be excluded from being displayed if it does not also appear outside of a quote. To say this another way, the person must post the address outside of a quote for it to count to being in this list.

It does exclude addresses insides quotes. The only addresses that show up there are the ones you mentioned in the body of your post.

PrimeNumber7

copper member

Activity: 1610

Merit: 1898

Amazon Prime Member #7

Quote from: bitmover on September 16, 2020, 10:23:04 AM

Quote from: TryNinja on September 16, 2020, 12:09:56 AM

- Find addresses by author
What about finding every known addresses an user has posted (BTC and ETH only) by searching for their username or checking their user page? Now you can!

Beautiful update.

I liked the design.

The field "Known Addresses" is a very inaccurate name imo. I saw about 10 addresses in my profile, and none of them were mine. MOst of them quotes or addresses that were being discussed.

Maybe you could use a different name, such as "possible addresses" or "mentioned addresses"

I would argue that the majority of the time, if you post an address in a quote, it won’t belong to you. I would suggest that any address posted inside a quote be excluded from being displayed if it does not also appear outside of a quote. To say this another way, the person must post the address outside of a quote for it to count to being in this list.

bitmover

legendary

Activity: 2212

Merit: 5622

Non-custodial BTC Wallet

Quote from: TryNinja on September 16, 2020, 10:35:44 AM

The bot doesn't know that. "known" doesn't necessarily mean you own it. It's just the ones the bot has found in any of your (scraped) posts. There could be others, so those are the ones he knows about. Cheesy

I can change it to "Mentioned Addresses" to make that clear.

But the definition of "known" is something which is generally recognized. In this case addresses which are generally recognized as mine. Your bot is not recognizing those are mine, but as "mentioned". Mentioned Addresses is a lot better.

Quote

Definition of known
: generally recognized

https://www.merriam-webster.com/dictionary/known

Topic: Ninjastic.space - BitcoinTalk Post/Address archive + API - page 24. (Read 15121 times)