Ninjastic.space - BitcoinTalk Post/Address archive + API - page 19.

Csmiami

copper member

Activity: 1582

Merit: 1319

I'm sometimes known as "miniadmin"

Quote from: fillippone on December 31, 2020, 08:30:15 AM

---

Did you also select the hours when applying the filter? I'm asking because this has been the reason I couldn't use it sometimes...

I did a quick search of that topic, and in the dates you mention, there's been 64 users posting.
Leaving you the link with the date parameters here --> https://ninjastic.space/search?after_date=2020-01-01T00%3A00%3A00.310Z&before_date=2020-12-31T23%3A59%3A59.310Z&topic_id=313900

And sorry if I'm getting in the middle of something, I know I'm not tryninja, but as I've faced the same problem a couple of times, I thought this could be the same situation

Edit:
First post: 53496705 by duesoldi on 2020-01-01 20:52:41 UTC
Last post: 55983122 by Amph on 2020-12-31 10:51:01 UTC

fillippone

legendary

Activity: 2114

Merit: 15144

Fully fledged Merit Cycler - Golden Feather 22-23

Hello @TryNinja.
As I am working on a meta post for the Italian section I wanted to try to find how many users wrote on the BitcoinPump! thread in 2020.
Well, it turns out I cannot set the date range from 01-01-2020 to 31-12-2020, as the date filters are ignored when searching.
Is this a bug or am I missing something?

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: FatFork on December 22, 2020, 07:06:31 PM

Yeah, and it works well, but can we search for full phrases and not just individual words?
Like when we use quotation marks on google and other search engines to wrap multiple words in one phrase to find the exact results.

You can now make complex searches. The hightlight option is disabled for now.

Quote from: Rizzrack on December 22, 2020, 03:41:13 AM

If the username has a space it thinks there are multiple keywords and may display hundreds of usernames

This should be fixed now.

Let me know if you find any issues.

hosseinimr93

legendary

Activity: 2380

Merit: 5213

Quote from: FatFork on December 22, 2020, 07:06:31 PM

Like when we use quotation marks on google and other search engines to wrap multiple words in one phrase to find the exact results.

That's planned and will be possible in future.
Click here to read the post already made by TryNinja.

FatFork

legendary

Activity: 1568

Merit: 2581

Top Crypto Casino

Quote from: TryNinja on December 22, 2020, 10:00:11 AM

Quote from: dkbit98 on December 22, 2020, 08:47:37 AM

TryNinja is it possible to use Ninjastic.space website to search for any term or any word without choosing specific member, post or topic but search all?

You can fill the content field and leave everything else empty: https://ninjastic.space/search

Yeah, and it works well, but can we search for full phrases and not just individual words?
Like when we use quotation marks on google and other search engines to wrap multiple words in one phrase to find the exact results.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: BlackHatCoiner on December 22, 2020, 11:28:05 AM

I searched #55387199 on Loyce's club, but it returns me 404 error. This post never existed.

It could have been made on a non-public board (such as Investigations or Mods-only).

BlackHatCoiner

legendary

Activity: 1512

Merit: 7340

Farewell, Leo

Quote from: TryNinja on December 22, 2020, 10:00:11 AM

Good catch. Is there is any chance you posted this, deleted it and posted again?

I highly doubt. Even if I did, the post id shouldn't be 55387194 but bigger than 55387199, so it's confirmed that I didn't.

I searched #55387199 on Loyce's club, but it returns me 404 error. This post never existed.

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: Rizzrack on December 22, 2020, 03:41:13 AM

@TryNinja I have a small request
On page https://ninjastic.space/search can you make the "author" field set to exact matches or add a dropdown to input uid
If the username has a space it thinks there are multiple keywords and may display hundreds of usernames
Otherwise very nice Cheesy

That was already fixed in the refactor I'm ~~being too lazy to finish~~ doing. I will push the fix to the current prod build. Thanks!

Quote from: BlackHatCoiner on December 22, 2020, 05:45:34 AM

Looking the link's location on ninjastic.space, the script has read something wrong. It's not the #55387199, but the #55387194.

Good catch. Is there is any chance you posted this, deleted it and posted again?

Quote from: dkbit98 on December 22, 2020, 08:47:37 AM

TryNinja is it possible to use Ninjastic.space website to search for any term or any word without choosing specific member, post or topic but search all?

You can fill the content field and leave everything else empty: https://ninjastic.space/search

dkbit98

legendary

Activity: 2212

Merit: 7064

Cashback 15%

TryNinja is it possible to use Ninjastic.space website to search for any term or any word without choosing specific member, post or topic but search all?

I noticed that forum search or google search are not perfect and they sometimes miss to show all correct results.

BlackHatCoiner

legendary

Activity: 1512

Merit: 7340

Farewell, Leo

I would like to report a bug. On my ninjastic profile page, on the "Deleted Posts", something must have gone wrong. Well, for some posts it's true, I deleted them and some others got removed by a moderator, but some others are not deleted but are written on that list.

For example:

The post is right here: https://bitcointalksearch.org/topic/m.55387194

Looking the link's location on ninjastic.space, the script has read something wrong. It's not the #55387199, but the #55387194.

Rizzrack

copper member

Activity: 764

Merit: 694

Defend Bitcoin and its PoW: bitcoincleanup.com

@TryNinja I have a small request
On page https://ninjastic.space/search can you make the "author" field set to exact matches or add a dropdown to input uid
If the username has a space it thinks there are multiple keywords and may display hundreds of usernames
Otherwise very nice Cheesy

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: BlackHatCoiner on December 22, 2020, 03:11:29 AM

The design it awesome, was this a template or did our Ninja write it by himself? I suck on front-end, so I don't know if was that hard.

Thanks. I made it myself with the help of the Ant Design library. I also suck on front-end, but I Try. Cheesy

Quote from: BlackHatCoiner on December 22, 2020, 03:11:29 AM

I don't understand, though, how you guys scrape all this giant information. @TryNinja, did you wrote a script that goes on every single post and saves it on a database?

Old posts = courtesy of our favorite ~~bot~~ human, LoyceV. I did write a script to scrape all of his posts.
New posts = scraped from the recent posts page.
Merits = scraped from the recent merits page.

Quote from: BlackHatCoiner on December 22, 2020, 03:11:29 AM

Once I search for my account does it search among all these millions posts?

Yes. Everything is stored in a PostgreSQL database and indexed for searching in a Elastic Search database (life saver).

Quote from: BlackHatCoiner on December 22, 2020, 03:11:29 AM

Is this ran on a VPS?

Yes. Contabo's "VPS L SSD".

Quote from: BlackHatCoiner on December 22, 2020, 03:11:29 AM

Is it expensive? I'm curious, because I get my account's result within a second.

The VPS costs ~€15 ($18) per month. It's not that powerful, but it does have 30 GB of RAM to help with the searching.

You should have seen how slow it was on the first version when the search was done through the PostgreSQL database. I'm talking about 3+ minutes, terrible results and some server crashing. Grin

BlackHatCoiner

legendary

Activity: 1512

Merit: 7340

Farewell, Leo

Wow, that's great. I guess that's a huge database, similar with Loyce's club. The design it awesome, was this a template or did our Ninja write it by himself? I suck on front-end, so I don't know if was that hard.

I don't understand, though, how you guys scrape all this giant information. @TryNinja, did you wrote a script that goes on every single post and saves it on a database? Once I search for my account does it search among all these millions posts? Is this ran on a VPS? ~~Is it expensive?~~ I'm curious, because I get my account's result within a second.

Edit: (Nevermind, just saw the 18$/month, that must be pretty worthy)

suchmoon

legendary

Activity: 3654

Merit: 8909

https://bpip.org

Quote from: hosseinimr93 on December 21, 2020, 05:10:26 PM

Quote from: LoyceV on December 21, 2020, 03:40:27 PM

Do you have any idea what happened to post 55860583? My version doesn't make sense.

According to modlog, that post had been made by jorenabanu19.

Your scrapper failed to scrap the user ID of the poster, probably because it was deleted so fast.

It was a copy-pasta post that had a bunch of those "watch live something or other" spam links, possibly edited in after the post was created.

hosseinimr93

legendary

Activity: 2380

Merit: 5213

Quote from: LoyceV on December 21, 2020, 03:40:27 PM

Do you have any idea what happened to post 55860583? My version doesn't make sense.

According to modlog, that post had been made by jorenabanu19.

Your scrapper failed to scrap the user ID of the poster, probably because it was deleted so fast.

Csmiami

copper member

Activity: 1582

Merit: 1319

I'm sometimes known as "miniadmin"

Quote from: LoyceV on December 21, 2020, 03:52:01 PM

Quote from: Csmiami on December 21, 2020, 03:50:13 PM

IIRC deleted accounts show as Anonymous.

Do accounts get deleted?

I read something that in the past you could request your account to be deleted, it amy take a while before I can get to search the reference tho, many IRL things going on before thursday....

Quote

Why do you think your version is wrong? that's exactly what I see when I search for the actual post, made on 2011: https://bitcointalksearch.org/topic/build-a-bitcoin-app-for-the-tonido-app-store-and-win-an-ipod-touch-3110

That's post 43455, post 55860583 was deleted.

I think that means enough studying for today.... I don't even read simple numbers

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: Csmiami on December 21, 2020, 03:50:13 PM

IIRC deleted accounts show as Anonymous.

Do accounts get deleted?

Quote from: ?? on ??

When was this scraped?

Last Friday. My guess would be it was a spambot copying an old post.

Quote

Maybe it's possible that theymos anonymized the user recently and somehow the post got a new id (long shot)?

Nope. That post has always been anonymous.

TryNinja

legendary

Activity: 2758

Merit: 6830

Quote from: LoyceV on December 21, 2020, 03:46:56 PM

I clearly failed to check for this, but as far as I know anonymous posts aren't possible. I didn't save the patrol snapshot so I can't check what was really there.

Oh, wait. When was this scraped? I just noticed that's the post ID 55860583 and the link/content leads to post ID 43455 from 2011.

Maybe it's possible that theymos anonymized the user recently and somehow the post got a new id (long shot)? I could see this happening due to the recent Ledger leak... But that probably doesn't make any sense. Cheesy

Csmiami

copper member

Activity: 1582

Merit: 1319

I'm sometimes known as "miniadmin"

Quote from: LoyceV on December 21, 2020, 03:46:56 PM

----

IIRC deleted accounts show as Anonymous. Why do you think your version is wrong? that's exactly what I see when I search for the actual post, made on 2011: https://bitcointalksearch.org/topic/build-a-bitcoin-app-for-the-tonido-app-store-and-win-an-ipod-touch-3110

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: TryNinja on December 21, 2020, 03:45:19 PM

My script only saved the posts which had an author (with a correct UID), so all Anonymous and Guest posts are missing.

I clearly failed to check for this, but as far as I know anonymous posts aren't possible. I didn't save the patrol snapshot so I can't check what was really there.

Topic: Ninjastic.space - BitcoinTalk Post/Address archive + API - page 19. (Read 15134 times)