It looks like they are already in this file. Thanks. 
You're right, I fixed the total count 
Topic: Ninjastic.space - BitcoinTalk Post/Address archive + API - page 27. (Read 16648 times)
I got you posts 54060000-55049998 (this file will self-destruct in 7 days). Does this work?
It looks like they are already in this file. Thanks.
I think I need the posts from around March 22th until that date.
I got you posts 54060000-55069999 (this file will self-destruct in 7 days). Does this work? I thought you had the more recent posts already
Nope. I only have your 'old posts' archive and the very most recent ones since the bot v2 was released around August 24th. I think I need the posts from around March 22th until that date.
I'm taking this quote from another topic here:
I know my database is very incomplete and many posts from this year are missing
I thought you had the more recent posts already, but if you want I can quite easily get you a compressed version of my posts archive. Can you increase maximum string size in the Search ("Content") field?
Done. The limit was there because the old search engine worked very poorly with big strings. Also added a "total results" count (without having to scroll down until there are no results), since I guess that's the info you want.But I don't think you need the html tag. IIRC, the search index stripes them, so you can just search for the full URL.
Can you increase maximum string size in the Search ("Content") field? I now can't search for this:
I agree with TryNinja. It is better to show all the posts where the address appears than to miss important information.
Maybe, in the future, when reporting ban evaders and bounty abusers, we can put their related addresses in quotes and that way they won't show up in the search?
This is actually a very good idea.
Maybe, in the future, when reporting ban evaders and bounty abusers, we can put their related addresses in quotes and that way they won't show up in the search?
Maybe what I can do is put a mark next to the posts in those boards so you can distinguish them.
This is actually a very good idea.
How about excluding certain boards, for example: Scam Accusations and Reputation? That way it won't connect us to the scammer addresses we reported.
I prefer to keep them.People can read the posts where you mentioned it and see if you posted the address as your own or if you are just talking about it.
Also, I can see people searching for an address and wanting to know if it was previously mentioned in a Scam Accusation, the Known Alts topic, etc... the more cases the better. You can always exclude the false-positives, but can't easily include the ones that didn't show.
Maybe what I can do is put a mark next to the posts in those boards so you can distinguish them.
Added:
- Graph of posts in the last 24 hours.
- Edited post difference (HTML). E.g: https://ninjastic.space/post/55206900 (click "check diff")
Finely made, nice!
Really a good idea and it makes "patrols" much easier.
You can include almost all parameters in your investigations.
Most important for me are links to Twitter or Facebook accounts and payout addresses... All this is available!
The website is clearly arranged and above all really helpful...
Thank you and best Regards
SiNeReiNZzz
Really a good idea and it makes "patrols" much easier.
You can include almost all parameters in your investigations.
Most important for me are links to Twitter or Facebook accounts and payout addresses... All this is available!
The website is clearly arranged and above all really helpful...
Thank you and best Regards
SiNeReiNZzz
I would argue that the majority of the time, if you post an address in a quote, it won’t belong to you. I would suggest that any address posted inside a quote be excluded from being displayed if it does not also appear outside of a quote. To say this another way, the person must post the address outside of a quote for it to count to being in this list.
It does exclude addresses insides quotes. The only addresses that show up there are the ones you mentioned in the body of your post.It's much better now. But, I have another suggestion.
How about excluding certain boards, for example: Scam Accusations and Reputation? That way it won't connect us to the scammer addresses we reported.
I would argue that the majority of the time, if you post an address in a quote, it won’t belong to you. I would suggest that any address posted inside a quote be excluded from being displayed if it does not also appear outside of a quote. To say this another way, the person must post the address outside of a quote for it to count to being in this list.
It does exclude addresses insides quotes. The only addresses that show up there are the ones you mentioned in the body of your post. 
- Find addresses by author
What about finding every known addresses an user has posted (BTC and ETH only) by searching for their username or checking their user page? Now you can!
What about finding every known addresses an user has posted (BTC and ETH only) by searching for their username or checking their user page? Now you can!
Beautiful update.
I liked the design.
The field "Known Addresses" is a very inaccurate name imo. I saw about 10 addresses in my profile, and none of them were mine. MOst of them quotes or addresses that were being discussed.
Maybe you could use a different name, such as "possible addresses" or "mentioned addresses"
The bot doesn't know that. "known" doesn't necessarily mean you own it. It's just the ones the bot has found in any of your (scraped) posts. There could be others, so those are the ones he knows about.
I can change it to "Mentioned Addresses" to make that clear.
I can change it to "Mentioned Addresses" to make that clear.
But the definition of "known" is something which is generally recognized. In this case addresses which are generally recognized as mine. Your bot is not recognizing those are mine, but as "mentioned". Mentioned Addresses is a lot better.
Quote
Definition of known
: generally recognized
https://www.merriam-webster.com/dictionary/known : generally recognized
An option not to include child board of selected board would make it better
Noted!Some suggestions for finding addresses by author:
- When listing ETH addresses, it also displays ETH transactions (TXIDs), perhaps the results should be filtered by string length
Hmm.. I don't know. I could technically use a blockchain's API for that. Will see. - When listing ETH addresses, it also displays ETH transactions (TXIDs), perhaps the results should be filtered by string length
- You should ignore the results that are within the quote tags (if possible)
It's on my TODO list.- Address grouping should not be case sensitive
I didn't think of that. But I'll leave it that way for now. At least BTC addresses are case sensitive, so it shouldn't be an issue.The field "Known Addresses" is a very inaccurate name imo. I saw about 10 addresses in my profile, and none of them were mine. MOst of them quotes or addresses that were being discussed.
The bot doesn't know that. "known" doesn't necessarily mean you own it. It's just the ones the bot has found in any of your (scraped) posts. There could be others, so those are the ones he knows about. I can change it to "Mentioned Addresses" to make that clear.
- Find addresses by author
What about finding every known addresses an user has posted (BTC and ETH only) by searching for their username or checking their user page? Now you can!
What about finding every known addresses an user has posted (BTC and ETH only) by searching for their username or checking their user page? Now you can!
Beautiful update.
I liked the design.
The field "Known Addresses" is a very inaccurate name imo. I saw about 10 addresses in my profile, and none of them were mine. MOst of them quotes or addresses that were being discussed.
Maybe you could use a different name, such as "possible addresses" or "mentioned addresses"
- Find addresses by author
What about finding every known addresses an user has posted (BTC and ETH only) by searching for their username or checking their user page? Now you can!
What about finding every known addresses an user has posted (BTC and ETH only) by searching for their username or checking their user page? Now you can!
This is a pretty nice update! I haven't explored everything yet.
Some suggestions for finding addresses by author:
- When listing ETH addresses, it also displays ETH transactions (TXIDs), perhaps the results should be filtered by string length
- You should ignore the results that are within the quote tags (if possible)
- Address grouping should not be case sensitive
Otherwise, great job!
Finally, the new update!
- Board filter
You can now search posts by board. It includes their childrens, so you can select "Wallet software" to search on "Armory", "Electrum", "Hardware Wallet", etc... or just go right to the child board to search only the "Electrum" or "Bounties (Altcoins)" board. Caveat: a very small number of posts has an unknown board. So, if you search for a board, there is a small chance of missing a few posts.
- Archive updated
The archive went from ~42m posts without a title or exact date to just a couple thousands without that data! This means that you can now search by date range without getting a bunch of random posts that shouldn't be there (or missing most of them).
- New search backend
Searching is now a LOT better. It's faster, more accurate, returns more data andshouldn't doesn't crash my server!
- User stats page
You can now visit the new page (http://ninjastic.space/user/TryNinja) to check some data about an user: most active boards, graph of posts made in the last 7 and 30 days and his known addresses! New data will come later (with your suggestions).
- Find addresses by author
What about finding every known addresses an user has posted (BTC and ETH only) by searching for their username or checking their user page? Now you can!
- Deleted post in the post edit history
The bot will detect if a post was deleted less than 5 minutes after it was made/scraped and will mark it as so in the "post edit history" card.
Tip: CTRL + F5 in the website if you don't see the changes. Tell me what do you think about it.
- Board filter
You can now search posts by board. It includes their childrens, so you can select "Wallet software" to search on "Armory", "Electrum", "Hardware Wallet", etc... or just go right to the child board to search only the "Electrum" or "Bounties (Altcoins)" board. Caveat: a very small number of posts has an unknown board. So, if you search for a board, there is a small chance of missing a few posts.
- Archive updated
The archive went from ~42m posts without a title or exact date to just a couple thousands without that data! This means that you can now search by date range without getting a bunch of random posts that shouldn't be there (or missing most of them).
- New search backend
Searching is now a LOT better. It's faster, more accurate, returns more data and
- User stats page
You can now visit the new page (http://ninjastic.space/user/TryNinja) to check some data about an user: most active boards, graph of posts made in the last 7 and 30 days and his known addresses! New data will come later (with your suggestions).
- Find addresses by author
What about finding every known addresses an user has posted (BTC and ETH only) by searching for their username or checking their user page? Now you can!
- Deleted post in the post edit history
The bot will detect if a post was deleted less than 5 minutes after it was made/scraped and will mark it as so in the "post edit history" card.
Tip: CTRL + F5 in the website if you don't see the changes. Tell me what do you think about it.
Here are the timestamps:
https://bpip.org/timestamps_202009130840.zip (~160MB compressed, ~1.3GB uncompressed)
As I mentioned earlier, it's in UTC, 24h format. Here's a sample:
https://bpip.org/timestamps_202009130840.zip (~160MB compressed, ~1.3GB uncompressed)
As I mentioned earlier, it's in UTC, 24h format. Here's a sample:
Code:
28,2009-11-22 18:04:28
29,2009-11-22 18:31:44
30,2009-11-22 18:32:00
31,2009-11-22 18:34:21
32,2009-11-22 18:35:15
33,2009-11-25 18:15:57
34,2009-11-25 18:17:23
36,2009-11-27 17:17:22
37,2009-11-27 17:27:09
38,2009-11-27 22:48:39
40,2009-12-09 05:34:46
41,2009-12-09 18:45:10
42,2009-12-09 19:25:31
43,2009-12-10 13:13:51
44,2009-12-10 14:00:17
45,2009-12-10 19:31:49
46,2009-12-10 20:49:02
47,2009-12-11 04:59:19
48,2009-12-11 17:20:29
49,2009-12-11 17:58:57
50,2009-12-11 19:27:55
51,2009-12-12 06:34:21
52,2009-12-12 13:08:17
53,2009-12-12 14:11:37
54,2009-12-12 17:52:44
55,2009-12-12 18:17:10
56,2009-12-12 18:23:59
57,2009-12-12 18:47:45
58,2009-12-12 20:46:14
59,2009-12-13 06:44:04
60,2009-12-13 06:46:30
61,2009-12-13 06:50:05
62,2009-12-13 16:51:25
63,2009-12-14 09:29:44
64,2009-12-14 13:09:48
65,2009-12-14 14:46:37
66,2009-12-14 15:01:39
67,2009-12-14 17:15:56
68,2009-12-15 05:21:09
69,2009-12-15 05:30:53
70,2009-12-15 20:37:32
71,2009-12-15 21:14:13
72,2009-12-16 15:49:23
73,2009-12-16 22:45:36
74,2009-12-17 11:36:36
75,2009-12-17 13:21:49
76,2009-12-17 13:23:43
77,2009-12-17 18:38:06
78,2009-12-18 15:11:53
79,2009-12-18 17:37:48
81,2009-12-30 01:40:50
82,2009-12-30 15:28:04
83,2010-01-01 18:09:58
84,2010-01-05 01:20:06
85,2010-01-05 20:00:46
86,2010-01-07 06:14:17
87,2010-01-11 16:13:20
88,2010-01-12 19:31:22
90,2010-01-13 04:13:37
91,2010-01-13 06:24:54
92,2010-01-13 07:45:57
93,2010-01-13 08:22:56
94,2010-01-13 17:12:16
95,2010-01-13 17:44:57
96,2010-01-13 19:08:19
97,2010-01-14 20:17:20
100,2010-01-15 09:42:18
101,2010-01-16 10:39:25
102,2010-01-16 14:27:15
103,2010-01-16 23:16:56
104,2010-01-16 23:22:55
105,2010-01-16 23:44:35
106,2010-01-16 23:45:22
107,2010-01-17 10:34:44
108,2010-01-17 22:55:31
109,2010-01-18 04:49:43
110,2010-01-18 11:06:35
111,2010-01-19 08:06:15
112,2010-01-20 20:07:15
113,2010-01-20 22:05:28
114,2010-01-21 17:30:39
115,2010-01-22 10:37:09
116,2010-01-24 02:48:37
117,2010-01-24 08:27:13
118,2010-01-24 09:52:48
119,2010-01-24 20:49:47
120,2010-01-24 20:52:59
121,2010-01-24 20:53:36
122,2010-01-24 22:48:30
123,2010-01-24 23:31:48
124,2010-01-25 02:42:13
125,2010-01-25 03:11:01
126,2010-01-25 04:06:03
127,2010-01-25 05:13:23
128,2010-01-25 05:28:39
129,2010-01-25 07:40:02
130,2010-01-25 16:30:50
131,2010-01-25 17:36:10
132,2010-01-25 19:25:29
29,2009-11-22 18:31:44
30,2009-11-22 18:32:00
31,2009-11-22 18:34:21
32,2009-11-22 18:35:15
33,2009-11-25 18:15:57
34,2009-11-25 18:17:23
36,2009-11-27 17:17:22
37,2009-11-27 17:27:09
38,2009-11-27 22:48:39
40,2009-12-09 05:34:46
41,2009-12-09 18:45:10
42,2009-12-09 19:25:31
43,2009-12-10 13:13:51
44,2009-12-10 14:00:17
45,2009-12-10 19:31:49
46,2009-12-10 20:49:02
47,2009-12-11 04:59:19
48,2009-12-11 17:20:29
49,2009-12-11 17:58:57
50,2009-12-11 19:27:55
51,2009-12-12 06:34:21
52,2009-12-12 13:08:17
53,2009-12-12 14:11:37
54,2009-12-12 17:52:44
55,2009-12-12 18:17:10
56,2009-12-12 18:23:59
57,2009-12-12 18:47:45
58,2009-12-12 20:46:14
59,2009-12-13 06:44:04
60,2009-12-13 06:46:30
61,2009-12-13 06:50:05
62,2009-12-13 16:51:25
63,2009-12-14 09:29:44
64,2009-12-14 13:09:48
65,2009-12-14 14:46:37
66,2009-12-14 15:01:39
67,2009-12-14 17:15:56
68,2009-12-15 05:21:09
69,2009-12-15 05:30:53
70,2009-12-15 20:37:32
71,2009-12-15 21:14:13
72,2009-12-16 15:49:23
73,2009-12-16 22:45:36
74,2009-12-17 11:36:36
75,2009-12-17 13:21:49
76,2009-12-17 13:23:43
77,2009-12-17 18:38:06
78,2009-12-18 15:11:53
79,2009-12-18 17:37:48
81,2009-12-30 01:40:50
82,2009-12-30 15:28:04
83,2010-01-01 18:09:58
84,2010-01-05 01:20:06
85,2010-01-05 20:00:46
86,2010-01-07 06:14:17
87,2010-01-11 16:13:20
88,2010-01-12 19:31:22
90,2010-01-13 04:13:37
91,2010-01-13 06:24:54
92,2010-01-13 07:45:57
93,2010-01-13 08:22:56
94,2010-01-13 17:12:16
95,2010-01-13 17:44:57
96,2010-01-13 19:08:19
97,2010-01-14 20:17:20
100,2010-01-15 09:42:18
101,2010-01-16 10:39:25
102,2010-01-16 14:27:15
103,2010-01-16 23:16:56
104,2010-01-16 23:22:55
105,2010-01-16 23:44:35
106,2010-01-16 23:45:22
107,2010-01-17 10:34:44
108,2010-01-17 22:55:31
109,2010-01-18 04:49:43
110,2010-01-18 11:06:35
111,2010-01-19 08:06:15
112,2010-01-20 20:07:15
113,2010-01-20 22:05:28
114,2010-01-21 17:30:39
115,2010-01-22 10:37:09
116,2010-01-24 02:48:37
117,2010-01-24 08:27:13
118,2010-01-24 09:52:48
119,2010-01-24 20:49:47
120,2010-01-24 20:52:59
121,2010-01-24 20:53:36
122,2010-01-24 22:48:30
123,2010-01-24 23:31:48
124,2010-01-25 02:42:13
125,2010-01-25 03:11:01
126,2010-01-25 04:06:03
127,2010-01-25 05:13:23
128,2010-01-25 05:28:39
129,2010-01-25 07:40:02
130,2010-01-25 16:30:50
131,2010-01-25 17:36:10
132,2010-01-25 19:25:29
There is a much better way to scrape recent posts without hitting that page every few seconds.
I have a feeling Theymos may have to eventually restrict parsing if everyone keeps doing it, so I'm putting an alternative up on github.
I have a feeling Theymos may have to eventually restrict parsing if everyone keeps doing it, so I'm putting an alternative up on github.
Jump to: