Topic: [BPIP] Bitcointalk Public Information Project [Back in Action] - page 7. (Read 19631 times)

If it helps: loyce.club/other/tmp/ibminer/ (scheduled to be deleted in 30 days) lists all userIDs who posted on the past 29-ish days. Note that the files contain duplicates if users posted more than once.

Thanks, and no not normal. As of yesterday the recent posts job had been locked for around ~26 days.. so the job hadn't been running on the parsers, picking up those posts, and updating the status of those users. I cleared the lock so it's been running correctly now for new posts, but I may need some time to figure out a way to either re-process posts through the parsers in that timeframe, or find some way to manually update. If any users continue to post, it should at least update the status on their next post.

Does this mean BPIP doesn't scrape all new posts? If that's the case, I can easily share a list of all users that posted "today". I could provide historic data too: I have this data starting May 23, 2019 already (I needed it to keep remove unbanned users from the list of banned users).

I don't know if I have not been so observant in the past but lately there seem to be so many profiles just woken up showing as "inactive" or "archived" several minutes or hours later after posting

Like this user: https://bpip.org/Profile?id=1286231



Is it normal?

Sorry for the delayed reply. I was not aware of how many users have same Trust score... wow!

Howeever, I don't think that was the reason for the change... I will explain:

- last time I posted here it was just after the DT change, when I was dropped to DT2. At that moment, my position on those ladders registered a fluctuation of 100 places (similar to current one)
- since then, my position on those two ladders faced only minor changes
- immediately after lattest DT change, when I returned to DT1, I noticed same huge fluctuation. So I believe that, when I checked my profile on bpip, the DT change was processed already by bpip.

Having these said, I think it would be a very big coincidence to have these huge fluctuations twice already, first time after being dropped from DT1 and second one after returning to DT1... So I believe this is the cause - the change from DT1 to DT2 and from DT2 to DT1 and not the Trust score change of other users having a Trust score close to mine.

I will still keep observing this at each DT change I'll meet, but who knows when I'll go again to DT2... In any case, I will pay attention each month, after DT users are reshuffled and, when I'll go to DT2 I'll come back to confirm if same huge fluctuation occurred. My curiosity was if such a big fluctuation is normal...

I think there are 2 causes:
1. BPIP doesn't update all profiles at the same time, so theymos' DT1-update doesn't instantly reflect in the correct new order.
2. Some people have quite a lot of inclusions, including self-scratching. So their own Trust score changes significantly depending on whether or not they're on DT1.

Scrolling through Most Trusted Profiles, there are many users with the same Trust score as yours. A small change is enough to bump you up or down quite a lot.

You can check a few users for yourself: check their Trust/Merit/Activity, and see if they're in the right position in the list.

I come back to this thread for a sequel of my earlier post. I have to confirm what ibminer said above. However, I still have one question...

So as you know, theymos made another reshuffle of DT members, on March 1st. By pure chance, I took a screenshot of my bpip profile before the reshuffle and one just after. Before the reshuffle I was in DT2 and after I reached back in DT1. Also have in mind that, during this time, my Trust score has not changed, nor my positions on "Most merit" and "Most earned merit" ladders (I received some merits between those pictures, yet I remained on same positions on both merit ladders). What I am trying to say is that, excepting the fact that I went back to DT1, nothing else changed (nothing which could have any implication on "Most trusted" and "Most recognized" ladders).

So the differences are the following:

Before the reshuffle:


After the reshuffle:


As you can see, there is a huge step forward on both "Most trusted" and "Most recognized" ladders. Last month, after the reshuffle, when I went to DT2 from DT1 I also noticed a huge difference on these ladders, but the huge step was back

Anyway, my question is the following: I understand that both of these ladders are influenced by the presence in DT1, respectively DT2; however, is it actually normal such a huge difference? There is a 69 positions difference on "Most recognized" ladder and 176 on "Most trusted"... I don't know but, from my perspective, this difference seems way too big to be natural...

ibminer, please don't take me wrong -- I am not questioning your algorithm here -- I posted all these just to ask if it would be a good idea to make a double-check, to ensure that all these scores are calculated correcly

That would be silly.  At the moment, it doesn't actually do anything with the DB.

I  think the main difference between the 'inactive' and 'expired' status is that an expired flag can never become active whereas an inactive flag can become active with the correct support. Don't quote me on that though.  

On the forum, the Flag shows Expired. Since it was inactive already, nothing else changes. If it would have been Active before it expired, the change would have been more noticeable.

I also thought it is so -- meaning that the "3 more supporting users than opposing users" should be only considered if they are DT users yet your case made me wonder if it really is so. As you see, theymos' sentence does not state anywhere anything about a condition for the users to be part of DT. If it is so -- and the supporting users have to be part of DT -- then theymos' definition should be changed. On the other side though, why is this the only flag which appears as expired and not as having insufficient support? It may also be the only one which has 3 more supporting users than opposing users from all the other type 2 or type 3 flags from your profile (again, I did not count the supporting / opposing users of the other flags but, since my curiosity is increasing I think I'll do it soon ). If it actually is so, then the definition is correct and the number of users is the only one which counts, no matter if they are DT or not.

If you have some time, you can do this test and count the opposing / supporting users of your flags which have insufficient support and see if there is any which has 3 more supporting users than opposing users. If there is at least one more which has 3 more supporting users than opposing users then indeed theymos' definition refers to DT users and it should be updated, as this particular flag has 3 more supporting users than those opposing it yet it was never valid and it is also not marked as expired now. But if there is none having 3 more supporting users than opposing users then it means the users do not have to be part of DT. Does this make sense?

What I meant was that, if those flags were active at some point then they became inactive you could see this difference regarding their change of status.


From what I understand, the situation is the following:


Reading the above sentence makes me believe that the respective flag was valid (therefore active) at some point, since it has 3 more supporting users than opposing users and now it's considered as expired becuase it's been more than 3 years since it was created.

It would be interesting to count the number of those opposing and supporting the other flags, which are marked with insufficient support now. Normally, none of them should have 3 more supporting users than opposing users. But if there is any which does not respect this condition then maybe something was changed in the flag system. I wanted to count the numbers of those opposing and supporting each flag with insufficient support but I did not have enough time to do it...

There is a difference and the difference is the following:

- first of all, you can only see that you have inactive flags at your profile (so no active ones are shown, if all your flags are expired or have insufficient support)



- second difference is that, although expired flags can be suppported or opposed, they are clearly emphasized at your profile as being inactive



As a consequence, they do not count anymore.

It means the flag expired -- as in it's not valid anymore. Each type of flag has a determined lifespan. According to theymos, type 1 never expires, type 2 expires after 3 years and type 3 after 10 years. That flag was a type 2 flag, created on Feb 13th, 2020, thus it expired on Feb 13th, 2023, three years after it was created. However, it may be recreated by its author.



(Although I did not find also a quote from theymos about Newbie flags, I take LoyceV's statement for granted.)

Thanks, it works as intended now. margon3 for example links to the last change in the Trust list back in 2020.

I hope you're not updating 16,000 database entries every hour, the file only changes once a week

So yes the thought has come up for a paywall of some sort which prevents the instant generation, and should hopefully avoid over exhausting resources, and I think it might be the easiest method... but it does fall into something that would involve some devoted development time. As much as I want to see it running personally, whenever I do get time to devote to BPIP code, in good conscience I try to aim at the existing TODO list which has its own priorities.. the generator is part of the list, but many other things would be seen as a higher priority.


Thank you.   I'll have to post some updated pictures at some point.

As it stands now, the generator pulls whatever stats exist on BPIP at the time of generating the image, currently there would be ~4-6 second gap of time between each card if it were generating a bulk set of cards back-to-back, within a particular year. There are not really any options to set a time range for it though. Historical ranks/scores aren't really kept in the DB, some ranks/scores could probably be re-calculated based on a date/time filter of posts & merit, but something like 'most trusted' (which also effects most recognized) couldn't be recalculated for a specific period of time without a historical set of data for feedback, which is not stored. But even without that option, it should be a close enough gap of time between cards that yearly sets would be possible and still make sense.


I wasn't trying to be secret about it  .. just haven't felt the need to post any updates. Nobody has really inquired about them, and I wanted to ultimately show a printed version I was happy with.
I'm happier now with some of the stuff I've printed this year but I've still been tweaking little things, again, as time permits.



I guess the insinuation comment comes from the expectations I have put on myself, as I'd have expected myself to have done this back in 2020, it's not a huge amount of code we're talking about here so I'm not sure if I completely missed it or what, I'm not seeing it in the work list items where we generally put every suggestion/idea/bug/etc., so I had more questions then answers for myself. Nonetheless, the site is now reading the latesttrustlist.txt and SM has gotten it setup on the server to download the .txt automatically every hour, so the older and newer trust lists should be appearing on profile pages now moving forward.

FWIW: I didn't mean to insinuate anything Just trying to improve BPIP so I don't have to build my own Search system

Cool, I appreciate it

Hey ibminer, thanks for your reply! I fully understand the reasons behind your decision. I understand why you kept working on your project and also why free time is not your friend. What can I say, I have same situation For 1.5 years I had two full time jobs. And, of course, family duties, a baby, other chores, all which don't leave too much free time... This is life!

However, I am glad to hear that you kept secretly ( ) working on your project. I remember reading Welsh's topic back then, while I was active here for just a month (indeed, my profile was made in 2017 but I only became active in late October 2019). And I remember I was so hungry to read everything I could on the forum. By pure chance I found Welsh's topic too and I found the idea to be very appealing. However, time passed and, as I kept reading every topic which came at hand, I was just like a dry sponge absorbing information; in the end I actually forgot about that topic's existence.

Yet you brought it back to my memory! Seeing that you kept working on this matter makes me hope that at some point in time your cards will come to fruition. And the fact that you kept working is what makes to be a small chance for this to happen. What can I say? I will encourage you to keep working, when you have some free time and who knows, maybe after some years something nice will happen 

---

To say the least, now you have a better picture of our favored DT member clown Have you seen also what feedback he left on my profile? I asked him in many occasions to make the slightest proof for his dumb allegations yet he never replied to me, lol

If rate limiting is a problem (not sure if I fully understand the issue at hand, though) - have you considered small Lightning payments for access? As in: very cheap for occasional usage, but too expensive to be able to spam the system?

I think they look super cool! I'd honestly like to own the physical cards of my favorite Bitcointalk members.. Only issue is that the stats change all the time, no? Would there be certain 'snapshots' in time at which a full set of the whole forum is made (e.g. 1/1/2023 edition, 1/1/2024 edition etc.)? Otherwise it is unfair to compare them (e.g. 2023 LoyceV vs. 2020 o_e_l_e_o).

I've spent too much time on it to just stop, not just the generator itself, but the overall designs and printing of them. It's become a slight obsession of mine.

I'm not saying it is completely off the table, the generator could still likely be used in some capacity, and I'm hoping it will be there one day. But I'm not sure it can be pulled off easily on a site like BPIP the way we had originally intended it. The only other ways I might foresee it working close to what we intended would involve much more development either with account creations/logins, or some sort of walled garden which prevents a direct instant generation of a card from any user (while avoiding a queuing system that will probably be just as abused). From a coding perspective, there are more important things to devote that type of time to.

I think the design of the cards has improved a lot. Outside of unavoidable performance issues, the generator itself has technically worked for quite some time, but I was not happy with the printed versions I ended up with. I'm happier now with them, but I've only done some DIY versions with newer designs (some using custom cores), and have been on & off working on a version without an avatar. That's been more of a personal project for me getting that stuff straight, which doesn't really involve much coding and doesn't involve as much accumulated time.

For me, design and coding require different levels of attention I end up needing to devote.. I can stop design work and pick it up a month later and only spend 10-20 minutes on it, and actually get something done.. but coding I need consistent devoted time of at least an hour to get anything productive done.. which IRL, with a [more than] full time job, family, and various other activities and events I can't mention without potentially doxing myself.. time isn't easy to come by.


Interesting links, thanks. And I'd agree with you on the trolling, mods do not.   I'm sure it's all open to the subjective interpretation of it, not excessive enough, etc. *shrug*


Not ignoring you here LoyceV, but based on what you've insinuated, I need some answers first before responding accurately. Regardless, I'll be working on getting the code done to read from the latesttrustlist.txt file as soon as I can.


I have not updated the original thread in a while, but some general info. and older samples could be found here.

What exactly are these cards all about? Now I'm intrigued..