Pages:
Author

Topic: [BPIP] Bitcointalk Public Information Project [Back in Action] - page 10. (Read 21754 times)

legendary
Activity: 1789
Merit: 2535
Goonies never say die.
The last entry in BPIP's DT Change Log is from May 20.
Long story short, one of the tables involved with the TrustLog has max'd out its identity column.. because, for some reason, the table appears to have started out with a high seed value which has caused it to max out after only ~1.3M 11.3M rows, when it should be able to handle ~2.1B rows as an int. Possibly at some point SM had to wipe this table and the seed started off high at that point, but I'm not 100% sure.

Once I feel comfortable I won't screw anything else up here by resetting these IDs, I'll need to reseed the table, which should get this flowing again.

edit: After re-counting the rows in the latest re-seeded table, I realized I miscounted digits when calculating the original table, which was actually ~11.3M rows, not 1.3M like was in this post, just updating to keep things accurate.
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Timelord2067 posted this in my topic, but it belongs here:
It looks as though BPIP hasn't updated in the last nearly a dozen days. theymos updated the DT lottery some fourteen hours ago by the looks of it and at least one of the organisers (of bpip) is now inactive if the website's Trust Log is anything to go by. 🤔
The last entry in BPIP's DT Change Log is from May 20.
legendary
Activity: 1789
Merit: 2535
Goonies never say die.
Deleted posts from mods value & rank:
To provide some follow-up on my prior post:
Looking into the counts & ranks closer here for this particular report, I was able to narrow down what was happening, but still need to fix it.

As it stands now:
a) The profile page is getting its value/count from a table/field which is being updated by parsers.
b) The report page is getting its value/count from a table/field which is not being used or updated by parsers.
c) Both pages are pulling their rank from the same table/field, which unfortunately has not been part of the stored procedure which calculates and updates ranks.

Because of this, the visible result is:
The report page for deleted posts by mods hasn't been updated at all (in terms of value/rank), for some time they've been static.
You can compare an archive from 2020 with a recent archive from the end of March and see the value & ranks haven't changed.

For the profile page, the value has been changing, as can be shown checking the archives of someone like JollyGood (current & archive from Feb 2021 and June 2022).
But the rank has remained static on the profile page as well, again due to the current table not being included in the rank calculations.

All this said, there is a chance for potential small gaps in capturing the modlog/seclog in the past, if there were parser issues or downtime occurring, which occasionally happened more in the earlier days of development then it would now, but I'd expect the count shown on the profile page is close to what it should be, based on what has been collected from the parsing since 2018.


I've some questions regarding BPIP security/moderator log. Does it shows the full list of security and modlog till date? or is there a point of date? I am checking a profile and just trying to make sure of something.
Here is what I'm seeing:
Seclog on BPIP goes back to March 16th, 2018.
Modlog on BPIP has ~1.3M saved records with an accurate date/time which ranges from August 3rd, 2018 until today. The modlog also has 177,836 records with a (default) unix timestamp from 1970. Based on the ID's of these records without a timestamp, they were all created (assumingly in order) before the range of time above. I didn't go back and look but the lack of a timestamp on these were likely related to a bug which was ultimately fixed in August of 2018, when it started capturing the date/time correctly.

Also is there a way where I can see full merit history of a profile?
Currently, you'd only be able to see:
Full sent: https://bpip.org/smerit.aspx?from=Avirunes
Full received: https://bpip.org/smerit.aspx?to=Avirunes

But it might not be a bad idea to have a view showing them merged together for a profile, which shouldn't be too difficult. I'll add this to my TODO list.
legendary
Activity: 3094
Merit: 1472
I am checking a profile and just trying to make sure of something.
If it helps: LoyceV's seclog log.

Quote
Also is there a way where I can see full merit history of a profile?
There's loyce.club/Merit/history/175302.html.

Ah LoyceV's tools. That will for sure do. Thanks a lot  Cheesy
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
I am checking a profile and just trying to make sure of something.
If it helps: LoyceV's seclog log.

Quote
Also is there a way where I can see full merit history of a profile?
There's loyce.club/Merit/history/175302.html.
legendary
Activity: 3094
Merit: 1472
I've some questions regarding BPIP security/moderator log. Does it shows the full list of security and modlog till date? or is there a point of date? I am checking a profile and just trying to make sure of something.

Also is there a way where I can see full merit history of a profile?
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
I recently had someone suggest I'd only had 33 posts deleted
I've seen BPIP overlook deleted posts from modlog, so this counter could be too low.

Quote
both loyce and ninja's sites have my post counts hundreds of posts higher than what bpip says
BPIP only counts posts deleted by Mods, I scrape all posts including the ones you delete by yourself later on. Judging by my own data, I have deleted about 2000 of my own posts (mostly bumps).

Quote
The other two sites are much closer to the count, but probably don't include those posts of mine that were deleted in the first four or five years after registration.
Correct.
legendary
Activity: 3696
Merit: 2219
💲🏎️💨🚓
Hey thanks for that. I had a feeling it was going to be a long shot.




I see a couple of people have been discussing the disparigy between bpip and other sites' information (in the above example numbers of posts deleted).  I recently had someone suggest I'd only had 33 posts deleted, but the thread was locked before I could point out both loyce and ninja's sites have my post counts hundreds of posts higher than what bpip says with the Forum's user profile page only showing the number of posts that *haven't been* deleted.

The other two sites are much closer to the count, but probably don't include those posts of mine that were deleted in the first four or five years after registration.
copper member
Activity: 2114
Merit: 1814
฿itcoin for all, All for ฿itcoin.
When were the first entries recorded by BPIP.org ?
Depends on the information you are looking for, but as far as I know. The log kept getting modified over time

The earliest records you will find are of most merited posts daily, and then Changed to Archived status, mostly because data would be displayed on the site even when Vod was still part of the project(I see some info as far as 2018)

legendary
Activity: 3696
Merit: 2219
💲🏎️💨🚓
Quote
https://bpip.org/metalog.aspx?d=2023-04-25

Quote
Meta Log Report
(2023 - 04 - 25)

When were the first entries recorded by BPIP.org ?
legendary
Activity: 1789
Merit: 2535
Goonies never say die.
~
I checked a few other profiles in the list, and for them, the data is consistent. How did MFahad go from 164 to 372?
Thanks, I'm seeing inconsistencies for many users as I spot checked, not just MFahad. Some are off by a few (Fredomago, cvm, Alex-007, etc.), some are off by larger amounts (joshy23, Adriano2010, JollyGood, etc), and a lot seem to match up correctly.

Trying to wrap my head around this one as I haven't really looked at this job (and related procedures) recently in the past, but there does appear to be a problem here which will need some troubleshooting which I can't do at the moment, I'll try to take a closer look as soon as I can. Smiley
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
I saw this post:
he is in 85th place (of 3529391 members) among the top spammers in terms of the number of deleted posts by moderators.
And indeed, bpip.org/Profile?p=MFahad shows:
Code:
Posts deleted by moderators: 372 (Rank: 85th)
However, bpip.org/Report?r=postsdeleted shows:
Code:
85 MFahad 164
I checked a few other profiles in the list, and for them, the data is consistent. How did MFahad go from 164 to 372?
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
or find some way to manually update.
If it helps: loyce.club/other/tmp/ibminer/ (scheduled to be deleted in 30 days) lists all userIDs who posted on the past 29-ish days. Note that the files contain duplicates if users posted more than once.
legendary
Activity: 1789
Merit: 2535
Goonies never say die.
~
Is it normal?
Does this mean BPIP doesn't scrape all new posts? If that's the case, I can easily share a list of all users that posted "today". I could provide historic data too:
~

Thanks, and no not normal. As of yesterday the recent posts job had been locked for around ~26 days.. so the job hadn't been running on the parsers, picking up those posts, and updating the status of those users. I cleared the lock so it's been running correctly now for new posts, but I may need some time to figure out a way to either re-process posts through the parsers in that timeframe, or find some way to manually update. If any users continue to post, it should at least update the status on their next post.

legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Is it normal?
Does this mean BPIP doesn't scrape all new posts? If that's the case, I can easily share a list of all users that posted "today". I could provide historic data too: I have this data starting May 23, 2019 already (I needed it to keep remove unbanned users from the list of banned users).
copper member
Activity: 2114
Merit: 1814
฿itcoin for all, All for ฿itcoin.
I don't know if I have not been so observant in the past but lately there seem to be so many profiles just woken up showing as "inactive" or "archived" several minutes or hours later after posting

Like this user: https://bpip.org/Profile?id=1286231



Is it normal?
legendary
Activity: 1680
Merit: 6524
Fully-fledged Merit Cycler|Spambuster'23|Pie Baker
I think there are 2 causes:
1. BPIP doesn't update all profiles at the same time, so theymos' DT1-update doesn't instantly reflect in the correct new order.
2. Some people have quite a lot of inclusions, including self-scratching. So their own Trust score changes significantly depending on whether or not they're on DT1.

Scrolling through Most Trusted Profiles, there are many users with the same Trust score as yours. A small change is enough to bump you up or down quite a lot.

Sorry for the delayed reply. I was not aware of how many users have same Trust score... wow!

Howeever, I don't think that was the reason for the change... I will explain:

- last time I posted here it was just after the DT change, when I was dropped to DT2. At that moment, my position on those ladders registered a fluctuation of 100 places (similar to current one)
- since then, my position on those two ladders faced only minor changes
- immediately after lattest DT change, when I returned to DT1, I noticed same huge fluctuation. So I believe that, when I checked my profile on bpip, the DT change was processed already by bpip.

Having these said, I think it would be a very big coincidence to have these huge fluctuations twice already, first time after being dropped from DT1 and second one after returning to DT1... So I believe this is the cause - the change from DT1 to DT2 and from DT2 to DT1 and not the Trust score change of other users having a Trust score close to mine.

I will still keep observing this at each DT change I'll meet, but who knows when I'll go again to DT2... In any case, I will pay attention each month, after DT users are reshuffled and, when I'll go to DT2 I'll come back to confirm if same huge fluctuation occurred. My curiosity was if such a big fluctuation is normal...
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
As you can see, there is a huge step forward on both "Most trusted" and "Most recognized" ladders. Last month, after the reshuffle, when I went to DT2 from DT1 I also noticed a huge difference on these ladders, but the huge step was back Smiley

Anyway, my question is the following: I understand that both of these ladders are influenced by the presence in DT1, respectively DT2; however, is it actually normal such a huge difference? There is a 69 positions difference on "Most recognized" ladder and 176 on "Most trusted"... I don't know but, from my perspective, this difference seems way too big to be natural...
I think there are 2 causes:
1. BPIP doesn't update all profiles at the same time, so theymos' DT1-update doesn't instantly reflect in the correct new order.
2. Some people have quite a lot of inclusions, including self-scratching. So their own Trust score changes significantly depending on whether or not they're on DT1.

Scrolling through Most Trusted Profiles, there are many users with the same Trust score as yours. A small change is enough to bump you up or down quite a lot.

Quote
ibminer, please don't take me wrong -- I am not questioning your algorithm here -- I posted all these just to ask if it would be a good idea to make a double-check, to ensure that all these scores are calculated correcly Smiley
You can check a few users for yourself: check their Trust/Merit/Activity, and see if they're in the right position in the list.
legendary
Activity: 1680
Merit: 6524
Fully-fledged Merit Cycler|Spambuster'23|Pie Baker
Are you taking into account the fluctuations of 1,000 other users, based on when they were last refreshed?

More importantly, in this context, the formula for most trusted doesn't care about merit, earned or not. Plus, the 'most trusted' would definitely fluctuate each month with the DT1/2 random selection process which shuffles trusted feedback around and would change the scores.

I come back to this thread for a sequel of my earlier post. I have to confirm what ibminer said above. However, I still have one question...

So as you know, theymos made another reshuffle of DT members, on March 1st. By pure chance, I took a screenshot of my bpip profile before the reshuffle and one just after. Before the reshuffle I was in DT2 and after I reached back in DT1. Also have in mind that, during this time, my Trust score has not changed, nor my positions on "Most merit" and "Most earned merit" ladders (I received some merits between those pictures, yet I remained on same positions on both merit ladders). What I am trying to say is that, excepting the fact that I went back to DT1, nothing else changed (nothing which could have any implication on "Most trusted" and "Most recognized" ladders).

So the differences are the following:

Before the reshuffle:


After the reshuffle:


As you can see, there is a huge step forward on both "Most trusted" and "Most recognized" ladders. Last month, after the reshuffle, when I went to DT2 from DT1 I also noticed a huge difference on these ladders, but the huge step was back Smiley

Anyway, my question is the following: I understand that both of these ladders are influenced by the presence in DT1, respectively DT2; however, is it actually normal such a huge difference? There is a 69 positions difference on "Most recognized" ladder and 176 on "Most trusted"... I don't know but, from my perspective, this difference seems way too big to be natural...

ibminer, please don't take me wrong -- I am not questioning your algorithm here -- I posted all these just to ask if it would be a good idea to make a double-check, to ensure that all these scores are calculated correcly Smiley
legendary
Activity: 1789
Merit: 2535
Goonies never say die.
I hope you're not updating 16,000 database entries every hour, the file only changes once a week Smiley
That would be silly. Tongue  At the moment, it doesn't actually do anything with the DB.

I  think the main difference between the 'inactive' and 'expired' status is that an expired flag can never become active whereas an inactive flag can become active with the correct support. Don't quote me on that though.  Grin
Pages:
Jump to: