Topic: Monthly Report Statistics - page 6. (Read 4324 times)

Yup, I just noticed it now.

---

I understand what you mean and I agree with the principle. It's same as in software or games testing, where reporting a bug for a flickering texture has a lower severity than a crash of the application or than a bug which makes you lose the progress you had so far. However, in testing industry, differentiating the severity of each issue was implemented from beginning and all bugs reported simply followed the reporting procedure: bug type X is considered blocker; bug type Y is considered as Critical; bug type Z is considered as Major; bug type XY is considered as Minor; bug type XZ is considered as Nice to have and so on.

Basically, you are suggesting a similar system here, for classifying each type of report, based on the impact on the forum of the reported post. But the difference between us (the forum) and the system used in bugs reporting is that here no such system was designed from beginning. So, in order to have a similar system a lot of work has to be done:

- classifying each type of report. But who will do that? theymos? theymos and mods? theymos and mods and users?
- how will this classification be done? Based on Unofficial list of (official) Bitcointalk.org rules & guidelines?
- after first two steps are completed then others come up -- educating mods to learn the new report severities; educating users to correctly use the new report severities (what will happen if mods will confound report severities when analyzing reports?)
- last step and, maybe, most difficult: how will be implemented this system? By changing the code of the forum?

As you see, although the idea is good, I don't know if it's also technically possible. Thinking just a little about it and so many questions appear. If it's too difficult to do it (although it may help a lot) then is it worth doing it? (Or, at least, trying to do it?) And so on...

And, excepting all of the above, having in mind that theymos will change, at some point, the forum software, the question about the usefulness of the effort come back: is it worth working on this old forum code to implement this feature, since the forum will be renewed in the future? (And, if it will be changed then all this work may have to be done twice?)

Even though I proposed this: Revised — Incentivizing moderation reports with millimerits, I think I've come to agree with some of the people in that topic that it's not a good idea to mix merit with reporting (I still like my "rMerit" idea, though).

Merit is meant to highlight good posts (read: contributes something) and allow good posters (read: habitually adds value) to rank up. When it's used for that purpose, everything is peaches and cream; lazy, low-effort posting is attenuated, and high-effort posting is boosted. When people get soft, and start handing out merit from a place of kindness or sympathy, or for reasons that aren't correlated with post quality, then you run the risk of inadvertently helping the kind of person that would go on to lower the already-low signal/noise ratio on Bitcointalk (by them joining a campaign and posting 30 times per week, whether they have anything interesting to say, or not).

IMHO, a large percentage of the people who would jump at that offer (prioritized post history reviews in exchange for good reports) would be the kind of people that I'm talking about (i.e. posters that struggle to earn merit in the normal way, which is usually an indication that their contributions wouldn't be missed if they were to stop posting). If you feel that there are members that are being overlooked and/or getting less merit than they deserve, then use this topic (or similar): [self-moderated] Report unmerited good posts to Merit Source.

After some brain storming, I came to a conclusion, it'd be productive to announce a new change, and the change is the following:

Ask members to take part in reporting at least 200 good reports with a high accuracy rate in a month and put them on high priority to give them 100 merits for their good quality posts every month, this is two birds with one stone, at the same time they help out the community and in return the community returns the favor by checking their post history every month and giving them their 100 earned merits. You could accept at least 100 among the good reporters and set them to do the helping for a month just to receive their special treat of receiving 100 merits for their good posts and even rank up, why not reward those who are good for the forum? That way they don't need to search the forum to see who is handing out merits so they could go with their hand out just for a few already deserved merits.

I agree. I wouldn't expect, and don't want that kind of access (i.e. database access). Getting a copy of the PHP and running it locally against a nearly-empty database with the same schema would be enough to give me more reach and let me produce better patches, but my impression (to date) is that theymos is not that eager to receive/review merge requests (at least, not complicated ones, or ones that are 99% there but need a little help from him to push them over the finish line), so there's very little incentive for him to trust me with his code (in fact, it might even be something he would end up regretting, because it's likely I would ramp up my forum improvement efforts in response). The only reason I raised the issue was so that I could justify saying: "Nope, not really." to "@PowerGlove, could you help please?".

---

@Ratimov and @GazetaBitcoin both had reservations about the "rMerit" idea, so I've updated it (6 posts up) with a justification for its complexity.

Usually, developers can be given a certain type of access which allows them to only see what they need to see. However, I'm not sure Bitcointalk was ever built with that in mind. SMF, might have been, but we've deviated quite a bit from that. The way theymos was talking, I think it might be problems with abuse rather than actually implementing things though. While, implementing it could break things unexpectedly, I'd imagine the motivation is there. The only problem is; developing it in such a way it isn't abused.  

I'm not sure we've got the answers to prevent that abuse either. It would be nice to hear theymos' actual specific concerns regarding it, and see it we as a community could actually come up with the answers.

I don't see Theymos giving you the keys of the forum so you can make your modifications, but maybe it's possible to give you a test (and private) version of the current version of Bitcointalk (without the sensitive stuff (Satoshi's PMs, Staff and Donators sections, the cave of dark secrets, etc...)).
I'm not a dev, I don't know if it's hard to do, but it might be worth asking, you have nothing to risk.
Of course Theymos has the last word if he implements or not your updates.

Well I think it would be kind of pointless if it was too easy. We can make several different badges as well or maybe the same badge but different colours for the number of reports. We could look at the latest stats on reporting and the monthly stats to determine where to set it at. I think there needs to be at least some point to reporting as 99% of people just won't do it unless it's for something urgent like malware etc (or just something that personally annoys them haha).


Anything that is reported will be counted. If a bot autodeletes something with no report then it probably won't count.


Yeah, good point. Maybe the badges can be time sensitive so you need to keep up the reporting constantly every month or whatever to keep them. That would encourage reporting on a consistent basis and not just mean people spam their way to achieve it then just abandon reporting all together once they get the badge, though I think there could be badges for huge reporting milestones.


Probably. actmyname had 38580 over a 12 month period when the last table was given: https://bitcointalksearch.org/topic/m.57980000

While your idea seems very interesting, you have also to admit that it is very complex. And its complexity, together with the complexity of proper implementing it may lead to even more debates and more questions. "That way, mods can send you rMerit according to how useful your report was." -- what makes the report useful? What makes a report more useful than another? What report is useless but still leading to the reported post being deleted? And so on. Practically there should be also defined a system classifying each kind of reports (zero-vale posts, offtopic posts, spam, scam, referral links, plagiarism, double bump and so on) -- and this would lead to even more complex debates.

Still, the idea is interesting but I think it would be way more difficult to implement it in a proper way than simply create some badges.

And, speaking about badges, who do you think - seriously speaking - that would abuse somehow by reporting posts? It never happened before. I agree, this does not mean that it'll never happen in the future. However, I believe that chances are too low to take them into consideration...

Sure, I'd like to help with this. As theymos has said before, when he initially set aside time to implement reporter badges, he realized that a proper implementation would take more effort than he could justify spending at the time.

As I understand it, other than some (relatively) easy-to-handle complications (like letting members opt out, if they like) the main problem is what to do about people trying to earn the badges in ways that wouldn't be helpful to the forum (and might even be harmful). Offering status and recognition for a high number of moderation reports is likely to stimulate unnecessary reporting and inspire dishonest types to think up low-effort ways to earn these badges.

The best solution I've been able to come up with so far is to introduce a new kind of merit (rMerit, and its sendable form: srMerit) and base the earning of badges on that. That way, mods can send you rMerit according to how useful your report was. If the mods have a consistent policy on how to distribute rMerit (i.e. sending none for unhelpful reports, and being generous with reports that are truly useful) then I think that would condition people away from the behaviors that theymos would like to avoid.

There would probably have to be some kind of rMerit airdrop to reward people that have already put a lot of effort into cleaning up the forum. This would ideally be handled manually because applying an automatic exchange rate (e.g. 1 good report = 1 rMerit) is unlikely to produce fair results. Maybe the airdrop could be handled in a latent way, with mods reviewing old reports and selectively sending rMerit for them slowly over time (probably only for members that have "applied" for airdrop consideration, otherwise it would create far too much work for the mods). Of course, another way to go would be to skip the airdrop altogether and just start everyone off on a clean slate.

I don't have access to the modified version of SMF that Bitcointalk runs on, and implementing the above as a patch against SMF 1.1.19 is unlikely to produce something that theymos would merge, so my hands are a little tied. I'm happy to work on implementing the above, but (realistically) I can only do so at theymos' request.

Edit: Some people think this proposal is unnecessarily complicated, so I’ll try to justify its complexity a little: The current system is effectively binary; you get 0 or 1 "point" for every post you report. That means that someone who has reported 1000 posts with malware links is "graded" the same way as someone who has reported 1000 ancient posts for violating the "double posting" rule. One kind of activity is really valuable to the forum, the other much less so. The "rMerit" idea is about adding more granularity to the system, so that the difference in value can be exposed and the "right" kind of reports can be incentivized and rewarded properly. At first glance, this would seem to have little to do with reporter badges, but if theymos' concerns are valid, then this additional granularity turns out to be an essential ingredient (IMHO) in preventing (or, at least discouraging) people from trying to earn these badges in the lowest-effort way possible.

Thanks for keeping us updated hilariousandco. It's an honor to be part of Bitcointalk's history. In this case with numbers after reporting other users misuse of the forum.

One way or another, I am sure that such badges would represent a great incentive for many forum users. Similar to other special badges, that were created in time -- the art contest recognition badges from 10th anniversary art contest or the  Bitcoin Pizza Bitcointalk badge, which was said to be temporary, yet it's still active. If those badges could be implemented correctly, then why such badges for reporters present a risk in having them correctly implemented...? I don't understand what problems could arise... after all, other badges were implemented and no problem was generated (code-wise).


A lot of users are approaching 100.000 reports? Is this true?

---

I also agree, merits would not be a good incentive, as it may be abused. On the other hand, maybe sMerits would not be such a bad idea though... I believe that it's harder to abuse sMerits than merits... Especially if they would be awarded in small amounts. However, this would also imply, probably, a lot of work (code-wise). Perhaps easiest solution is represented by the badges.

But main question is: is theymos still willing to do it?

If someone can't be bothered to write a half-decent post (as that's all that is needed in order to get merit) I really doubt that it will want to go around and report shitposts of others. If anything, meriting reporters will only lead to abuse of the system where people will use alt accounts to write shitposts and then report them on the main one. It would be a perfect system for account farmers, and that's something that is much harder now due existing merit system.

This happens when you create a system for ranking up in a pyramidical structure form, in a long term you'll be left with a group of high ranks, decreasing traffic, number of off topic posts to report, forum needs to be fed with new blood. One way for it is by removing merit from the ranking requirement, another way is by giving merits and smerits to reporters. Personally having smerits is an incentive for me to actively report.

It is a chart for Monthly Report Statistics since 2016. Enjoy!

The vertical red lines are for 01January dates. It seems reporters expectedly to have rest times in Christmas and New year holidays.

The only thing is; if it isn't implemented correctly then the novelty will wear off, and we'll be back where we started. Currently, we don't have much motivation for reporting other than helping out. However, it does seem that a lot of users don't find that motivating enough, which again is fine. As you've talked about it, as well as I, and several others it's largely a thankless task.

So, for reporting badges to see long term success we probably need to offer badges based on total reports, as well as within a certain time period; say monthly. So, the top monthly reporters get a badge to distinguish that, as well as badges for all time reports. Implementing a leader board, has it's downsides due to abuse, but would probably motivate a lot of users.

As well, as having large enough badges that users don't achieve them all within a year or two. Since. a lot of users are now either at or approaching 100k reports.

I guess if we get the next statistics period from Dec to Jan there will be a grow up on them again maybe.
Also i dont know if the Bot from Mitchell also counts to the reports that shown in the statistics , maybe hilariousandco can light us or me up on that.
And yes i think that the badges thing would animate more USers to report things .

I have nothing against badges, but I think that it should be taken into account that the lowest limit for obtaining them is set high enough that it cannot be reached easily, and on the other hand, that it does not demotivate users to start making reports at all. And while we wait for that to happen, maybe it would be good to publish the list of top reporters for last year and the year before last, because if badges are any kind of technical challenge, the list certainly isn't.

If what you are saying is true, that despite the number of posts going 50% down the amount of shitposts remained the same or even increased, that would mean that forum became much worse place than it was just a  year ago. I honestly haven't noticed such a big change, at least not in the parts of the forum where I am active.


I have to agree with this, situation changed dramatically in that regard and while few years ago high ranking accounts were getting permanenly banned for a sentence they plagiarized years ago (which was too harsh imho), now mods became too lenient and letting obvious offender walk around like nothing happened (good example of that is naim027 who is still roaming around).

There is a userscript coded by @mprep

• [Userscript] Unofficial Bitcointalk Reporter Badges

theymos approved SMF patch from PowerGlove and I don't know difference between userscript and SMF patch but if there is a little adjustment to convert a userscript to a SMF patch, theymos will more easily to apply it.

@PowerGlove, could you help please?

That's why I think theymos should introduce badges with a couple of small perks to make it worthwhile. Reporting is a thankless task and most get bored of doing it pretty quickly so at least offer a small incentive/thanks.