Topic: Monthly Report Statistics - page 4. (Read 3698 times)

Campaign managers hiring based on number of reports? I'm not sure they'd be that interested in that. Since, all they ultimately care about is the amount of posts, and the quality of posts. While, it'd be great for them to be a little more strict in certain areas when it comes to accepting users on their campaigns, I'm not sure encouraging them to count other things other than number of posts, and quality to be within their interests, and the communities.

I just wish the lower quality campaigns would step up their game a little. I don't need them to completely change, and I don't think reporting should have any effect on their campaigns.

Yep, I agree. That was bad wording on my part. I should have said unhelpful strategies.

Glad if they were worth reading


Without having the technical skills you have, from what I just read, I understand that at code level the work is not so much as I (and others?) thought it could be. This is good news! Regarding the idea to test this change on SMF, in order to see if it's worth implementing it on the new forum software -- again, it makes sense. And I also think that rPoints sound better than rMerits, as this name would make less confusion than "rMerits".

It seems you have all settled Now we need to see theymos' oppinion...

---

I'd make a mention here: as a dedicated reporter which I am, from my own point of view, I can say that there are not any low-effort strategies. I understand that a report breaking the rule related to plagiarism may be considered as way more valuable than a report for multiple posts in a row. However, if a reporter is skilled on plagiarism issues and he manages to get a few cases per month (per year?) should not be seen as God-a-like compared with a reporter which did not find any plagiarism issues but, instead, made 5000 reports of zero-value posts or of double bumping or of multiple posts in a row. The reports made by the lattest may have a lower value (taken one by one) compared with finding a case of plagiarism; yet, in order to make 5000 (good) reports of zero-value posts, this reporter would spend dozen of hours.

So we have a first reporter, which works a few hours the entire year and reports a few (good) cases of plagiarism -- all good, bravo to him! But the second one, although he made reports with lower value, worked much, much harder than the first. Does this make any sense?

Maybe reporting 50-100 zero-value posts is easy. But if someone reports 10.000 - 15.000 - 20.000 such posts, just imagine how much time it takes to do it (first of all, finding them; second of all, analyzing them into the context of the topic; then reporting them; writing a comment for each report etc.). I, for one, know how much time it takes. In some days I reported 800 posts. But, for doing that, I needed 12 hours or so, in a context when I already had prepared all those posts and I just had to hit "Report" button. If we count also the time needed for finding them and analyzing them -- you do the math

To add to PowerGlove's proposal, maybe after implementing his idea and seeing the results, we could also encourage campaign managers to pay a little more attention to the members with these badges? Not making it a requirement to join though.
Also it's a good idea if mods could decide who gets the rpoints to avoid farming. Thanks PowerGlove for your contribution. +5 merits. (have zero to send).

Yup, I get what you're saying, and I appreciate your points.

I would answer them one-by-one, but I'm not that invested in this idea (other than believing that it's a technically sound countermeasure for theymos' concerns). So, I'll address your main points (as I see them):

I don't believe that the administrative effort would be that significant. With the existing merit system, theymos seems to have taken a very hands-off approach, and it's working out pretty well, in my opinion. With the rMerit system, theymos might have to be a little more prescriptive, and write some rough guidelines for the mods to follow, but this would be (mostly) a once-off effort.

I don't believe this would be that complicated to implement. I can appreciate how my complexity estimate would seem to have little value, but I did recently finish a 2FA patch for the forum, and I can tell you without hesitation that a basic implementation of reporter badges based on rMerit would be easy by comparison.

I get what you're saying about duplicated work and Epochtalk making it difficult to justify effort being spent on SMF, but on the other hand, I think there's an opportunity to test and refine this idea on SMF before deciding whether it's worth adding to Epochtalk.

One last thought, which I didn't make clear in my original proposal: I'm not suggesting making "rMerit" a thing that shows up on your profile page, or above your avatar. It's not meant to be on an equal footing with proper merit, it's just an implementation detail. The only people that should be worried about how it works, what it means, and where it's displayed are the people who are shooting for reporter badges. If it were implemented tomorrow, most people wouldn't even notice (there are almost no user-facing changes, besides the badges themselves). The way I see it working (at least, initially) is that when you go to report a post, you might see something like this:



Seeing [rMerit: ...] would be enough of a clue for interested people to search the forum for "rMerit" and find the topic(s) describing rMerit ranks/badges and how much rMerit can be expected (typically) for different kinds of reports. Actually, as I'm explaining this, I'm realizing that calling it "rMerit" was probably a mistake and something like "rPoints" might sit better with people and lead to less confusion. Anyway, my point is that this doesn't have to be a "big" oh-my-god-this-changes-everything feature; it can be implemented in a very low-key way that only really affects the mods (and, of course, slowly affects the histogram of moderation reports, which is its whole purpose).

One last-last thought, for the people that haven't been following the discussion: An important reason that this proposal would fare better (than the current system) against potential abuse problems, is that it separates the "action" taken in response to a report from the "reward" given in response to a report. Aside from underpinning moderation reports with an adjustable incentive structure, it opens up a class of reports that could still be marked as "good" and be acted on by the mods, but that would result in no reward (0 rMerit). Presumably, most people wouldn't bother with those types of reports. Deciding what kinds of reports should go in this class (e.g. really old posts with minor rule violations) would allow theymos to stop people from trying to "grind" these badges with low-effort strategies.

Yup, I just noticed it now.

---

I understand what you mean and I agree with the principle. It's same as in software or games testing, where reporting a bug for a flickering texture has a lower severity than a crash of the application or than a bug which makes you lose the progress you had so far. However, in testing industry, differentiating the severity of each issue was implemented from beginning and all bugs reported simply followed the reporting procedure: bug type X is considered blocker; bug type Y is considered as Critical; bug type Z is considered as Major; bug type XY is considered as Minor; bug type XZ is considered as Nice to have and so on.

Basically, you are suggesting a similar system here, for classifying each type of report, based on the impact on the forum of the reported post. But the difference between us (the forum) and the system used in bugs reporting is that here no such system was designed from beginning. So, in order to have a similar system a lot of work has to be done:

- classifying each type of report. But who will do that? theymos? theymos and mods? theymos and mods and users?
- how will this classification be done? Based on Unofficial list of (official) Bitcointalk.org rules & guidelines?
- after first two steps are completed then others come up -- educating mods to learn the new report severities; educating users to correctly use the new report severities (what will happen if mods will confound report severities when analyzing reports?)
- last step and, maybe, most difficult: how will be implemented this system? By changing the code of the forum?

As you see, although the idea is good, I don't know if it's also technically possible. Thinking just a little about it and so many questions appear. If it's too difficult to do it (although it may help a lot) then is it worth doing it? (Or, at least, trying to do it?) And so on...

And, excepting all of the above, having in mind that theymos will change, at some point, the forum software, the question about the usefulness of the effort come back: is it worth working on this old forum code to implement this feature, since the forum will be renewed in the future? (And, if it will be changed then all this work may have to be done twice?)

Even though I proposed this: Revised — Incentivizing moderation reports with millimerits, I think I've come to agree with some of the people in that topic that it's not a good idea to mix merit with reporting (I still like my "rMerit" idea, though).

Merit is meant to highlight good posts (read: contributes something) and allow good posters (read: habitually adds value) to rank up. When it's used for that purpose, everything is peaches and cream; lazy, low-effort posting is attenuated, and high-effort posting is boosted. When people get soft, and start handing out merit from a place of kindness or sympathy, or for reasons that aren't correlated with post quality, then you run the risk of inadvertently helping the kind of person that would go on to lower the already-low signal/noise ratio on Bitcointalk (by them joining a campaign and posting 30 times per week, whether they have anything interesting to say, or not).

IMHO, a large percentage of the people who would jump at that offer (prioritized post history reviews in exchange for good reports) would be the kind of people that I'm talking about (i.e. posters that struggle to earn merit in the normal way, which is usually an indication that their contributions wouldn't be missed if they were to stop posting). If you feel that there are members that are being overlooked and/or getting less merit than they deserve, then use this topic (or similar): [self-moderated] Report unmerited good posts to Merit Source.

After some brain storming, I came to a conclusion, it'd be productive to announce a new change, and the change is the following:

Ask members to take part in reporting at least 200 good reports with a high accuracy rate in a month and put them on high priority to give them 100 merits for their good quality posts every month, this is two birds with one stone, at the same time they help out the community and in return the community returns the favor by checking their post history every month and giving them their 100 earned merits. You could accept at least 100 among the good reporters and set them to do the helping for a month just to receive their special treat of receiving 100 merits for their good posts and even rank up, why not reward those who are good for the forum? That way they don't need to search the forum to see who is handing out merits so they could go with their hand out just for a few already deserved merits.

I agree. I wouldn't expect, and don't want that kind of access (i.e. database access). Getting a copy of the PHP and running it locally against a nearly-empty database with the same schema would be enough to give me more reach and let me produce better patches, but my impression (to date) is that theymos is not that eager to receive/review merge requests (at least, not complicated ones, or ones that are 99% there but need a little help from him to push them over the finish line), so there's very little incentive for him to trust me with his code (in fact, it might even be something he would end up regretting, because it's likely I would ramp up my forum improvement efforts in response). The only reason I raised the issue was so that I could justify saying: "Nope, not really." to "@PowerGlove, could you help please?".

---

@Ratimov and @GazetaBitcoin both had reservations about the "rMerit" idea, so I've updated it (6 posts up) with a justification for its complexity.

Usually, developers can be given a certain type of access which allows them to only see what they need to see. However, I'm not sure Bitcointalk was ever built with that in mind. SMF, might have been, but we've deviated quite a bit from that. The way theymos was talking, I think it might be problems with abuse rather than actually implementing things though. While, implementing it could break things unexpectedly, I'd imagine the motivation is there. The only problem is; developing it in such a way it isn't abused.  

I'm not sure we've got the answers to prevent that abuse either. It would be nice to hear theymos' actual specific concerns regarding it, and see it we as a community could actually come up with the answers.

I don't see Theymos giving you the keys of the forum so you can make your modifications, but maybe it's possible to give you a test (and private) version of the current version of Bitcointalk (without the sensitive stuff (Satoshi's PMs, Staff and Donators sections, the cave of dark secrets, etc...)).
I'm not a dev, I don't know if it's hard to do, but it might be worth asking, you have nothing to risk.
Of course Theymos has the last word if he implements or not your updates.

Well I think it would be kind of pointless if it was too easy. We can make several different badges as well or maybe the same badge but different colours for the number of reports. We could look at the latest stats on reporting and the monthly stats to determine where to set it at. I think there needs to be at least some point to reporting as 99% of people just won't do it unless it's for something urgent like malware etc (or just something that personally annoys them haha).


Anything that is reported will be counted. If a bot autodeletes something with no report then it probably won't count.


Yeah, good point. Maybe the badges can be time sensitive so you need to keep up the reporting constantly every month or whatever to keep them. That would encourage reporting on a consistent basis and not just mean people spam their way to achieve it then just abandon reporting all together once they get the badge, though I think there could be badges for huge reporting milestones.


Probably. actmyname had 38580 over a 12 month period when the last table was given: https://bitcointalksearch.org/topic/m.57980000

While your idea seems very interesting, you have also to admit that it is very complex. And its complexity, together with the complexity of proper implementing it may lead to even more debates and more questions. "That way, mods can send you rMerit according to how useful your report was." -- what makes the report useful? What makes a report more useful than another? What report is useless but still leading to the reported post being deleted? And so on. Practically there should be also defined a system classifying each kind of reports (zero-vale posts, offtopic posts, spam, scam, referral links, plagiarism, double bump and so on) -- and this would lead to even more complex debates.

Still, the idea is interesting but I think it would be way more difficult to implement it in a proper way than simply create some badges.

And, speaking about badges, who do you think - seriously speaking - that would abuse somehow by reporting posts? It never happened before. I agree, this does not mean that it'll never happen in the future. However, I believe that chances are too low to take them into consideration...

Sure, I'd like to help with this. As theymos has said before, when he initially set aside time to implement reporter badges, he realized that a proper implementation would take more effort than he could justify spending at the time.

As I understand it, other than some (relatively) easy-to-handle complications (like letting members opt out, if they like) the main problem is what to do about people trying to earn the badges in ways that wouldn't be helpful to the forum (and might even be harmful). Offering status and recognition for a high number of moderation reports is likely to stimulate unnecessary reporting and inspire dishonest types to think up low-effort ways to earn these badges.

The best solution I've been able to come up with so far is to introduce a new kind of merit (rMerit, and its sendable form: srMerit) and base the earning of badges on that. That way, mods can send you rMerit according to how useful your report was. If the mods have a consistent policy on how to distribute rMerit (i.e. sending none for unhelpful reports, and being generous with reports that are truly useful) then I think that would condition people away from the behaviors that theymos would like to avoid.

There would probably have to be some kind of rMerit airdrop to reward people that have already put a lot of effort into cleaning up the forum. This would ideally be handled manually because applying an automatic exchange rate (e.g. 1 good report = 1 rMerit) is unlikely to produce fair results. Maybe the airdrop could be handled in a latent way, with mods reviewing old reports and selectively sending rMerit for them slowly over time (probably only for members that have "applied" for airdrop consideration, otherwise it would create far too much work for the mods). Of course, another way to go would be to skip the airdrop altogether and just start everyone off on a clean slate.

I don't have access to the modified version of SMF that Bitcointalk runs on, and implementing the above as a patch against SMF 1.1.19 is unlikely to produce something that theymos would merge, so my hands are a little tied. I'm happy to work on implementing the above, but (realistically) I can only do so at theymos' request.

Edit: Some people think this proposal is unnecessarily complicated, so I’ll try to justify its complexity a little: The current system is effectively binary; you get 0 or 1 "point" for every post you report. That means that someone who has reported 1000 posts with malware links is "graded" the same way as someone who has reported 1000 ancient posts for violating the "double posting" rule. One kind of activity is really valuable to the forum, the other much less so. The "rMerit" idea is about adding more granularity to the system, so that the difference in value can be exposed and the "right" kind of reports can be incentivized and rewarded properly. At first glance, this would seem to have little to do with reporter badges, but if theymos' concerns are valid, then this additional granularity turns out to be an essential ingredient (IMHO) in preventing (or, at least discouraging) people from trying to earn these badges in the lowest-effort way possible.

Thanks for keeping us updated hilariousandco. It's an honor to be part of Bitcointalk's history. In this case with numbers after reporting other users misuse of the forum.

One way or another, I am sure that such badges would represent a great incentive for many forum users. Similar to other special badges, that were created in time -- the art contest recognition badges from 10th anniversary art contest or the  Bitcoin Pizza Bitcointalk badge, which was said to be temporary, yet it's still active. If those badges could be implemented correctly, then why such badges for reporters present a risk in having them correctly implemented...? I don't understand what problems could arise... after all, other badges were implemented and no problem was generated (code-wise).


A lot of users are approaching 100.000 reports? Is this true?

---

I also agree, merits would not be a good incentive, as it may be abused. On the other hand, maybe sMerits would not be such a bad idea though... I believe that it's harder to abuse sMerits than merits... Especially if they would be awarded in small amounts. However, this would also imply, probably, a lot of work (code-wise). Perhaps easiest solution is represented by the badges.

But main question is: is theymos still willing to do it?

If someone can't be bothered to write a half-decent post (as that's all that is needed in order to get merit) I really doubt that it will want to go around and report shitposts of others. If anything, meriting reporters will only lead to abuse of the system where people will use alt accounts to write shitposts and then report them on the main one. It would be a perfect system for account farmers, and that's something that is much harder now due existing merit system.

This happens when you create a system for ranking up in a pyramidical structure form, in a long term you'll be left with a group of high ranks, decreasing traffic, number of off topic posts to report, forum needs to be fed with new blood. One way for it is by removing merit from the ranking requirement, another way is by giving merits and smerits to reporters. Personally having smerits is an incentive for me to actively report.

It is a chart for Monthly Report Statistics since 2016. Enjoy!

The vertical red lines are for 01January dates. It seems reporters expectedly to have rest times in Christmas and New year holidays.

The only thing is; if it isn't implemented correctly then the novelty will wear off, and we'll be back where we started. Currently, we don't have much motivation for reporting other than helping out. However, it does seem that a lot of users don't find that motivating enough, which again is fine. As you've talked about it, as well as I, and several others it's largely a thankless task.

So, for reporting badges to see long term success we probably need to offer badges based on total reports, as well as within a certain time period; say monthly. So, the top monthly reporters get a badge to distinguish that, as well as badges for all time reports. Implementing a leader board, has it's downsides due to abuse, but would probably motivate a lot of users.

As well, as having large enough badges that users don't achieve them all within a year or two. Since. a lot of users are now either at or approaching 100k reports.