If it's nothing major and because of some glitch, just ignore
Topic: [BPIP] Bitcointalk Public Information Project [Back in Action] - page 21. (Read 19683 times)
September 21, 2020, 11:10:37 PM
Hey just wanted to report that the page is not showing the rank, for the number of earned merits, activity, Feedback Trust and Recognition. I see that is was last parsed 6 mins ago, so I presume this is some kind of an error.It does not take me to the full list when I click on 0, but it takes me to the homepage instead.
If it's nothing major and because of some glitch, just ignore
If it's nothing major and because of some glitch, just ignore
September 17, 2020, 07:31:15 PM
This user woke up and was last active recently but his profile is still marked as inactive. Could it be a simple mistake?
Whenever you need data from an account on BPIP, you can simply click on "Refresh Profile" button at top of the page and see the updated profile. 
I just clicked on "Refresh Profile" button. That profile is now updated.
BPIP
September 17, 2020, 06:59:21 PM
This user woke up and was last active recently but his profile is still marked as inactive. Could it be a simple mistake?
Inactive users are not updated as often as active ones... so it takes a while for BPIP to "notice" the change in the last active date since there isn't any notification about it and we can't re-parse all users every day. However if the user makes a post or sends a merit we would update their profile within minutes.
https://bpip.org/Profile?id=1307188
Quote
Last Parsed: 18 days (Planned for 11 days from now)
September 17, 2020, 06:30:37 PM
September 17, 2020, 04:51:22 PM
You make me feel so young.
Hmmm, didn't you say that right before crawling into bed naked with a young girl???
Yep. :/
September 16, 2020, 07:14:14 AM
You make me feel so young.
After the physical and emotional stress I've endured so far in 2020, I'll estimate I will have aged 15+ years by 2021. lol
So what are the odds the active members drop is related to COVID-19?... or, maybe just someone's account farm went offline or something? Where's the poll?? lol
.. BPIP should really get some historical tracking going with those stats ..
After the physical and emotional stress I've endured so far in 2020, I'll estimate I will have aged 15+ years by 2021. lol
So what are the odds the active members drop is related to COVID-19?... or, maybe just someone's account farm went offline or something? Where's the poll?? lol
.. BPIP should really get some historical tracking going with those stats ..
September 15, 2020, 10:13:37 PM
Active members was hovering above 90k for a few days, but it just dropped below. :/
When I was your age, we didn't have these fancy websites to warn us things were failing. Sometimes they'd fail right when we were using 'em...
When I was your age, we didn't have these fancy websites to warn us things were failing. Sometimes they'd fail right when we were using 'em...
September 15, 2020, 06:55:02 AM
CloudWatch does seem like a bit more than is necessary, although I'm not familiar with it and have never used it. It probably makes sense to do what suchmoon is saying here, if I'm understanding correctly. We are already storing each job and a timestamp when it was last run, so maybe one page like 'modcheck?mod=flagparse' which checks the DB for timestamps on that particular job, and returns OK/Error based on the result?
I guess we could do the same thing with the logic we are using in the logo as well (like when DB is offline, the logo turns red, if the DB is up and the parsing has stopped, the logo is not spinning). We could just check the DB timestamps and if anything hasn't last run in... lets say 20 minutes to give buffer for any long-running jobs.. the logo spins but turns yellow, giving the indication something inside the parser is not running. Could get more specific here in the length of time and each job, but just a general thought.
I guess we could do the same thing with the logic we are using in the logo as well (like when DB is offline, the logo turns red, if the DB is up and the parsing has stopped, the logo is not spinning). We could just check the DB timestamps and if anything hasn't last run in... lets say 20 minutes to give buffer for any long-running jobs.. the logo spins but turns yellow, giving the indication something inside the parser is not running. Could get more specific here in the length of time and each job, but just a general thought.
September 15, 2020, 06:06:20 AM
AFAIK CloudWatch has agents to gather info from places like database servers and log files although to be honest I'd prefer to stay off Amazon-specific stuff in case they go full evil and hike the prices up and force us to leave. So for monitoring I was thinking to use a third-party uptime service that could check (among other things) web server's heartbeat and basic status (OK/ERROR) of every important module. That would require just a simple web page so perhaps easier to set up than CloudWatch.
I'm guessing that BPIP is running on ASP.Net Core, from inspecting the request headers sent by it. If that's true then there's a way to directly wire heartbeats to it: https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/health-checks?view=aspnetcore-3.1, though I admit that ASP.Net is not my strong suite so there isn't much else I can recommend.
You could also throw in some functionality that sends an email whenever a module goes down, so the error statuses don't go unnoticed.
September 14, 2020, 09:47:08 PM
The problem here is all of BPIP's updating is happening within programs inside rented servers (I guess?). The Cloudwatch event somehow has to determine whether BPIP updating is broken and you can't do that without creating a middleware inside the server that collects BPIP diagnostics and send them somewhere like S3 where cloudwatch can access them.
From my development experience, large tasks like this are as laborious and boring to implement, especially if you're not getting paid to do it, as they are necessary and useful.
From my development experience, large tasks like this are as laborious and boring to implement, especially if you're not getting paid to do it, as they are necessary and useful.
AFAIK CloudWatch has agents to gather info from places like database servers and log files although to be honest I'd prefer to stay off Amazon-specific stuff in case they go full evil and hike the prices up and force us to leave. So for monitoring I was thinking to use a third-party uptime service that could check (among other things) web server's heartbeat and basic status (OK/ERROR) of every important module. That would require just a simple web page so perhaps easier to set up than CloudWatch.
September 14, 2020, 09:37:36 PM
Or to be more accurate, we need to set up monitoring to alert us when stuff like that breaks but that's item #214 on the TODO list.
You use AWS services, right?
https://aws.amazon.com/cloudwatch/
The problem here is BPIP is running inside rented servers on AWS, and not across the über hyper-scaled AWS platform (correct me if I'm wrong). The Cloudwatch event somehow has to determine whether BPIP updating is broken and you can't do that without creating a middleware inside the server that collects BPIP diagnostics and send them somewhere like S3 where cloudwatch can access them.
From my development experience, large tasks like this are as laborious and boring to implement, especially if you're not getting paid to do it, as they are necessary and useful.
September 14, 2020, 07:49:40 PM
Or to be more accurate, we need to set up monitoring to alert us when stuff like that breaks but that's item #214 on the TODO list.
You use AWS services, right?
https://aws.amazon.com/cloudwatch/
September 14, 2020, 06:09:46 AM
Flaglog is running again, it has added new flags and is currently about 85% of the way through the weekly re-parsing of all flags... no big errors occurred here, just the parser losing connection to the DB and me not realizing it. 
Well, most of the reports pull into the profile page as general information/stats, so just because someone doesn't use the flaglog itself, they could be looking at the number of flags against a user... which will look normal in a situation where the parser has stopped parsing flags, so it might be good to at least have some sort of general alert that a particular parser has stopped, although, I agree it is probably not the highest priority right now... I'll try to check the parser log more often!
The monitoring is done by the users, in this case Ratimov. If it breaks and nobody notices, it's not needed
Well, most of the reports pull into the profile page as general information/stats, so just because someone doesn't use the flaglog itself, they could be looking at the number of flags against a user... which will look normal in a situation where the parser has stopped parsing flags, so it might be good to at least have some sort of general alert that a particular parser has stopped, although, I agree it is probably not the highest priority right now... I'll try to check the parser log more often!
September 14, 2020, 04:50:00 AM
we need to set up monitoring to alert us when stuff like that breaks
The monitoring is done by the users, in this case Ratimov. If it breaks and nobody notices, it's not needed True, I'm keeping an eye on the site as I'm almost every day checking things so if something fails I will let you guys know as i did it many times before
Out of curiosity, how many unique views the site has per day?
September 14, 2020, 01:47:08 AM
we need to set up monitoring to alert us when stuff like that breaks
The monitoring is done by the users, in this case Ratimov. If it breaks and nobody notices, it's not needed September 14, 2020, 12:58:54 AM
What happened to the flags updates?
I've pinged ibminer, flag parsing is his baby. Something must've gone broken but I'm sure he'll pick it up and fill in missing flags.
If I were Suchmoon, I would have locked everything down for the Sep 11 19th anniversary. After all, we expect another attack on the 20th, but this year would have surprised us. He was just being safe.
We won't let terrorism define us
Or to be more accurate, we need to set up monitoring to alert us when stuff like that breaks but that's item #214 on the TODO list.
September 13, 2020, 11:02:53 PM
What happened to the flags updates?
If I were Suchmoon, I would have locked everything down for the Sep 11 19th anniversary. After all, we expect another attack on the 20th, but this year would have surprised us. He was just being safe.
September 12, 2020, 09:37:24 AM
but I don't think it gets processed.
I've still been manually processing this file.. need to get back with suchmoon on adding a task to the server to do this for me but I just hadn't gotten to the recent update yet. It's good now. Thanks for the reminder! September 12, 2020, 04:17:26 AM
Users with a newly created custom Trust list (such as protrader786) don't get a working link to my Trust list viewer yet.
Every week, I create a file user_ids.txt for this, but I don't think it gets processed.
Every week, I create a file user_ids.txt for this, but I don't think it gets processed.
September 01, 2020, 08:04:18 PM
I'm not sure you changed anything here, but it seems merit count again does not work well.
For example, user @suchmoon has earned "Merit Score 5497" but in merit information table stay "Has received 4495 ". There is a difference of 2 latest earned merit. I don't count merit assigned at the beginning.
For example, user @suchmoon has earned "Merit Score 5497" but in merit information table stay "Has received 4495 ". There is a difference of 2 latest earned merit. I don't count merit assigned at the beginning.
Good catch, thank you. I had a parser crash yesterday and didn't clean up the job queue properly so the merit update got stuck. Should be updated within the next few minutes.
Under normal circumstances there can be a slight discrepancy between sent merits and total merits because those two pieces of information are scraped from two different places. It shouldn't last more than a minute or so.
Jump to: