[ANN] bitcoindata.science | sponsored by BC.Game

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

Quote from: dkbit98 on December 11, 2024, 04:04:47 PM

Quote from: bitmover on December 11, 2024, 09:29:28 AM

Ir is not , because it was working fine like 3 days ago.

Server response time is the same

Are there issues with any other specific topics in forum, or just the one that you mentioned with 50 images?
It is a bit strange that TalkImg recently had some serious issues, and your service is also using images, maybe there is some connection.

just this one, when you request up to 5 images it works fine.
This is something related to rate limits, maximum number of requests per second

joker_josue

legendary

Activity: 1862

Merit: 5154

**In BTC since 2013**

Quote from: dkbit98 on December 11, 2024, 04:04:47 PM

Quote from: bitmover on December 11, 2024, 09:29:28 AM

Ir is not , because it was working fine like 3 days ago.

Server response time is the same

Are there issues with any other specific topics in forum, or just the one that you mentioned with 50 images?
It is a bit strange that TalkImg recently had some serious issues, and your service is also using images, maybe there is some connection.

It's different, what happened to TalkImg was that it suffered a supposed DDoS attack.

From the looks of it, someone is trying to make money by spreading attacks on services and the like, somehow linked to the forum. I say this based on some feedback I received from other users.

Even so, this will not be the case. It was probably some configuration change, perhaps an update (?), but we still can't understand what it is.

dkbit98

legendary

Activity: 2212

Merit: 7064

Quote from: bitmover on December 11, 2024, 09:29:28 AM

Ir is not , because it was working fine like 3 days ago.

Server response time is the same

Are there issues with any other specific topics in forum, or just the one that you mentioned with 50 images?
It is a bit strange that TalkImg recently had some serious issues, and your service is also using images, maybe there is some connection.

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

Quote from: joker_josue on December 11, 2024, 08:51:49 AM

I still think it's the proxy's waiting time for the server.

Ir is not , because it was working fine like 3 days ago.

Server response time is the same

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: bitmover on December 11, 2024, 07:27:12 AM

Quote from: Pmalek on December 11, 2024, 04:47:39 AM

Could the forum be causing the errors? Perhaps it's time to tag @theymos in the discussion and see if he has any ideas.

It is possible that he changed something in the server recently, which may be blocking too many requests....

Nothing should have been changed:

Quote from: theymos on November 06, 2024, 10:38:45 AM

Yeah, now the imageproxy is under a DDoS. (You may still see some images due to caching in your browser.)

I'm not going to try to fix this.

joker_josue

legendary

Activity: 1862

Merit: 5154

**In BTC since 2013**

Quote from: bitmover on December 11, 2024, 07:27:12 AM

Quote from: Pmalek on December 11, 2024, 04:47:39 AM

Could the forum be causing the errors? Perhaps it's time to tag @theymos in the discussion and see if he has any ideas.

It is possible that he changed something in the server recently, which may be blocking too many requests....

I was taking a look at this problem today, I couldn't solve now.

I still think it's the proxy's waiting time for the server.
Maybe @theymos can give the proxies more waiting time for requests to the Bitmover website.
I don't know, it's just an idea.

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

Quote from: Pmalek on December 11, 2024, 04:47:39 AM

Could the forum be causing the errors? Perhaps it's time to tag @theymos in the discussion and see if he has any ideas.

It is possible that he changed something in the server recently, which may be blocking too many requests....

I was taking a look at this problem today, I couldn't solve now.

Pmalek

legendary

Activity: 2730

Merit: 7065

I don't think the error is related to the number of images and requests. I just created a topic in the Archival board and added a table with only 10 casinos there. There are problems displaying the images and rates there as well. Sometimes they load just fine, other times there are image errors.
https://bitcointalksearch.org/topic/--5522385

The real questions is, how did it work in the past and what changed in the meantime? Could be server-related, a Cloudflare issue, a new setting that needs to be configured, or some other software-related change.
Could the forum be causing the errors? Perhaps it's time to tag @theymos in the discussion and see if he has any ideas.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: bitmover on December 10, 2024, 03:52:15 PM

Quote from: joker_josue on December 10, 2024, 02:33:21 PM

you could reduce these requests, as the site would present images that were already generated seconds before, by another user.
I cached the price data already, as mentioned above, so I don't request information from third party. But the user need to ask 50 images , one by one, as this the way the forum works

It's not likely you can cache the images from a previous user who requested them, as they'll quickly be outdated. And caching "ahead" would mean you have to recreate all images every few minutes for the rest of eternity, to keep fresh copies in your cache. So it won't work.

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

Quote from: joker_josue on December 10, 2024, 02:33:21 PM

Quote from: bitmover on December 10, 2024, 06:44:57 AM

Anyway, there are many requests made by pmalek topic, ideally there shoyld be a way to reduce those requests (about 50)

Is it 50 per second? Undecided

Have you ever thought about creating a caching system?
It is true that the purpose of the script is to display the price in real time when it is loaded. But, I believe that the objective will not be significantly achieved if the information is updated every x minutes. So, you could reduce these requests, as the site would present images that were already generated seconds before, by another user.

Now, I don't know how you can make this work, and whether you would have to change how the site works a lot. But the idea remains.

Even with a cache system he would also need to request 50 images and that will block the page anyway...

I cached the price data already, as mentioned above, so I don't request information from third party. But the user need to ask 50 images , one by one, as this the way the forum works

joker_josue

legendary

Activity: 1862

Merit: 5154

**In BTC since 2013**

Quote from: bitmover on December 10, 2024, 06:44:57 AM

Anyway, there are many requests made by pmalek topic, ideally there shoyld be a way to reduce those requests (about 50)

Is it 50 per second? Undecided

Have you ever thought about creating a caching system?
It is true that the purpose of the script is to display the price in real time when it is loaded. But, I believe that the objective will not be significantly achieved if the information is updated every x minutes. So, you could reduce these requests, as the site would present images that were already generated seconds before, by another user.

Now, I don't know how you can make this work, and whether you would have to change how the site works a lot. But the idea remains.

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

Quote from: examplens on December 10, 2024, 06:56:55 AM

Quote from: bitmover on December 10, 2024, 06:44:57 AM

What intrigues me is that it was working and suddenly stopped.

Anyway, there are many requests made by pmalek topic, ideally there shoyld be a way to reduce those requests (about 50)

I am working on it Wink

One additional question, does bitcoindata.science, for all the 50+ requests it receives, also make 50+ requests according to CMC or Coingecko (or from where you already get values)

No, I fixed this long ago.

I get the bitcoin price periodically and save it in the database and here:
https://bitcoindata.science/api/priceusd.json

Those images get the price from this json above (if not from the json, they get from the database), not from third party API.

examplens

legendary

Activity: 3472

Merit: 3507

Crypto Swap Exchange

Quote from: bitmover on December 10, 2024, 06:44:57 AM

What intrigues me is that it was working and suddenly stopped.

Anyway, there are many requests made by pmalek topic, ideally there shoyld be a way to reduce those requests (about 50)

I am working on it Wink

One additional question, does bitcoindata.science, for all the 50+ requests it receives, also make 50+ requests according to CMC or Coingecko (or from where you already get values)

Have you thought about conceptualizing these requests differently, for example, there is one API request and all other calls at the same time (or a period of a few seconds) use the same information from the first initial request.
For example, the Bitcoin price in a few seconds is always very approximately the same and would not affect the accuracy of the information provided.

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

Quote from: Pmalek on December 10, 2024, 04:59:10 AM

It looks like I've caused you guys a lot of work and headache. I am sorry about that. I appreciate the time you are taking in trying to fix the problem which seems much more complex than I initially thought. I also didn't know there were multiple parties involved in the bitcoindata.science project, but it looks like examplens has something to do with it as well. There is also joker_josue who is trying to assist.

I work alone in bitcoindata.science.
Examplens shares his server with me, for free, making all possible.

Quote from: LoyceV on December 10, 2024, 05:34:01 AM

Quote from: Pmalek on December 10, 2024, 04:59:10 AM

I appreciate the time you are taking in trying to fix the problem which seems much more complex than I initially thought. I also didn't know there were multiple parties involved

From my experience: it starts simple, and everything works. Then a spammer drops by, followed by a DDoS and protective measures start messing with regular usage. Welcome to the internet, it sucks Tongue

What intrigues me is that it was working and suddenly stopped.

Anyway, there are many requests made by pmalek topic, ideally there shoyld be a way to reduce those requests (about 50)

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: Pmalek on December 10, 2024, 04:59:10 AM

I appreciate the time you are taking in trying to fix the problem which seems much more complex than I initially thought. I also didn't know there were multiple parties involved

From my experience: it starts simple, and everything works. Then a spammer drops by, followed by a DDoS and protective measures start messing with regular usage. Welcome to the internet, it sucks Tongue

Pmalek

legendary

Activity: 2730

Merit: 7065

It looks like I've caused you guys a lot of work and headache. I am sorry about that. I appreciate the time you are taking in trying to fix the problem which seems much more complex than I initially thought. I also didn't know there were multiple parties involved in the bitcoindata.science project, but it looks like examplens has something to do with it as well. There is also joker_josue who is trying to assist.

joker_josue

legendary

Activity: 1862

Merit: 5154

**In BTC since 2013**

Quote from: bitmover on December 09, 2024, 08:20:02 PM

Thanks. I added those rules to mine. But same error... Same amount of images not loading Sad

This is why I think this is not a cloudflare problem.

I already had rules which would make the same effect.

Well, I still think it is related to the server response time and the proxy waiting time. One ends up canceling out the other, and the connection is interrupted. Undecided

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

Quote from: joker_josue on December 09, 2024, 04:30:59 PM

I added it in two places of the WAF rules.

Go to "Security > WAF > Tools" and add the following:
https://www.talkimg.com/images/2024/12/09/p45Yv.png

In "Custom rules" I also did this:
https://www.talkimg.com/images/2024/12/09/p478H.png

Here you can also add the IP. But I think the most efficient are the rules in WAF > Tools.

Now, the IP can sometimes change (rare but it can happen), so you can monitor it here: https://www.meuenderecoip.com/descobrir-ip-do-site.php

Hope this helps.

Cloudflare really isn't a practical service. It may be good and have a lot of features, but it's a bit confusing to organize. It took me a while to understand what was effective or not. Furthermore, their support is not very good, even for paying customers. At least I didn't get any kind of support. It was a lot of trial and error, based on some of their (complex) tutorials.

Thanks. I added those rules to mine. But same error... Same amount of images not loading Sad

This is why I think this is not a cloudflare problem.

I already had rules which would make the same effect.

joker_josue

legendary

Activity: 1862

Merit: 5154

**In BTC since 2013**

Quote from: bitmover on December 09, 2024, 04:11:59 PM

What is the server proxy IP? Did you add it to the WAF settings ?

I added it in two places of the WAF rules.

Go to "Security > WAF > Tools" and add the following:
https://www.talkimg.com/images/2024/12/09/p45Yv.png

In "Custom rules" I also did this:
https://www.talkimg.com/images/2024/12/09/p478H.png

Here you can also add the IP. But I think the most efficient are the rules in WAF > Tools.

Now, the IP can sometimes change (rare but it can happen), so you can monitor it here: https://www.meuenderecoip.com/descobrir-ip-do-site.php

Hope this helps.

Cloudflare really isn't a practical service. It may be good and have a lot of features, but it's a bit confusing to organize. It took me a while to understand what was effective or not. Furthermore, their support is not very good, even for paying customers. At least I didn't get any kind of support. It was a lot of trial and error, based on some of their (complex) tutorials.

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

Quote from: joker_josue on December 09, 2024, 03:07:42 PM

I did some more tests on the topic (several refreshes) and there comes a point where it stops showing all the images, which really indicates some blockage to the image proxy.
In addition to the exception to the forum, you have to specifically give it to the proxy.

I suggest adding the ASN of the forum server to the list of provisions/executions.

What is the server proxy IP? Did you add it to the WAF settings ?

have looked in several cloudflare configurations, and i don't think it there is much to do there.

I even added a free limit rule to all /api/* endpoints

Topic: [ANN] bitcoindata.science | sponsored by BC.Game (Read 3500 times)