Pages:
Author

Topic: Poll on solving the imgur issue - page 3. (Read 2167 times)

hero member
Activity: 1659
Merit: 687
LoyceV on the road. Or couch.
June 08, 2023, 12:18:21 PM
Quote
I don't mind receiving the images, but at the moment I can't receive 800k all at once. We had to be selective.  Undecided
Unfortunately, being selective doesn't work to replace all links.

I know! Nor is it because of space or bandwidth. It's because of the number of files to be loaded.

I've been thinking about how I can help with that. Undecided
If you have the space, why would the number of pictures be a problem? Just disable directory view in your webserver and it should just be able to handle it.
copper member
Activity: 1666
Merit: 1901
Amazon Prime Member #7
June 08, 2023, 11:59:54 AM
It looks like imgur has an api, but it doesn't look like the API can retrieve images currently. Perhaps they may be willing to add such functionality.
If they add it as a paid feature, I don't think the forum should go down that road. It would set a precendent for other image hosts to do the same.
I disagree. Imgur is providing a service (hosting images), and deserves to earn a fair profit. It would cost money for the forum to host images on its own infrastructure, and as long as the cost to use imgur (and all other image hosting sites) is less than the cost of self-hosting, it will make sense to pay.

I presume that imgur usually makes its money from ads when people access images on its imgur site (not the raw image that is accessed via the forum image proxy). I presume that the image proxy complicates various metrics that imgur uses.
legendary
Activity: 1862
Merit: 5154
**In BTC since 2013**
June 08, 2023, 11:30:22 AM
Quote
I don't mind receiving the images, but at the moment I can't receive 800k all at once. We had to be selective.  Undecided
Unfortunately, being selective doesn't work to replace all links.

I know! Nor is it because of space or bandwidth. It's because of the number of files to be loaded.

I've been thinking about how I can help with that. Undecided
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
June 08, 2023, 11:22:18 AM
You can look at my data to get an idea.  Roll Eyes
That's only recent images, I'm curious how often old images are downloaded too.

Quote
I don't mind receiving the images, but at the moment I can't receive 800k all at once. We had to be selective.  Undecided
Unfortunately, being selective doesn't work to replace all links.
legendary
Activity: 1862
Merit: 5154
**In BTC since 2013**
June 08, 2023, 11:19:22 AM
And in terms of bandwidth, how do you handle that?
I don't: I don't want to have to deal with possible copyright issues Sad
Maybe theymos can share how much bandwidth the image proxy consumes in total.

You can look at my data to get an idea.  Roll Eyes

I don't mind receiving the images, but at the moment I can't receive 800k all at once. We had to be selective.  Undecided
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
June 08, 2023, 11:13:30 AM
And in terms of bandwidth, how do you handle that?
I don't: I don't want to have to deal with possible copyright issues Sad
Maybe theymos can share how much bandwidth the image proxy consumes in total.
legendary
Activity: 1862
Merit: 5154
**In BTC since 2013**
June 08, 2023, 11:02:09 AM
If it were up to me, that would be as easy as replacing "m.imgur.com" by "m.talkimg.com". My filesystem seems to perform just fine with 800k images in one directory.

OK. I thought it was all the pictures on the forum.
And in terms of bandwidth, how do you handle that?
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
June 08, 2023, 08:55:07 AM
I think a sensible way to respond to the Imgur issue would be for theymos to ask LoyceV and joker_josue to come up with a CSV file of TalkImg "fallback" links (i.e. old Imgur link -> new TalkImg link)
If it were up to me, that would be as easy as replacing "m.imgur.com" by "m.talkimg.com". My filesystem seems to perform just fine with 800k images in one directory.

@LoyceV
I know you made a backup of all the images linked on the site, which made a total of 800k.

But, can you tell me how many images are from imgur?
All of them are from imgur Wink I'm also trying to download all other images, but it gets stuck on (failing) timeouts for many different websites.
legendary
Activity: 1862
Merit: 5154
**In BTC since 2013**
June 08, 2023, 07:41:48 AM
I think a sensible way to respond to the Imgur issue would be for theymos to ask LoyceV and joker_josue to come up with a CSV file of TalkImg "fallback" links (i.e. old Imgur link -> new TalkImg link), and then modify the image proxy to make use of that file (maybe by importing it into a SQLite database, or loading it into a hash table at startup, or something).

A CSV file or SQLite database won't do, it's gotta be a dedicated database server which can scale if Bitcointalk and consequentially the image proxy server is hammered with incoming requests.

His suggestion, from what I understand is the proxy to redirect a link to another link.
And in that case, the proxy just has to know that the link X happens to point to Y. This could be a solution that would avoid messing with the DB of forum posts.



@LoyceV
I know you made a backup of all the images linked on the site, which made a total of 800k.

But, can you tell me how many images are from imgur?
hero member
Activity: 510
Merit: 4005
June 08, 2023, 07:40:15 AM
A CSV file or SQLite database won't do, it's gotta be a dedicated database server which can scale if Bitcointalk and consequentially the image proxy server is hammered with incoming requests.
This is not my first rodeo NAT, I've written a decent amount of performance-critical software in my time. I wouldn't have suggested something that I didn't have first-hand experience with, so trust me when I say that slurping up a text file into a hash table at startup or relying on SQLite (either file-backed or :memory: style) will scale just fine for this particular problem.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
June 08, 2023, 07:17:43 AM
I think a sensible way to respond to the Imgur issue would be for theymos to ask LoyceV and joker_josue to come up with a CSV file of TalkImg "fallback" links (i.e. old Imgur link -> new TalkImg link), and then modify the image proxy to make use of that file (maybe by importing it into a SQLite database, or loading it into a hash table at startup, or something).

That way, everything (mostly; banning future Imgur links was a good move) stays the same (image proxy still shielding IP addresses, images stay Tor-friendly, etc.) and the problem gets fixed transparently (and Imgur can then set itself on fire, for all we care). Also lays the technical foundation for the next time something like this happens.

Just my 2 sats.

A CSV file or SQLite database won't do, it's gotta be a dedicated database server which can scale if Bitcointalk and consequentially the image proxy server is hammered with incoming requests.

Also if we are going to rely on TalkImg as a backbone, I still believe that the hosting costs should be subsidized by the forum treasury if required for network capacity to be met.
hero member
Activity: 510
Merit: 4005
June 08, 2023, 07:03:17 AM
I think a sensible way to respond to the Imgur issue would be for theymos to ask LoyceV and joker_josue to come up with a CSV file of TalkImg "fallback" links (i.e. old Imgur link -> new TalkImg link), and then modify the image proxy to make use of that file (maybe by importing it into a SQLite database, or loading it into a hash table at startup, or something).

That way, everything (mostly; banning future Imgur links was a good move) stays the same (image proxy still shielding IP addresses, images stay Tor-friendly, etc.) and the problem gets fixed transparently (and Imgur can then set itself on fire, for all we care). Also lays the technical foundation for the next time something like this happens.

Just my 2 sats.
legendary
Activity: 2534
Merit: 1713
Top Crypto Casino
June 08, 2023, 06:16:06 AM
Nope it's not working... it was working for some time but it's down again for last few days.
Oh no here we go again. If it has failed to supply images again then rendering members unable to upload to imgur is the correct step forward taken by theymos.

I start to think they created a bandwidth control system for each site. That is, when a certain site reaches a certain bandwidth, they block the site for a number of days, until the counter is reset. Perhaps they want to implement a system so that sites pay to use their bandwidth.
Stranger things have occurred in the past, I would not be surprised if your theory was correct and to a degree from their hosting perspective it might be a strategy that keeps them rotating their biggest audience.

Given that imgur seems to be accesible again
It's not Sad
This whole imgur images are available or not on issue will probably be resolved by theymos in the coming days. Option #2 will handle the images-not-found issue with links and as for future images, there will have be to be other hosting solutions. Apart from some learning issues I cannot recall ever having a problem with Postimages
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
June 08, 2023, 03:34:19 AM
#99
Given that imgur seems to be accesible again
It's not Sad
legendary
Activity: 1862
Merit: 5154
**In BTC since 2013**
June 08, 2023, 03:34:16 AM
#98
Given that imgur seems to be accesible again (for the time being at least) but for the images that now or in future are no longer accessible I think using Option #2 would be a way to simply get it over and done with and let the past be the past.

I start to think they created a bandwidth control system for each site. That is, when a certain site reaches a certain bandwidth, they block the site for a number of days, until the counter is reset. Perhaps they want to implement a system so that sites pay to use their bandwidth.

This is just my theory. Because I find it strange, for a few days the images work and then they stop working for a while, then they work again. I don't see them manually turning on/off access to Bitcointalk. But it's just a theory, it could be wrong.

copper member
Activity: 1526
Merit: 2890
June 08, 2023, 03:33:32 AM
#97
Given that imgur seems to be accesible again (for the time being at least) but for the images that now or in future are no longer accessible I think using Option #2 would be a way to simply get it over and done with and let the past be the past.


Nope it's not working... it was working for some time but it's down again for last few days.

As per the current poll stats 54.3% votes already opted for option 2.

But like someone pointed earlier we also have to think which one hurts more in terms of SEO and user experience broken image or broken link?
legendary
Activity: 2534
Merit: 1713
Top Crypto Casino
June 08, 2023, 03:13:32 AM
#96
Given that imgur seems to be accesible again (for the time being at least) but for the images that now or in future are no longer accessible I think using Option #2 would be a way to simply get it over and done with and let the past be the past.

If imgur will provide an error for future uploads it puts the onus of the member to select an image hosting website which should serve the purpose and also keep a back up of all images uplaoded from now onwards.

It looks like imgur is no longer blocking the image proxy. I didn't change anything on the forum's end, so imgur must've loosened up their limits. I guess it's not necessary to convert img images into links as long as imgur keeps working. But to prevent this from happening again, you'll get an error if you try to embed imgur images in posts from now on.
legendary
Activity: 1789
Merit: 2535
Goonies never say die.
June 04, 2023, 06:09:47 AM
#95
I still think some form of DIY is probably the only good long-term solution for the forum, then again I'm a DIY kinda guy in general, so maybe I'm biased. Tongue

I'd expect most image hosts to eventually start deleting images, or they just go belly up for one reason or another, get sold to another company with new policies, etc.

Relying on any 3rd party image hosts will eventually lead to missing images when something happens users may not have time to react to, or just that users may not be active during the time they have to react.


Links should absolutely be way better than having broken images that can't be viewed anyways. Although one would have to visit the link to view image, it would be one way to ease the stress right now, until something is done to rectify the imgur issue.
It depends on the image, if a host deletes the image, Bitcointalk's invalid image would be better then a broken link, for SEO.

If the image still exists on the host, the link would be better as it would not hurt SEO and would have a valid link out to an image that works.

That said, I think replacing images with links takes away from the flow of certain threads and, overall, hurt them... especially threads like the art contest, tutorials, and others.
hero member
Activity: 714
Merit: 1010
Crypto Swap Exchange
June 04, 2023, 05:24:15 AM
#94
To me option 4 is best and what I chose. I never used imgur, usually ImgBB, but will switch to talkimg as it sounds good from other user's comments and is more under control of people here. I don't see much of a point to try to stick to imgur, make a clean cut to those suckers. They f***ed up once or more and will likely keep up being unreliable. No, thanks!
full member
Activity: 980
Merit: 237
June 04, 2023, 01:55:56 AM
#93
There are a lot of broken imgur links. How do you think this should be resolved? I've come up with these possible solutions:

2: Make it so that all past imgur embedded images show up as a clickable link instead of as an embedded image, as if the poster had just posted the imgur link instead of using the [img] tag. I could also maybe make it so that if you click the link, it will JavaScript-expand into an embedded image (using the specified width etc.) without going through the image proxy.


I think the best solution for those broken images is the option no 2, because if the links of those images are available then things will be easy for the posters, and a kind of notification should be sent to all the users whose links are broken and the notification should also take them to the posts where the images are broken. That way the poster will be able to update the image to a new host like TalkImg.com to fix the mess created by imgur.

I think it will also be helpful for other users because even if the original poster fails to update his/her links, the users will still be able to access the images at imgur by clicking on the links. Because of the imgur problem the images are mostly down right now and we are unable to view the URL of the images because they are embedded as images. The availability of the links can be very useful in such times.

Another solution to such problem is to have both options of clickable links and embedded images. The links should be placed below the embedded images so users can easily access the links and view the images on imgur. I think with this solution the users will be happy because they can view the images and even if the images aren't working as embeddings then still they can click the link of the images. The link should be clickable otherwise many users aren't so good at copy/pasting of the links.
Links should absolutely be way better than having broken images that can't be viewed anyways. Although one would have to visit the link to view image, it would be one way to ease the stress right now, until something is done to rectify the imgur issue.
Pages:
Jump to: