bitcointalk.org image proxy doesn't specify user agent

shahzadafzal

copper member

Activity: 1526

Merit: 2890

Quote from: npuath on January 30, 2023, 02:24:46 AM

In 2013, the HTTP/1.1 specification was revised to include User-Agent (IETFs capital emphasis):

Quote from: IETF

A user agent SHOULD send a User-Agent field in each request unless specifically configured not to do so.

https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-p2-semantics-23#section-5.5.3

That's just a recommendation or you can best practice but not a rule. Many bots/searchengine bots doesn't set user agent.

Quote from: npuath on January 30, 2023, 02:24:46 AM

My suggestion to let the forum proxy include User-Agent is purely pragmatic, in that it may help alleviating some users' issues, given that some image servers block requests without it.

Image hosting sites like imgur imgbb all accepting requests and it's working fine. I don't think theymos is going to update the image proxy unless there's major show stopper issue.

npuath

copper member

Activity: 42

Merit: 67

Quote from: Foxpup on January 30, 2023, 01:37:14 AM

The User-Agent header is entirely optional in the HTTP protocol, and its absence is in no way indicative of a malformed or malicious request. Is there some reason you're blocking requests that omit it?

The Internet Engineering Task Force (IETF) tries very hard to be careful in its wordings, and differs between terms such as "may", "should" and "must", for example. I had a (very) small part in the inclusion of User-Agent in the 1992 draft of HTTP/2 (my bold emphasis):

Quote from: IETF

This line if present gives the software program used by the original client. This is for statistical purposes and the tracing of protocol violations. It should be included. The first white space delimited word must be the software product name, with an optional slash and version designator. Other products which form part of the user agent may be put as separate words.

https://www.w3.org/Protocols/HTTP/HTRQ_Headers.html#user-agent

In 2013, the HTTP/1.1 specification was revised to include User-Agent (IETFs capital emphasis):

Quote from: IETF

A user agent SHOULD send a User-Agent field in each request unless specifically configured not to do so.

https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-p2-semantics-23#section-5.5.3

Granted, since then, the User-Agent string has been used and abused in every possible way, and has become overly long, lying about what it really is and used for branding and advertising.

My suggestion to let the forum proxy include User-Agent is purely pragmatic, in that it may help alleviating some users' issues, given that some image servers block requests without it.

Foxpup

legendary

Activity: 4522

Merit: 3183

Vile Vixen and Miss Bitcointalk 2021-2023

The User-Agent header is entirely optional in the HTTP protocol, and its absence is in no way indicative of a malformed or malicious request. Is there some reason you're blocking requests that omit it?

npuath

copper member

Activity: 42

Merit: 67

When developing an API for historical BTC quotes yielding images, I had problems inserting the images in forum posts using the [img] BBCode - the forum image proxy replied Invalid image.

Since the image URIs worked elsewhere, I studied my server logs, and it turned out that my server blocked the requests, becuase no User agent was supplied from the image proxy. Removing the check on my server fixed the problem.

Before I found out, I searched the forum, and it seems as if more than I have had similar problems. In some of these cases it may have been because of the missing UA; maybe somebody wants to add it.

Topic: bitcointalk.org image proxy doesn't specify user agent (Read 99 times)