Skip to content

Front About Search
  You are not logged in Link icon Log in Link icon Join
You are here: Home » Members » Edd Dumbill » Articles » RSS reader programmers: fix your referrer fields! » Which standard?

Log in

Comment

Above in this comment thread: RSS reader programmers: fix your referrer fields! »

Comment

Discussion icon Which standard?

Posted by: qmacro at 2003-02-03

Hi Edd (nice to see WtW back again)

I may be missing something (my brane is only half on), but I'm not sure what you're expecting to appear in the referer entry when an aggregator grabs something from your site. Perhaps you're talking about a different context, but the two I can think of don't leave much room for inspiration:

  • aggregator is grabbing the RSS feed: there is no referer, it's just a URL that the aggregator has been told to fetch. Putting fake referer data in the request is just as much a sticking plaster as that to which you referred, perhaps worse, as it's wrong information
  • aggregator user clicks on href in HTML display inside aggregator: what URL do you suggest as a referer when the HTML is locally rendered and is (for want of a better type) like a file:// URL?

And the other one I can think of is fine:

  • aggregator is web-based (e.g. http://www.pipetree.com/cgi-bin/blosxom/djnews): there's no problem here as the web browser will send the appropriate referer URL information

I agree that setting a fake referer URL is not a good thing, but purely because it's not what the real referer is (it's impossible to determine, in other words, which I guess is why there's a second best URL pointing to the aggregator homepage).

Which web standard are you referring to here?

Also, from what I can see, aggregators hitting my site (including NetNewsWire et al.) correctly announce themselves in the user-agent field.

Hope that made at least some sense..

dj

ps. suggestion: any chance that the original post is displayed above the comment form to provide context (i.e. so I can remember what I'm replying to)

Comment

Discussion icon This standard...

Posted by: edd at 2003-02-03

aggregator is grabbing the RSS feed: there is no referer, it's just a URL that the aggregator has been told to fetch. Putting fake referer data in the request is just as much a sticking plaster as that to which you referred, perhaps worse, as it's wrong information

This is the situation I'm referring to in my article, yes. See the link to Kottke's bit. An empty referrer field is the right thing to do here.

aggregator user clicks on href in HTML display inside aggregator: what URL do you suggest as a referer when the HTML is locally rendered and is (for want of a better type) like a file:// URL?

I would guess that the URL of the originating RSS file would make sense there.

aggregator is web-based (e.g. http://www.pipetree.com/cgi-bin/blosxom/djnews): there's no problem here as the web browser will send the appropriate referer URL information

Exactly.

Which web standard are you referring to here?

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.36

ps. suggestion: any chance that the original post is displayed above the comment form to provide context (i.e. so I can remember what I'm replying to)

Agreed it would be good, I'll have to dig in to Plone to see how easy it is to turn this one...

Replies to this comment

Discussion icon Oh, that standard (Posted by qmacro at 2003-02-03)