My RSS Reader - Handling Indy's HTTP 304 Not Modified

While working on “My RSS Reader”, I came across a peculiar Indy behaviour. In retrieving feeds from URLs, I used Indy's TIdHTTP component.

It appears that Indy throws two different kind of exceptions for HTTP 304 for a HTTP Conditional Get. At times, it'll give a EIdHTTPProtocolException. At times, it'll give a EIdReadTimeout.

Try a simple experiment. New Delphi application. Drop an edit, memo and a button component. The code for the experiment is given below. If you need a proxy, remember to set it.

Press the retrieve button twice. The first time, you'll see a HTTP 1.1/200 OK. The second time, depending on whether you're debugging or just running the code, you'll either see an EIdHTTPProtocolException or an EIdReadTimeout.

Now, in IdHTTP.pas, inside TIdHTTPProtocol.ProcessResponse, when the ResponseCode is 304, the program flow directs it to execute the RaiseException routine. In Indy 9.0.14, this is line 1547 of IdHTTP.pas. Place a breakpoint there, or at the beginning of the RaiseException routine. Click on Retrieve again. The IDE stops at the RaiseException routine. Trace or step through it using F7 or F8. If the call to FHTTP.ReadResult(Response) succeeds, EIdHTTPProtocolException would be raised. However, if the call fails, a EIdReadtimeout would be raised.

Thus, IdHTTP is not giving a definite response to HTTP 304. I handled this in my code by checking for either EIdReadtimeout, or EIdHTTPProtocolException, but this is not a graceful solution, since if I set IdHTTP.Readtimeout to, say 2000 ms, when a HTTP 304 occurs, I wouldn't be able to tell whether it's really a timeout, or it's a HTTP 304, since both will raise a EIdReadTimeout.

Also, a HTTP 304 will always take Indy at least 2000 ms to give a response, instead of immediately, because RaiseException sets the ReadTimeout property to 2000 ms.

type
  TForm1 = class(TForm)
    Edit1: TEdit;
    Button1: TButton;
    Memo1: TMemo;
    IdHTTP1: TIdHTTP;
    procedure FormCreate(Sender: TObject);
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.FormCreate(Sender: TObject);
begin
  // Set my proxy
  // IdHTTP1.ProxyParams.ProxyServer := '';
  // IdHTTP1.ProxyParams.ProxyPort   := 8080;
  Edit1.Text := 'http://homepages.borland.com/aohlsson/blog_beta/feed.xml';
end;

procedure TForm1.Button1Click(Sender: TObject);
var
  S: string;
begin
  try
    S := IdHTTP1.Get(Edit1.Text);
    Memo1.Lines.Add(IdHTTP1.ResponseText);
    IdHTTP1.Request.LastModified := IdHTTP1.Response.LastModified;
  except
    on E: EIdHTTPProtocolException do
      Memo1.Lines.Add('EIdHTTPProtocolException: '+IdHTTP1.ResponseText);
    on E: Exception do
      Memo1.Lines.Add(Format('%s: %s ResponseText: %s', [E.ClassName, E.Message,
       IdHTTP1.ResponseText]));
  end;
end;