http-chunked-progress/draft-lnageleisen-http-chunked-progress-00

Network Working Group                                      L. Nageleisen
Internet-Draft                                             April 8, 2014
Expires: October 10, 2014


                    Chunked Progress extension to HTTP
                draft-lnageleisen-http-chunked-progress-00

Abstract

   This document describes Chunked Progress, an extension to
   Transfer-Encoding: Chunked as defined in RFC2616 [RFC2616].  Chunked
   Progress introduces a backwards-compatible, RFC2616 compliant method
   to notify the client of transfer advancement in situations where the
   server has knowledge of progress but cannot know the resource size
   ahead of time.

Status of this memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on April 1st, 2014.

Copyright Notice

   Copyright (c) 2013 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

1. Overview

   User agents have been displaying progress as feedback to the user for
   some time already. As the nature of applications using HTTP as
   evolved to an increaingly dynamic nature, having the User Agent
   estimate progress based of bytes received is getting less useful, as
   data has to be generated a priori as a whole before being sent over
   the wire together with Content-Size, or Content-Size has to be
   omitted altogether, even when the server could actually estimate
   progress on the fly. In fact, Content-Size is used as a proxy for
   actual progress towards completion.

   While informing the client of the requested resource content size is
   useful, it is useless when the exact resource size in bytes is not
   known ahead of time.  Nonetheless, the server may have knowledge of
   the progress expressable as a unitless number towards completion.

   It is important to distinguish the two notions of progress and
   completion. Completion is handled by having received Content-Size
   bytes, or with chunked content encoding without a Content-Size
   header, when a zero-length chunk is received. Content-size is a
   measure of completion, from which progress can be derived in
   increasingly limited cases.

   Even then, it can be an awkward proxy: imagine a scenario where a
   known number of files of various sizes will have their SHA1 value
   computed and sent over the wire. The content size is known beforehand
   by a simple operation, yet progress is best measured by the number of
   bytes read, not the number of bytes sent.

   Thankfully, Transfer-Encoding: chunked amounts to a form of
   multiplexing, where metadata can be sent interleaved with data on the
   same channel.  By providing complementary information establishing
   progress towards completion, this extension aims to reduce latency
   and resource usage while increasing feedback in a backwards
   compatible way. Real world typical scenarios unable to generate
   progress even though it can be known on the server side include:

   - gzipping files on the fly
   - generating data from result rows of a database request
   - generating data by walking a tree of files

2. Progress Chunk Extension

2.1. Request Header

   The client SHOULD send the Chunk-Extensions: progress header in its
   request if it supports the feature.  If the client sent the
   Chunk-Extensions: progress header, the server SHOULD include the
   progress chunk extension in the response chunks, compliant with
   either basic or extended mode of operation.  Otherwise, the server
   MAY include the progress chunk extension, but MUST comply with the
   basic mode of operation.

2.2 Basic mode of operation

   Since all HTTP/1.1 applications MUST be able to receive and decode
   the "chunked" transfer-coding, and MUST ignore chunk-extension
   extensions they do not understand, the server MAY reliably include
   the extension in chunks regardless of actual client-side support.

   The chunk extension name MUST be the string "progress", while the
   chunk extension value MUST be a short "floating point" number
   comprised between 0 and 1, representing absolute progress towards
   completion.  The special, negative value "-1" SHOULD be used when
   server progress tracking has been lost or compromised, and means
   progress status is "undefined".

       chunk-ext-name = "progress"
       chunk-ext-val = "-1"
                     | ( "0" [ "." 0*3DIGIT ] )
                     | ( "1" [ "." 0*3("0") ] )

   The progress chunk extension MAY be omitted.  If no progress chunk
   extension is present, the client MUST assume the current progress
   chunk extension value to be equal to the previous progress chunk
   extension value.  If no previous chunk extension has been encountered
   yet, the chunk extension value MUST be assumed to be "undefined".

   The client MUST NOT assume the progress value to be monotonically
   increasing, as the server MAY send any value it deems significant,
   including "undefined".  Nonetheless, the client MAY implement logic
   presenting this information as monotonically increasing.

   The client MUST NOT assume the value 1 to mean completion, due to
   possible rounding errors and insufficient precision.

   If the client did not send the appropriate Request Header, the server
   MUST NOT send zero-length chunks unless all data has been sent, this
   to ensure backwards compatibility.  To comply with Transfer-Encoding:
   chunked, the client, having not sent the request header, MUST accept
   a zero-length chunk as an end of data, whether or not this chunk has
   progress extension.

2.3 Enhanced mode of operation

   If the client sent the appropriate Request Header, the server MAY
   send zero-length chunks with progress information.  In such a case,
   the client MUST NOT assume that all data has been sent as is the case
   with naked transfer encoding chunked, and MUST wait for a zero-length
   chunk without a progress extension to assume completion.  Indeed in
   most cases, the server MAY skip sending the last progress chunk and
   end the data stream with this last zero-length bare chunk instead of
   sending two consecutive zero-length chunks.  Nonetheless, the server
   MAY, out of courtesy, send both, one notifying progress having
   reached 1 and one marking completion of chunked data transfer.

3. Notes

   The mechanism by which this extension sends additional metadata via
   chunked extensions, even when there is no actual data to be sent,
   effectively implements a much more general multiplexing extension,
   which other chunked extensions may use.  This area needs particular
   scrutiny, as it alters the Transfer-Encoding: chunked mode of
   operation.  Mabye this should best be extracted in a newly defined
   Transfer-Encoding, entirely distinct from chunked.

   Suggestions of improvements are welcome to increase compatibility
   with proxies, especially in the enhanced mode of operation.

   It is regrettable that much state handling has to be done in order to
   implement those modes of operations, but it is necessary to extend
   current functionality all the while keeping backwards compatibility.

   Clients and servers may restrict themselves to implement only basic
   mode of operation, greatly simplifying the required work but limiting
   potential functionality in the most dynamic cases.

Author's Addresses

   Loic Nageleisen

   Email: loic.nageleisen@gmail.com