Thread: Proposal to remove message length constant from F/B protocol document

Proposal to remove message length constant from F/B protocol document

From
Tatsuo Ishii
Date:
Currently we document the message size length as a constant like
Int32(5) for some message types in the frontend backend protocol
doument[1].

ReadyForQuery (B)

    Byte1('Z')

        Identifies the message type. ReadyForQuery is sent whenever the backend is ready for a new query cycle.
    Int32(5)

        Length of message contents in bytes, including self.
:
:

I think this may bring a misunderstanding to some implementer of the
protocol. i.e.  He may wright a code like this to save the work to
interpret the message length field (this is a backend side code).

if (message_char == 'Z')
{
    read_rest_of_message(5);
    :
    :
}

instead of:

if (message_char == 'Z')
{
    read_message_lenfgth_field;
    convert_it_to_host_order;
    read_rest_of_message(n);
    :
    :
}

I think the former code is not good because we might change the
message length in the future when protocol extensions are implemented
for this particular message type, and the message length may become
different length or even variable length. So my propsal is changing:

    Int32(5)

to:
    Int32

In the same sense we may want to change the sentence at the very
beginning of the section, particulary this: "the message format is
defined so that the message end can be found without reference to the
byte count".

  Notice that although each message includes a byte count at the
  beginning, the message format is defined so that the message end can
  be found without reference to the byte count. This aids validity
  checking. (The CopyData message is an exception, because it forms
  part of a data stream; the contents of any individual CopyData
  message cannot be interpretable on their own.)

[1] https://www.postgresql.org/docs/current/protocol-message-formats.html

What do you think?

Best reagards,
--
Tatsuo Ishii
SRA OSS K.K.
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp



Re: Proposal to remove message length constant from F/B protocol document

From
"David G. Johnston"
Date:
On Fri, Nov 1, 2024 at 4:53 AM Tatsuo Ishii <ishii@postgresql.org> wrote:
I think this may bring a misunderstanding to some implementer of the
protocol. i.e.  He may wright a code like this to save the work to
interpret the message length field (this is a backend side code).

if (message_char == 'Z')
{
        read_rest_of_message(5);
        :
        :
}

I concur that given our documentation we are encouraging developers to optimize for known fixed-length messages.  Why is now an appropriate time to change that guidance?  What has changed between when this guidance was added and now that invalidates that guidance?

We are going to have to assume people have done just this if we ever do make an impacting change in this area.  I'd rather keep the documentation consistent so if that change happens in 6 years the documentation still reflects the reality we are dealing with.

I can see adding an explicit warning that these length constants are not promised to never change in the future - but also that we'd have some kind of versioning bump that would go along with it.  If the optimization is taken at least test up-front that the version being seen is known to have these fixed values.

David J.