Pyjo.IOLoop.Messages - Messages on a stream

from Pyjo.IOLoop.Messages import Messages
from Pyjo.IOLoop.Stdio import Stdio

# Echo the lines of the standard input in capitals
messages = Messages(Stdio())
messages.on('message', lambda messages, message: messages.send(message.upper()))

# Or import the module, as in Google Python Style Guide

import Pyjo.IOLoop.Messages
import Pyjo.IOLoop.Stdio

# Echo the lines of the standard input in capitals
messages = Pyjo.IOLoop.Messages.new(Pyjo.IOLoop.Stdio.new())
messages.on('message', lambda messages, message: messages.send(message.upper()))
messages.on('close', lambda messages: messages.stream.reactor.stop())
messages.start()

# The messages of the Language Server Protocol, framed by Content-Length headers
messages = Pyjo.IOLoop.Messages.new(stream, framing=Pyjo.IOLoop.Messages.ContentLength())

# The standard output and input of a server in a child process
child = Pyjo.IOLoop.ChildProcess.new().spawn(['./server.py'])
messages = Pyjo.IOLoop.Messages.new(child.stdout, output=child.stdin)
messages.send('{"jsonrpc": "2.0", "method": "update", "params": [1]}')

# Start reactor if necessary
if not messages.stream.reactor.is_running:
    messages.stream.reactor.start()

Pyjo.IOLoop.Messages is a stream of messages on a Pyjo.IOLoop.Stream, a stream of bytes, as a connection of TCP or of a Unix domain socket, the standard input and output of Pyjo.IOLoop.Stdio, or the standard output and input of a Pyjo.IOLoop.ChildProcess. The framing of the messages tells where one ends and the next begins, as the line of Lines, the default, the headers of ContentLength or the length of LengthPrefix. A message is bytes, and the frame of a message is written as a whole, so that the frames of two messages never interleave.

A message that would be larger than max_message_size closes the stream gracefully with an error, as the stream can not tell where the next message begins: nothing more is read, and the messages that have been sent are written.

A framing is an object with the method decode, which takes the next message that is whole from the front of a bytearray of what has been read and returns it, or None until it is whole, raising a Pyjo.Exception for a message that is larger than the maximum size, and the method encode, which returns the frame of a message. A framing keeps what it knows of the stream that it decodes, as the length of the message whose headers it has read, so each stream of messages has one of its own.

# Messages of netstrings, as 3:abc,
class Netstrings:
    def decode(self, buffer, max_size):
        if (colon := buffer.find(b':')) < 0:
            return None
        length = int(buffer[:colon])
        if max_size and length > max_size:
            raise Pyjo.Exception.new('Maximum message size exceeded')
        if len(buffer) < colon + length + 2:
            return None
        message = bytes(buffer[colon + 1 : colon + 1 + length])
        del buffer[: colon + length + 2]
        return message

    def encode(self, message):
        return b'%d:%s,' % (len(message), message)

messages = Pyjo.IOLoop.Messages.new(stream, framing=Netstrings())

Events

Pyjo.IOLoop.Messages inherits all events from Pyjo.EventEmitter and can emit the following new ones.

close

messages.on('close', lambda messages: ...)

Emitted once the stream has been closed, or the input, when the input and the output are two, as the peer ends its output when it has no more to send.

end

messages.on('end', lambda messages: ...)

Emitted once the input has ended, as the event end of Pyjo.IOLoop.Stdio, while messages can still be sent, as the responses of the requests that are in progress. Without a subscriber of this event, the stream closes gracefully, once all messages have been written; with one, the subscriber closes it.

messages.on('end', lambda messages: messages.close_gracefully())

error

messages.on('error', lambda messages, err: ...)

Emitted if an error occurs on the stream, as a message that is larger than the maximum size, fatal if unhandled.

messages.on('error', lambda messages, err: print(f'This looks bad: {err}'))

message

messages.on('message', lambda messages, message: ...)

Emitted for each message that has arrived, without its framing.

messages.on('message', lambda messages, message: print(f'Message: {message!r}'))

timeout

messages.on('timeout', lambda messages: ...)

Emitted if the stream has been inactive for too long and will get closed automatically, as the event timeout of Pyjo.IOLoop.Stream.

See also

Pyjo.IOLoop.Stream, Pyjo.IOLoop.Stdio, Pyjoyment, https://mojolicious.org.

It has no counterpart in Mojolicious, and the framings follow the transport stdio of the Model Context Protocol, the base protocol of the Language Server Protocol, and Native Messaging of the extensions of the browsers.

Classes

class Pyjo.IOLoop.Messages.ContentLength[source]

Messages with headers, of which Content-Length has their size, as the base protocol of the Language Server Protocol and of the Debug Adapter Protocol have them. The other headers, as Content-Type, are read and not written.

framing = Pyjo.IOLoop.Messages.ContentLength.new()
max_header_size = 8192
size = framing.max_header_size
framing.max_header_size = 1024

Maximum size of the headers of a message in bytes, defaults to 8192.

class Pyjo.IOLoop.Messages.Framing(*args, **kwargs)[source]

The framing of the messages of a stream, as Lines.

decode(buffer: bytearray, max_size: int) → bytes | None[source]
message = framing.decode(buffer, 16777216)

Take the next message that is whole from the front of buffer and return it, without its framing, or None until it is whole, or raise a Pyjo.Exception for a message that is larger than max_size, unless it is 0.

encode(message: bytes) → bytes[source]
frame = framing.encode(b'Hello')

Return the frame of a message.

class Pyjo.IOLoop.Messages.LengthPrefix(size: int = 4, byteorder: Literal['little', 'big'] = 'little')[source]

Messages that follow their size, as an unsigned integer of size bytes, as Native Messaging of the extensions of the browsers has them, with 4 bytes in the native byte order.

framing = Pyjo.IOLoop.Messages.LengthPrefix.new()
framing = Pyjo.IOLoop.Messages.LengthPrefix.new(size=2, byteorder='big')
byteorder: Literal['little', 'big']
byteorder = framing.byteorder
framing.byteorder = 'big'

Byte order of the length of a message, 'little' or 'big', defaults to the native one, sys.byteorder.

size
size = framing.size
framing.size = 2

Size of the length of a message in bytes, defaults to 4.

class Pyjo.IOLoop.Messages.Lines(terminator: bytes = b'\n')[source]

Messages of one line each, as the transport stdio of the Model Context Protocol and many protocols of text have them, which end with a line feed, or a carriage return and a line feed, which terminator writes. A message has no line feed.

framing = Pyjo.IOLoop.Messages.Lines.new()
framing = Pyjo.IOLoop.Messages.Lines.new(terminator=b'\r\n')
terminator
terminator = framing.terminator
framing.terminator = b'\r\n'

Terminator of the lines that are written, defaults to b'\n'. Both are read.

class Pyjo.IOLoop.Messages.Pyjo_IOLoop_Messages(stream: Pyjo_IOLoop_Stream, *, framing: Framing | None = None, output: Pyjo_IOLoop_Stream | None = None)[source]

Pyjo.IOLoop.Messages inherits all methods from Pyjo.EventEmitter and implements the following new ones.

close() → None[source]
messages.close()

Close the stream immediately.

close_gracefully() → None[source]
messages.close_gracefully()

Close the stream gracefully, once all messages have been written, or, when the input and the output are two, as the standard input and output of a child process, the output, and the input is read until the peer ends it, so that the responses of the peer still come in.

# The last request of a server of the standard input and output, and its response
messages.send(request).close_gracefully()
close_gracefully_p() → Pyjo_Promise[source]
promise = messages.close_gracefully_p()

Same as close_gracefully(), but returns a Pyjo.Promise object, which is fulfilled once the stream has been closed, also when it is closed already.

await messages.send_p('Bye!')
await messages.close_gracefully_p()
framing: Framing
framing = messages.framing
messages.framing = Pyjo.IOLoop.Messages.ContentLength()

Framing of the messages, defaults to a Lines object.

iterate() → Pyjo_Iterator[bytes][source]
async for message in messages.iterate():
    ...

The messages as a Pyjo.Iterator, until the end of the stream, which message_p() takes.

# Echo in capitals, one message after another
async for message in messages:
    await messages.send_p(message.upper())

# The messages as JSON, in a chain
async for request in messages.iterate().map(Pyjo.JSON.decode_json):
    ...
property max_message_size: int[source]
size = messages.max_message_size
messages.max_message_size = 1024

Maximum size of a message in bytes, defaults to the value of the PYJO_MAX_MESSAGE_SIZE environment variable or 16777216 (16MiB). Setting the value to 0 will allow messages of indefinite size.

message_p() → Pyjo_Promise[source]
promise = messages.message_p()

Return a Pyjo.Promise object, which is fulfilled with the next message, or None at the end of the stream, the event close or end, or when it is closed already, and rejected with the error of the event error, or a TimeoutError when the stream has been inactive for too long. The stream reads only while a promise waits, as read_p() of Pyjo.IOLoop.Stream does, and the subscribers of the event message get the messages that it reads then. A loop of them handles the messages one after another, where the events handle them at the same time, as serve() of Pyjo.JSON.RPC does.

while (message := await messages.message_p()) is not None:
    print(message)
once_p(name: str) → Pyjo_Promise[source]
promise = messages.once_p('close')

Same as once_p() of Pyjo.EventEmitter, but fulfilled at once for the event close of a stream that is closed already, as it comes no more.

output
stream = messages.output

Pyjo.IOLoop.Stream object that the messages are written to, defaults to stream.

send(message: bytes | str, cb: Callable[[Self], Any] | None = None) → Self[source]
messages = messages.send(b'Hello')
messages = messages.send('Hello')
messages = messages.send('Hello', lambda messages: ...)

Send a message, bytes, or str that is encoded in UTF-8, the optional drain callback will be executed once all data has been written.

send_p(message: bytes | str) → Pyjo_Promise[source]
promise = messages.send_p(b'Hello')

Same as send(), but returns a Pyjo.Promise object, which is fulfilled once all data has been written, and rejected if the stream closes before, or is closed already, or with a TimeoutError when it has been inactive for too long.

await messages.send_p('Hello')
start() → None[source]
messages.start()

Start or resume reading messages from the stream.

stop() → None[source]
messages.stop()

Stop reading messages from the stream.

stream
stream = messages.stream

Pyjo.IOLoop.Stream object that the messages are read from, and written to, unless output is another one.