Events
Event types, their encoded form, the client end of a slot's event queue and event dispatch, in adorm.events. The events topic describes the path from a slot to the client and how adorm interprets each event.
from adorm.events import BackendEvent, CallbackEvent, EventBus, EventChannelEvent types
Events are frozen dataclasses and compare by value. Each class has a kind, the value of "event" in its encoded form.
adorm.events.Event
Base class of every event.
to_dict(encode=encode_value)
The event as JSON-safe data, {"event": <kind>, ...}. encode encodes result values and payloads; the default, adorm.codec.encode_value, infers each value's type. Raises TypeError for a request id that is neither a string nor an integer, and on the base class.
from_dict(data, decode=decode_value)
A static method: the event to_dict() encoded. decode decodes result values and payloads. Raises ValueError for data that is not a mapping, an unknown "event", a missing field, and a field of the wrong type: a request id that is neither a string nor an integer, a name that is not a string, a channel that is neither a string nor null, a started that is not a boolean.
event = ResultEvent(12, [{"id": 1, "name": "Ana"}])
data = event.to_dict()
# {"event": "result", "request_id": 12,
# "value": {"type": "list", "value": [{"type": "dict", "value": {"id": {"type": "int", "value": 1}, ...}}]}}
assert Event.from_dict(json.loads(json.dumps(data))) == eventadorm.events.ResultEvent(request_id, value)
The result of the request request_id: rows as dicts, a scalar or a count. Kind "result".
adorm.events.ErrorEvent(request_id, error)
The request request_id failed with error, an adorm exception. Kind "error"; the error is encoded with encode_error(), which encodes validation params with the event's encode.
adorm.events.CancelledEvent(request_id, started)
The request request_id was cancelled at the client's request. Kind "cancelled".
startedisFalsewhen the request's call did not start: it was queued, or cancelled before the slot handed it to the connection (for example while the slot set its connection options).startedisTruewhen it was cancelled in the connection while running; whether it took effect is unknown.
adorm.events.CallbackEvent(name, payload=None)
A point in a connection's life the client runs callbacks for. Kind "callback". The slot delivers the names in RESERVED_CALLBACKS; a driver connection emits other names.
# inside a driver connection, after the target cleared a cache the client keeps a copy of
self.emit(CallbackEvent("cache_cleared", {"keys": ["club:1"]}))adorm.events.BackendEvent(name, channel=None, payload=None)
An event the database or the target raised without a request. Kind "backend". name is the kind of event ("notify" for PostgreSQL NOTIFY), channel its source within that kind (the NOTIFY channel), payload its data. A driver lists the names it emits in Capabilities.backend_events.
# inside a driver connection, when PostgreSQL delivers NOTIFY orders, '42'
self.emit(BackendEvent("notify", "orders", "42"))adorm.events.NOTIFY
"notify": the BackendEvent name of a PostgreSQL notification (NOTIFY, pg_notify()) on a channel the connection listens on (adorm.listen()). channel is the notification's channel, exactly as sent; payload its payload string, "" when it has none.
BackendEvent("notify", "orders", "42") # SELECT pg_notify('orders', '42')
BackendEvent("notify", "Orders", "") # NOTIFY "Orders"adorm.events.NOTICE
"notice": the BackendEvent name of a message the server sends with a statement's result, such as PL/pgSQL's RAISE NOTICE or a WARNING.
channel: the untranslated severity ("NOTICE","WARNING","INFO","DEBUG","LOG").payload: a dict withseverity,sqlstate,message,detailandhint;Nonefor a field the server did not send.
The server's client_min_messages selects the severities sent (default: NOTICE and above). The event arrives before the outcome of the request whose statement caused it.
# RAISE WARNING 'check the clubs' USING ERRCODE = '01000';
BackendEvent("notice", "WARNING", {"severity": "WARNING", "sqlstate": "01000", "message": "check the clubs",
"detail": None, "hint": None})adorm.events.CONNECTION_LOST
"connection_lost": the BackendEvent name a connection emits, at most once, when its database connection ends without adorm closing it (for example, the server terminates the session). channel is None, payload the reason as text. The connection is unusable afterwards; a Listener raises InterfaceError when it reaches the event.
A driver that declares it detects the end at least while the connection listens and no call runs. The native PostgreSQL driver detects it at any time, the Django driver only while the connection listens.
BackendEvent("connection_lost", None, "the server connection ended")adorm.events.OUTCOME_EVENTS
(ResultEvent, ErrorEvent, CancelledEvent): the event types that end a request. A slot delivers exactly one of them for every request it accepted.
adorm.events.ON_COMMIT
"on_commit": the CallbackEvent name a slot delivers when a commit request succeeds, before the request's outcome.
adorm.events.ON_ROLLBACK
"on_rollback": the CallbackEvent name a slot delivers when a rollback request succeeds, before the request's outcome, and when it rolls back a transaction whose client sent no request within the lease's commit_timeout.
adorm.events.RESERVED_CALLBACKS
frozenset({ON_COMMIT, ON_ROLLBACK}). A slot drops a CallbackEvent with one of these names emitted by its driver connection and logs an error on the adorm.executor logger.
adorm.events.Emit
The type of the emit callable: Callable[[Event], None]. A slot accepts only CallbackEvent and BackendEvent through it and drops any other event with an error on the adorm.executor logger.
Encoded errors
adorm.events.encode_error(error, encode=encode_value)
error as JSON-safe data, {"class": <name>, "message": <text>, ...}:
| Exception | Encoded |
|---|---|
a class of adorm.exceptions | its class name and message |
| a subclass defined elsewhere | the nearest class of adorm.exceptions it derives from |
a model's DoesNotExist or MultipleObjectsReturned | "class" plus "model", the model's label |
Error and its subclasses | also sqlstate |
IntegrityError and its subclasses | also constraint_name, table_name, schema_name, detail and model when set, and columns, values and fields as lists when not empty |
SessionTimeout, DeadlockError | also timeout and the sessions as (using, session_name) pairs |
OperationTimeout | also phase, timeout and the session as a (using, session_name) pair |
ValidationError | the messages by field, or the list of messages; each error's message, code and params under "errors" (by field, or a list), or under "error" for a single error; encode encodes each param, and a param it cannot encode becomes its text |
| any other exception | DatabaseError, with the message "<class name>: <message>" |
The driver's original exception, the objects of a ProtectedError or RestrictedError and the operation an OperationTimeout names are not encoded.
adorm.events.decode_error(data, decode=decode_value)
The exception encode_error() encoded; decode decodes validation params. Sessions are restored with create_session(). An unregistered model label gives the generic ObjectDoesNotExist or MultipleObjectsReturned, an unknown class DatabaseError.
Raises ValueError for:
- data without a string
"class"; - a validation error entry without a string
message, with a non-stringcodeor withparamsthat are not a mapping; - an
IntegrityErrorwhoseconstraint_name,table_name,schema_name,detailormodelis not a string, or whosecolumns,valuesorfieldsare not a list of strings.
data = encode_error(IntegrityError("duplicate key", sqlstate="23505"))
# {"class": "IntegrityError", "message": "duplicate key", "sqlstate": "23505"}
error = decode_error(data) # IntegrityError("duplicate key"), error.sqlstate == "23505"
data = encode_error(map_django_error(unique_violation))
# {"class": "IntegrityError", "message": "duplicate key value violates unique constraint \"clubs_member_email_key\"…",
# "sqlstate": "23505", "constraint_name": "clubs_member_email_key", "table_name": "clubs_member",
# "schema_name": "public", "detail": "Key (email)=(ana@example.com) already exists.", "model": "clubs.Member",
# "columns": ["email"], "values": ["ana@example.com"], "fields": ["email"]}
decode_error(data).key # {'email': 'ana@example.com'}
data = encode_error(ValidationError({"name": [ValidationError("Taken.", "unique", {"unique_check": ("name",)})]}))
# {"class": "ValidationError", "message": {"name": ["Taken."]},
# "errors": {"name": [{"message": "Taken.", "code": "unique",
# "params": {"unique_check": {"type": "tuple", "value": [{"type": "str", "value": "name"}]}}}]}}
decode_error(data).error_dict["name"][0].code # 'unique'
data = encode_error(ValidationError("“%(value)s” is not a valid UUID.", "invalid", {"value": "x"}))
# {"class": "ValidationError", "message": ["“x” is not a valid UUID."],
# "error": {"message": "“%(value)s” is not a valid UUID.", "code": "invalid",
# "params": {"value": {"type": "str", "value": "x"}}}}
decode_error(data).code # 'invalid'The client end
adorm.events.EventChannel(fallback=None)
The client end of a slot's event queue. Each SlotLease has one as channel, with the pool's events bus as fallback. deliver() interprets each event:
| Event | Interpretation |
|---|---|
an outcome of a request awaited with expect() | settles its future: ResultEvent sets the result, ErrorEvent the exception, CancelledEvent cancels it |
| an outcome nobody awaits | put on events; dropped after close() |
a CallbackEvent with handlers registered by on() for its name | runs the handlers, in registration order |
any other CallbackEvent, a BackendEvent | put on events; dispatched to fallback after close() |
A handler that raises is logged on the adorm.events logger and the others still run.
| Member | Meaning |
|---|---|
events | an asyncio.Queue of the events nobody claimed, in delivery order |
fallback | the EventBus that receives unclaimed callbacks and backend events once the channel closes |
closed | whether close() was called |
awaited | how many requests expect() awaits an outcome for |
expect(request_id)
Returns an asyncio.Future that the outcome of request_id settles. An id that is already awaited raises ValueError; a closed channel raises InterfaceError.
forget(request_id)
Stops awaiting request_id; its outcome, when it arrives, is treated as one nobody awaits.
on(name, handler)
Runs handler(event) for every CallbackEvent named name instead of queueing it. Returns a function that removes this registration; calling it again does nothing. A handler that is not callable raises TypeError.
deliver(event)
Interprets event. Anything but an Event raises TypeError.
close(error=None)
Dispatches the callbacks and backend events still on events to fallback, drops the outcomes still there, and fails the futures still awaiting an outcome with error, by default InterfaceError. A second call does nothing.
channel = EventChannel(fallback=pool.events)
future = channel.expect(request.request_id)
channel.on("cache_cleared", lambda event: cache.drop(event.payload["keys"]))
channel.deliver(CallbackEvent("cache_cleared", {"keys": ["club:1"]})) # runs the handler
channel.deliver(ResultEvent(request.request_id, 3))
assert await future == 3
channel.close()Dispatch
adorm.events.EventBus
Dispatches events to subscribed handlers, synchronously, in subscription order. A handler that raises is logged on the adorm.events logger and the remaining handlers run. A handler subscribed during a dispatch first runs for the next event; one unsubscribed during a dispatch does not run for the current one. Each pool has one as events.
subscribe(event_type, handler, *, name=None, channel=None)
Calls handler(event) for every dispatched event that is an instance of event_type and, when given, whose name and channel attributes equal name and channel. Returns a function that unsubscribes; calling it again does nothing.
dispatch(event)
Calls every matching handler and returns how many were called.
bus = EventBus()
unsubscribe = bus.subscribe(BackendEvent, lambda e: print("order", e.payload), name="notify", channel="orders")
bus.subscribe(CallbackEvent, lambda e: print("cache cleared"), name="cache_cleared")
bus.dispatch(BackendEvent("notify", "orders", "42")) # prints "order 42", returns 1
bus.dispatch(CallbackEvent("cache_cleared")) # prints "cache cleared", returns 1
unsubscribe()Listening for notifications
adorm.listen(*channels, session=None)
A Listener for channels on session, the current session by default. Each channel name is checked with check_channel() at once, and against the driver's max_channel_length before a slot is taken.
async with adorm.listen("orders", "invoices") as listener:
async for event in listener:
print(event.channel, event.payload) # orders 42adorm.Listener
Listener(*channels, session=None)
Backend notifications (PostgreSQL's LISTEN / NOTIFY) received on a slot the async with block holds.
- Entering acquires a slot for
sessionfrom theTransactionRunner, asatomic()does: the acquisition timeout raisesSessionTimeout, a flow holding another slotDeadlockError. - A driver that cannot listen (
Driver.check_listen():notifymissing from itsbackend_events, Django'sTestCase) raisesNotSupportedErrorbefore a slot is taken. - The connection then listens on each channel; a failure or cancellation there releases the slot and raises.
- While the block runs, its flow holds the slot; an operation of that flow that needs another slot of a full pool raises
DeadlockError. - Leaving the block releases the slot. A slot whose connection listens on a channel closes it (
CloseReason.LISTENING); otherwise the connection returns to the pool. - Only the task that entered the block may leave it. A listener is entered once.
| Member | Does |
|---|---|
async get() | the next notification, BackendEvent("notify", channel, payload), waiting for it. Notifications arrive in the order the connection received them; callbacks and other backend events of the slot go to the pool's EventBus. A cancelled wait loses no notification. Once the connection is lost (CONNECTION_LOST), the slot closes or its pool closes, the notifications received before are returned first, then it raises InterfaceError in every waiting call and every later one. Several tasks may wait at once; each notification goes to one of them |
async for event in listener | get() in a loop; ends with the InterfaceError of a lost connection |
async listen(*channels) | listens on channels too; a channel already listened on is skipped |
async unlisten(*channels) | stops listening on channels; a channel not listened on is skipped. Notifications already received stay queued |
channels | the channels the connection listens on (DriverConnection.listening), in the order they were added; a channel of a cancelled listen() counts while the connection counts it |
lease | the SlotLease of the held slot; None outside the block |
session | the session the slot is acquired for |
async close() | releases the slot, as leaving the block does; a get() another task waits in raises InterfaceError. A second call does nothing |
get(), listen() and unlisten() outside the block raise InterfaceError.
async def watch_orders():
async with adorm.listen("orders") as listener:
await listener.listen("invoices")
while True:
try:
event = await listener.get()
except adorm.InterfaceError:
return # connection lost: the caller listens again
if event.channel == "orders":
await refresh_order(int(event.payload))