v26.4.18
v26.4.18
External database drivers instrument themselves
Ory Polis no longer registers OpenTelemetry instrumentation for DynamoDB, MongoDB or Redis, and
@opentelemetry/instrumentation-aws-sdk, @opentelemetry/instrumentation-mongodb and @opentelemetry/instrumentation-redis are
no longer dependencies of @boxyhq/saml-jackson. Database spans for these databases stop with this release; a driver package that
wants them registers its own instrumentation, as described in the External databases documentation. Spans for the built-in SQL
databases are unchanged.
A page read by pageToken no longer gets slower the deeper it is
On the SQL databases, a list call that continues from a pageToken now starts reading at that position. Before, the database
still read and discarded every record ahead of it, as it does for pageOffset, so walking a large namespace or directory page by
page took longer with every page. With one million records, a page 100,000 records deep went from 19 ms to 1 ms on PostgreSQL and
from seconds to 2 ms on MariaDB; MySQL already read such a page in 2 ms. Pages requested by pageOffset are unchanged, so prefer
pageToken for deep or complete walks.
PostgreSQL, CockroachDB and YugabyteDB get a new index for this, _jackson_store_namespace_created_key on
jackson_store ("namespace", "createdAt", "key"), which replaces _jackson_store_namespace_created. Deployments that run
migrations themselves (DB_MANUAL_MIGRATION=true) need to run the two new migrations; on PostgreSQL and YugabyteDB the index is
built concurrently, and the old one is dropped only once the new one is valid. Deployments that let Polis update the schema at
startup get the index built without CONCURRENTLY on PostgreSQL and YugabyteDB, which holds up writes to jackson_store while it
is built. For a large table, set DB_MANUAL_MIGRATION=true and run the migrations before starting the new version.
The MongoDB driver package gets the same treatment. Its lists were sorted without an index in their order, so every page read and
sorted the whole namespace or index match. It now creates two indexes at startup, namespace_created on
{ namespace, createdAt, _id } and indexes_created on { indexes, createdAt, _id }, and a page reads only its own documents.
The first start on a collection that has documents waits for the indexes to be built. The namespace_1 and indexes_1 indexes of
earlier versions are no longer used and can be dropped once every instance is upgraded.
List endpoints page by token on every database
Every paginated list endpoint now returns a pageToken when a page follows, whatever the database. Until now only DynamoDB did;
on every other database a client walked a list with pageOffset, which makes the database skip over every record before the
offset on each request and gets slower the deeper the walk goes. Passing the pageToken back reads the next page from where the
previous one ended, at the same cost on every page, and is not disturbed by records deleted in the meantime.
pageOffset keeps working as before. A pageToken is only meaningful to the Ory Polis version and database that issued it; one
kept across an upgrade is rejected with a 400 and the walk is started again without it. The SCIM endpoints, which page by
startIndex as the SCIM standard requires, are unchanged.
The Admin Portal pages its lists by token
Every list in the Admin Portal (connections, directories, users, groups, group members, webhook events, setup links, SSO traces
and Identity Federation apps) now asks for a page with the pageToken the page before it returned, and no longer by offset, so a
page deep in a long list loads as fast as the first. The address of a page carries its token, which makes a reload, a shared link
and the browser's back button read the same page the same way. Next is offered only when the server has another page. Previous
returns through the pages you came from; on a page opened directly from a link, it goes to the first page.
GET /api/v1/dsync/product and the Admin Portal's own directory list now return a pageToken on every database. They returned
one on DynamoDB only, so the list of directories could not be continued by token on the others. pageOffset works on the endpoint
as before, and so does directories.getAll and directories.filterBy in the npm library.