New Ory Agent Security is now live! Claim your complimentary test drive. Get Started!

Skip to main content

API Reference

The following guides provide information about the APIs and SDKs that are available for the Directory Sync service.

Directory​

Below are the properties of a directory connection.

Properties​

  • name: The name of the directory
  • tenant: The tenant ID of the tenant you want to create the directory for.
  • product: The product ID of the product you want to create the directory for.
  • type: The directory provider type. See the Directory Providers for more information.
  • webhook.endpoint: The webhook URL to which the directory connection will POST the events.
  • webhook.secret: The webhook secret used to sign the webhook payload.
  • log_webhook_events: Indicate if the webhook events should be logged.
  • deactivated: Indicate if the directory connection is deactivated. A deactivated directory refuses SCIM requests with a 403 SCIM error, Directory is deactivated, so the IdP reports its sync as failing, and a deactivated Google Workspace directory is not synced.
note

How the IdP handles the refused changes depends on the IdP:

  • Microsoft Entra ID keeps them and retries them, and puts the provisioning job in quarantine once enough of them fail, retrying once a day. Activate the directory again within four weeks and Entra catches up on its own. After four weeks in quarantine, Entra disables the provisioning job: activating the directory is then not enough, and an admin has to restart provisioning in Entra.
  • Okta keeps them as failed tasks on the Tasks page of the Admin Console. After you activate the directory again, retry them there.

To pause a directory without the IdP reporting errors, turn off provisioning in the IdP as well.

info

The tenant and product shouldn't contain the : character since we use it as a delimiter internally.

Initialize Directory Sync​

const opts = {
  externalUrl: "https://my-cool-app.com",
  db: {
    engine: "mongo",
    url: "mongodb://localhost:27017/my-cool-app",
  },
}

let directorySyncController

// Please note that the initialization of @boxyhq/saml-jackson is async, you cannot run it at the top level.
async function init() {
  const jackson = await require("@boxyhq/saml-jackson").controllers(opts)

  directorySyncController = jackson.directorySyncController
}

Create a new directory​

Create a new directory connection.

Request​

const { data, error } = await directorySyncController.directories.create({
  name: "App",
  tenant: "ory",
  product: "polis",
  type: "onelogin-scim-v2",
  webhook_url: "https://my-cool-app.com/webhook",
  webhook_secret: "my-secret",
})

Response​

{
  "data": {
    "id": "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
    "name": "App",
    "tenant": "ory",
    "product": "polis",
    "type": "onelogin-scim-v2",
    "log_webhook_events": false,
    "scim": {
      "path": "/api/scim/v2.0/58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
      "secret": "IJzAoevjD_liiiy-VkDtXg",
      "endpoint": "http://localhost:5225/api/scim/v2.0/58b5cd9dfaa39d47eb8f5f88631f9a629a232016"
    },
    "webhook": {
      "endpoint": "https://my-cool-app.com/webhook",
      "secret": "my-secret"
    }
  },
  "error": null
}

Get directories​

Get the list of directories for the given tenant and product. A tenant can have multiple directories for same or different products.

The response contains every directory for that tenant and product. This endpoint takes no pagination parameters and returns the complete list in one response.

Request​

const tenant = "ory"
const product = "polis"

const { data, error } = await directorySyncController.directories.getByTenantAndProduct(tenant, product)

Response​

{
  "data": [
    {
      "id": "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
      "name": "App",
      "tenant": "ory",
      "product": "polis",
      "type": "onelogin-scim-v2",
      "log_webhook_events": false,
      "scim": {
        "path": "/api/scim/v2.0/58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
        "secret": "IJzAoevjD_liiiy-VkDtXg",
        "endpoint": "http://localhost:5225/api/scim/v2.0/58b5cd9dfaa39d47eb8f5f88631f9a629a232016"
      },
      "webhook": {
        "endpoint": "https://my-cool-app.com/webhook",
        "secret": "my-secret"
      }
    }
  ],
  "error": null
}

Get directories by product​

Get the directories for a product across all tenants. The product query parameter is required: a request without it is rejected with a 400 and the message Please provide a product.

Page through the directories with pageOffset and pageLimit. On DynamoDB, which pages by token, the response also carries a pageToken while more directories follow; pass it back as the pageToken query parameter to get the next page.

Request​

const { data, error, pageToken } = await directorySyncController.directories.filterBy({
  product: "polis",
  pageOffset: 0,
  pageLimit: 10,
})

Response​

The data array holds directory objects in the same shape as Get directories.

{
  "data": [
    {
      "id": "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
      "name": "App",
      "tenant": "ory",
      "product": "polis",
      "type": "onelogin-scim-v2"
    }
  ],
  "error": null
}

Get a directory​

Get the details of a directory by its unique id.

Request​

const directoryId = "58b5cd9dfaa39d47eb8f5f88631f9a629a232016"

const { data, error } = await directorySyncController.directories.get(directoryId)

Response​

{
  "data": {
    "id": "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
    "name": "App",
    "tenant": "ory",
    "product": "polis",
    "type": "onelogin-scim-v2",
    "log_webhook_events": false,
    "scim": {
      "path": "/api/scim/v2.0/58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
      "secret": "IJzAoevjD_liiiy-VkDtXg",
      "endpoint": "http://localhost:5225/api/scim/v2.0/58b5cd9dfaa39d47eb8f5f88631f9a629a232016"
    },
    "webhook": {
      "endpoint": "https://my-cool-app.com/webhook",
      "secret": "my-secret"
    }
  },
  "error": null
}

List directory users​

List all the users in a directory.

Request​

const tenant = "ory"
const product = "polis"

const { data, error } = await directorySyncController.users.withTenantAndProduct(tenant, product).getAll()

Response​

{
  "data": [
    {
      "id": "6296f71e-15fd-4af4-86ee-d6623b3ef1a4",
      "first_name": "Aswin",
      "last_name": "Venugopal",
      "email": "[email protected]",
      "active": true,
      "raw": {
        "schemas": ["urn:ietf:params:scim:schemas:core:2.0:User"],
        "userName": "[email protected]",
        "name": {
          "givenName": "Aswin",
          "familyName": "Venugopal",
          "honorificPrefix": "Sir"
        },
        "emails": [
          {
            "primary": true,
            "value": "[email protected]",
            "type": "work"
          }
        ],
        "displayName": "Aswin Venugopal",
        "addresses": [
          {
            "primary": true,
            "postalCode": "123"
          }
        ],
        "locale": "en-US",
        "externalId": "00u34iw1hm16RmjS95d7",
        "groups": [],
        "active": true,
        "id": "6296f71e-15fd-4af4-86ee-d6623b3ef1a4"
      }
    },
    {
      "id": "ebc31d6e-7d62-4f81-b9e5-eb5f1a04ee92",
      "first_name": "Kiran",
      "last_name": "Krishnan",
      "email": "[email protected]",
      "active": true,
      "raw": {
        "schemas": ["urn:ietf:params:scim:schemas:core:2.0:User"],
        "userName": "[email protected]",
        "name": {
          "givenName": "Kiran",
          "familyName": "Krishnan"
        },
        "emails": [
          {
            "primary": true,
            "value": "[email protected]",
            "type": "work"
          }
        ],
        "displayName": "Kiran Krishnan",
        "addresses": [
          {
            "primary": true,
            "region": "Kerala"
          }
        ],
        "locale": "en-US",
        "externalId": "00u3e3cmpdDydXdzV5d7",
        "groups": [],
        "active": true,
        "id": "ebc31d6e-7d62-4f81-b9e5-eb5f1a04ee92"
      }
    }
  ],
  "error": null
}

Get a directory user​

Get the details of a directory user.

Request​

const tenant = "ory"
const product = "flex"
const userId = "ebc31d6e-7d62-4f81-b9e5-eb5f1a04ee92"

const users = await directorySyncController.users.withTenantAndProduct(tenant, product).get(userId)

Response​

{
  "data": {
    "id": "ebc31d6e-7d62-4f81-b9e5-eb5f1a04ee92",
    "first_name": "Kiran",
    "last_name": "Krishnan",
    "email": "[email protected]",
    "active": true,
    "raw": {
      "schemas": ["urn:ietf:params:scim:schemas:core:2.0:User"],
      "userName": "[email protected]",
      "name": {
        "givenName": "Kiran",
        "familyName": "Krishnan"
      },
      "emails": [
        {
          "primary": true,
          "value": "[email protected]",
          "type": "work"
        }
      ],
      "displayName": "Kiran Krishnan",
      "addresses": [
        {
          "primary": true,
          "region": "Kerala"
        }
      ],
      "locale": "en-US",
      "externalId": "00u3e3cmpdDydXdzV5d7",
      "groups": [],
      "active": true,
      "id": "ebc31d6e-7d62-4f81-b9e5-eb5f1a04ee92"
    }
  },
  "error": null
}

List directory groups​

List all the groups in a directory.

Request​

const tenant = "ory"
const product = "polis"

const users = await directorySyncController.groups.withTenantAndProduct(tenant, product).getAll()

Response​

{
  "data": [
    {
      "id": "44d08c0e-d185-4a5e-80a6-b47a717ffaa5",
      "name": "Developers",
      "raw": {
        "schemas": ["urn:ietf:params:scim:schemas:core:2.0:Group"],
        "displayName": "Developers",
        "members": [
          {
            "value": "6296f71e-15fd-4af4-86ee-d6623b3ef1a4",
            "display": "[email protected]"
          },
          {
            "value": "ebc31d6e-7d62-4f81-b9e5-eb5f1a04ee92",
            "display": "[email protected]"
          }
        ],
        "id": "44d08c0e-d185-4a5e-80a6-b47a717ffaa5"
      }
    }
  ],
  "error": null
}

Get a directory group​

Get the details of a directory group.

Request​

const tenant = "ory"
const product = "polis"
const groupId = "44d08c0e-d185-4a5e-80a6-b47a717ffaa5"

const users = await directorySyncController.groups.withTenantAndProduct(tenant, product).get(groupId)

Response​

{
  "data": {
    "id": "44d08c0e-d185-4a5e-80a6-b47a717ffaa5",
    "name": "Developers",
    "raw": {
      "schemas": ["urn:ietf:params:scim:schemas:core:2.0:Group"],
      "displayName": "Developers",
      "members": [
        {
          "value": "6296f71e-15fd-4af4-86ee-d6623b3ef1a4",
          "display": "[email protected]"
        },
        {
          "value": "ebc31d6e-7d62-4f81-b9e5-eb5f1a04ee92",
          "display": "[email protected]"
        }
      ],
      "id": "44d08c0e-d185-4a5e-80a6-b47a717ffaa5"
    },
    "members": [
      {
        "group_id": "44d08c0e-d185-4a5e-80a6-b47a717ffaa5",
        "user_id": "6296f71e-15fd-4af4-86ee-d6623b3ef1a4"
      },
      {
        "group_id": "44d08c0e-d185-4a5e-80a6-b47a717ffaa5",
        "user_id": "ebc31d6e-7d62-4f81-b9e5-eb5f1a04ee92"
      }
    ]
  },
  "error": null
}

List group members​

Get the members of a directory group. Only user IDs are returned.

Page through the members with pageOffset and pageLimit. On DynamoDB, which pages by token, the response also carries a pageToken while more members follow; pass it back as the pageToken query parameter to get the next page.

(Introduced in v1.20.6)

Request​

await directorySyncController.groups.withTenantAndProduct("ory", "polis").getGroupMembers({
  groupId: "44d08c0e-d185-4a5e-80a6-b47a717ffaa5",
  pageOffset: 0,
  pageLimit: 10,
})

Response​

{
  "data": [
    {
      "user_id": "107130779649255553459"
    },
    {
      "user_id": "107471187046551199726"
    }
  ]
}

List webhook event logs​

Get the webhook event logs of a directory. Ory Polis logs the events it sends to a directory's webhook when the directory has log_webhook_events set. Identify the directory with directoryId, or with tenant and product.

Request​

await directorySyncController.webhookLogs.withTenantAndProduct("ory", "polis").getAll({
  directoryId: "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
  pageOffset: 0,
  pageLimit: 10,
})

Response​

{
  "data": [
    {
      "id": "8b3a1f6e-7a9d-4a3f-9a6b-0e5c2d7f1b44",
      "webhook_endpoint": "https://example.com/webhook",
      "created_at": "2024-03-05T17:06:26.074Z",
      "status_code": 200,
      "delivered": true,
      "payload": {
        "directory_id": "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
        "event": "user.created",
        "tenant": "ory",
        "product": "polis",
        "data": {}
      }
    }
  ]
}

Get a webhook event log​

Get one webhook event log by its ID. Identify the directory with directoryId, or with tenant and product. data is null when no event log has that ID.

Request​

await directorySyncController.webhookLogs.withTenantAndProduct("ory", "polis").get("8b3a1f6e-7a9d-4a3f-9a6b-0e5c2d7f1b44")

Response​

The event log, under data, in the same shape as an entry of List webhook event logs.

Delete webhook event logs​

Delete every webhook event log of a directory.

Request​

await directorySyncController.webhookLogs
  .withTenantAndProduct("ory", "polis")
  .deleteAll("58b5cd9dfaa39d47eb8f5f88631f9a629a232016")

Response​

{
  "data": null
}

Google Directory Sync​

Google Directory Sync specific properties:

  • google_domain: The Google Workspace domain name.
  • google_access_token: The Google Admin access token.
  • google_refresh_token: The Google Admin refresh token.

Ory Polis requires following API scopes:

  • https://www.googleapis.com/auth/admin.directory.group.member.readonly
  • https://www.googleapis.com/auth/admin.directory.group.readonly

1. Get Authentication URL​

Get the URL to authenticate with admin credentials. Google will prompt the user to authenticate with the admin credentials and authorize the application to access the Google Workspace directories.

Request​

await directorySyncController.google.generateAuthorizationUrl({
  directoryId: "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
  token: "<the directory's google_authorization_token>",
})

Response​

{
  "data": {
    "authorizationUrl": "https://accounts.google.com/..."
  },
  "error": null
}

Who can start the authorization​

Starting the Google consent flow installs credentials on the directory, so it takes more than the directory id. Every Google directory holds an unguessable google_authorization_token, issued when the directory is created. The google_authorization_url returned for the directory already carries it, together with the directory id, so it can be copied from the Admin Portal or the setup link and handed to the Google Workspace administrator as it is. A request to the authorize endpoint without the directory's current token is refused.

Completing the consent screen replaces the token, and so does setting google_access_token or google_refresh_token through the directory API. Every URL shared before stops working, as does any consent flow still underway, and the next URL the Admin Portal shows carries the new token. Ory Polis therefore does not require a session on the authorize endpoint: the person who completes the consent is the customer's Google Workspace administrator, and the token in the URL is what authorizes them.

How long an authorization URL lasts​

Each authorization URL carries an opaque state value that Ory Polis stores against the directory. The callback reads the directory from that stored value, so a URL can only install tokens on the directory it was generated for. The stored value also records which authorization token started the flow, so a flow started before the token was replaced does not complete.

A directory has one authorization URL at a time, and it is always the most recent one. This also applies to the authorization link the Admin Portal shows for a Google directory, which points at the same endpoint.

  • Asking for a URL issues a new one and retires the previous one. A URL you generated earlier stops working.
  • A URL works once. Completing the Google consent screen uses it up.
  • An unused URL expires 10 minutes after it was issued.

Generate the URL when you are ready to hand it to the Google Workspace administrator, rather than saving one to send later. If the callback reports an invalid or expired state, generate the URL again and share the new one.

2. Get Access Token​

Get the access token from the authorization code.

Request​

await directorySyncController.google.getAccessToken({
  directoryId: "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
  code: "authorization-code",
})

Response​

{
  "data": {
    "access_token": "ya29.a0AWY7CknlXtELCHzdKuS...",
    "refresh_token": "1//03MJpa-zseicbCgYIARAAGAMSNwF-...",
    "scope": "https://www.googleapis.com/auth/admin.directory.group.member.readonly https://www.googleapis.com/auth/admin.directory.group.readonly",
    "token_type": "Bearer",
    "expiry_date": 1684343934305
  },
  "error": null
}

3. Set Access Token to Directory​

Set the access token and refresh token to the directory. We'll use the access token to make requests to the Google Directory API.

Request​

await directorySyncController.google.setToken({
  directoryId: "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
  accessToken: "ya29.a0AWY7CknlXtELCHzdKuS...",
  refreshToken: "1//03MJpa-zseicbCgYIARAAGAMSNwF-...",
})

Response​

{
  "data": {
    "id": "58b5cd9dfaa39d47eb8f5f88631f9a629a232016",
    "name": "App",
    "tenant": "ory",
    "product": "polis",
    "type": "google",
    "google_domain": "ory.com",
    "google_access_token": "ya29.a0AWY7CknlXtELCHzdKuS...",
    "google_refresh_token": "1//03MJpa-zseicbCgYIARAAGAMSNwF-..."
  },
  "error": null
}

4. Sync Directory​

Ory Polis can be configured to sync your Google Workspace directory.

const callback = (event: DirectorySyncEvent) => {
  console.log(event);
};

await directorySyncController.sync(callback);

You'll ideally want to run the sync on a schedule (e.g. every 2 hours). You can use a cron job to invoke this URL on a schedule.

See more information about the Directory Sync Event below.

Handle the Requests from Identity Providers​

Make sure your application can handle the requests from Identity Providers.

Routes​

Typically, you will want to add the following routes to your application to handle the requests. However, the Methods can vary for some Identity Providers.

RouteMethodsEvent
/UsersPOSTA new user has been assigned to a SCIM app
/Users/:idPUT, PATCHA user information has been updated
/Users/:idDELETEA user has been removed from a SCIM app
/GroupsPOSTA new group has been pushed
/Groups/:idPUTGroup name has been changed
/Groups/:idPATCHGroup name or membership has been changed
/Groups/:idDELETEGroup has been removed

Every request must carry the directory's SCIM secret as a bearer token. A request with a missing or wrong token, or for a directory ID that does not exist, answers 401 with a SCIM error body whose detail is Unauthorized.

Group membership PATCH operations​

A PATCH on /Groups/:id applies its Operations in order. add and remove emit a group.user_added or group.user_removed event for every listed member, whether or not the membership changed. replace and clearing the membership compare against the current membership first and emit events only for the members that are added or removed.

OperationEffect
add with path: "members"Adds the listed members.
remove with path: "members" and a valueRemoves the listed members.
remove with a path of members[value eq "<id>"]Removes the member selected by the filter.
remove with path: "members" and no valueClears the membership.
replace with path: "members"Replaces the full membership: members not in the list are removed, new ones are added.
replace without a path and a value carrying displayName and/or membersRenames the group and/or replaces the full membership.

Replacing or clearing the membership is not bounded by DSYNC_MAX_INLINE_GROUP_MEMBERS; the current membership is processed page by page.

A remove without a path is rejected with 400 and scimType: "noTarget". A members value that is not a list of objects with a string value is rejected with 400 and scimType: "invalidValue". Invalid operations are rejected before any operation in the request is applied.

User Requests​

const { data, status } = await directorySyncController.requests.handle(request)

The shape of the request should be as follows:

{
  method: HTTPMethod;
  body?: any;
  directoryId: string;
  resourceId: string;
  resourceType: "users",
  apiSecret: string;
  query: {
    count?: number;
    startIndex?: number;
    filter?: string;
  };
};

Group Requests​

Handling the group requests is similar to handling the user requests.

const { data, status } = await directorySyncController.requests.handle(request)

The shape of the request should be as follows:

{
  method: HTTPMethod;
  body?: any;
  directoryId: string;
  resourceId: string;
  resourceType: "groups",
  apiSecret: string;
  query: {
    count?: number;
    startIndex?: number;
    filter?: string;
  };
};

Error responses​

When a request fails, status holds the HTTP status code and data holds a SCIM error body as defined in RFC 7644 section 3.12. Return both to the identity provider unchanged.

{
  "schemas": ["urn:ietf:params:scim:api:messages:2.0:Error"],
  "status": "409",
  "scimType": "uniqueness",
  "detail": "User already exists"
}
AttributeDescription
schemasAlways ["urn:ietf:params:scim:api:messages:2.0:Error"].
statusThe HTTP status code as a string. It matches the numeric status returned next to data.
scimTypeA SCIM error keyword, present only when the specification defines one for the condition. uniqueness marks a duplicate user or group (409). On a 400 for a PATCH request: invalidSyntax marks a body that is not an object, an Operations list that is missing or empty, or an operation that is not an object with a string op; invalidValue marks an unsupported op, an add or replace without a value, or a malformed group members payload; invalidPath marks a path that is not a non-empty string; noTarget marks a remove without a path.
detailA human-readable description of the error. Match on status and scimType rather than on this text.

Concurrent updates of one user or group​

PATCH /Users/:id, PUT /Groups/:id and a rename through PATCH /Groups/:id each read the stored record, merge the request into it and write the whole record back. The read is not part of the write transaction. When a database running at serializable isolation (CockroachDB, YugabyteDB, PostgreSQL at SERIALIZABLE) reports that the write lost a conflict with another transaction that updated the same record, Ory Polis replays the request a bounded number of times, reading the record again before each replay, so the change the other request committed is carried into the result rather than overwritten. A replay that succeeds answers 200. A PATCH on /Groups/:id with several operations applies each one to the record the previous operation wrote.

This narrows the window for a lost update but does not close it: the database reports a conflict only when the two write transactions overlap. An update that commits after this request's read and before its write transaction starts is not detected, and this request's write replaces it. Databases below serializable isolation, such as PostgreSQL at its default READ COMMITTED, report no conflict, and the last write wins as before.

The replay reads the record under the same checks as the original request. It answers 404 when the record is gone, whether because the request it lost to was the DELETE, or because the id has since been created again by another directory of the same tenant and product; that record is neither changed nor returned. The detail is User not found or Group not found.

A write that fails for any other reason, or that is still rejected after the last replay, is returned as a SCIM error body with the failure's status code. A user PATCH used to answer such a failure with 200 and an empty body.

Unrecognised routes​

A request whose path is not one of the routes above, including a request to the bare directory root /api/scim/v2.0/{directoryId}, answers 404 with a SCIM error body whose detail is Not found.

Callback Function​

You can optionally pass a callback function as a second parameter to the handle method requests.handle. Ory Polis will invoke the callback with the event object as the first argument after handling the request. You can use the event object to determine the action to take.

const callback = (event: DirectorySyncEvent) => {
  console.log(event);
};

const { data, status } = await directorySyncController.requests.handle(
  request,
  callback
);

Batch processing events​

You can enable batch processing of directory sync events instead of receiving events in real-time on your webhook endpoint. Once enabled, Ory Polis will queue the events and process them in batches. The batch size can be configured using the DSYNC_WEBHOOK_BATCH_SIZE environment variable.

With batching, the events of a directory reach its webhook in the order they were made, and a failed delivery is retried:

  • Each run sends the queued events oldest first, the events of one directory together in one request.
  • When a directory's delivery fails, or the directory cannot be read, its events stay queued and none of its later events is sent ahead of them. A failed delivery is retried on a schedule that starts at 30 seconds, doubles with each attempt and settles at one hour; set DSYNC_WEBHOOK_BATCH_RETRY_INTERVAL to change the start. A 400 or 422 response moves the events to a dead-letter store instead of retrying them.
  • The other directories go on, so one failing webhook does not hold back the rest.

Ideally run the events processing endpoint on a schedule using cron job.

curl -X POST \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  http://localhost:5225/api/v1/dsync/cron/process-events

Alternatively you can set the DSYNC_WEBHOOK_BATCH_CRON_INTERVAL env var.

Ory Network

The largest Identity and Access Management network in the world. So you can get back to building your business.

Sign up for free