JournalEngineering

Field guide / 5 min read

OpenAI Responses webhooks: verify, enqueue, and reconcile background jobs

A production pattern for signature verification, idempotent event acceptance, queued processing, terminal states, and background Response reconciliation.

Aug 24, 20265 min readBy ISH Team
OpenAI Responses webhooks: verify, enqueue, and reconcile background jobs
Advertisement

OpenAI Responses webhooks: verify, enqueue, and reconcile background jobs

Background mode lets an OpenAI Responses API request continue after the initiating HTTP request has returned. When the work reaches a terminal state, OpenAI can send a webhook event such as response.completed, response.failed, response.cancelled, or response.incomplete.

The receiver has a narrow job: authenticate the event, record it once, and hand processing to a queue. A worker can then retrieve the Response, update the application job, and notify the user. Slow database queries, model-output parsing, and downstream calls stay away from the public webhook endpoint.

Store a local job before starting the Response

Create the application job first, then start the model call with background: true. Save the returned Response ID on the same job record.

const job = await db.jobs.create({
  kind: "repository-summary",
  status: "starting",
  requestedBy: userId,
});

const response = await openai.responses.create({
  model: "gpt-5.6-terra",
  input: buildRepositoryPrompt(repository),
  background: true,
  metadata: { job_id: job.id },
});

await db.jobs.update(job.id, {
  status: "running",
  responseId: response.id,
});

The local job ID is the product-facing identifier. The Response ID identifies provider work. Keeping both allows support staff to trace a user request without exposing a provider identifier as the application's primary key. Do not put secrets or private payloads in metadata.

A Response can start before the application saves its ID. Store enough information to reconcile a job if the second write fails, and make the start operation safe to retry according to your own application rules. The agent tracing guide covers useful correlation fields.

Verify the raw request body

The official OpenAI Node SDK provides client.webhooks.unwrap(), which verifies the signature and parses the event. It requires the raw JSON string. Parsing the request body before verification changes the input and breaks the verification flow.

import OpenAI from "openai";

const openai = new OpenAI({
  webhookSecret: process.env.OPENAI_WEBHOOK_SECRET,
});

export async function POST(request: Request) {
  const body = await request.text();

  let event;
  try {
    event = openai.webhooks.unwrap(body, request.headers);
  } catch {
    return new Response("Invalid signature", { status: 400 });
  }

  await acceptWebhookEvent(event);
  return new Response("ok", { status: 200 });
}

Keep the webhook secret in the deployment secret store and separate it from the API key. Never log the raw body or signature headers. Logs need the event ID, event type, Response ID, acceptance result, and an internal trace ID.

The SDK also exposes verifySignature() when verification and JSON parsing need to happen separately. unwrap() is simpler when the endpoint only needs a verified event object.

Make event acceptance idempotent

Webhook event objects include a unique event ID. Put a unique constraint on that ID and use a transaction to record the event and create the queue job. If the insert conflicts, return success because the event has already been accepted.

create table openai_webhook_events (
  event_id text primary key,
  event_type text not null,
  response_id text not null,
  received_at timestamptz not null default now(),
  processed_at timestamptz
);

Idempotent acceptance prevents repeated delivery, worker retries, or an operator replay from applying the same state transition twice. The worker should also update the application job conditionally. A completed job must not return to running because an older event is processed later.

Model the application states explicitly:

starting -> running -> completed
                    -> failed
                    -> cancelled
                    -> incomplete

Reject invalid signatures. For a verified event type that the application does not yet handle, record it, mark it as unhandled, and return success. This avoids repeated processing attempts while preserving evidence for an update.

Retrieve the Response in the worker

Webhook payloads identify the Response. Treat the retrieved Response as the current provider record rather than building product state from the webhook envelope alone.

const response = await openai.responses.retrieve(responseId);

switch (response.status) {
  case "completed":
    await finishJob(jobId, response.output);
    break;
  case "failed":
  case "cancelled":
  case "incomplete":
    await closeJob(jobId, response.status, response.error);
    break;
  default:
    await retryLater(jobId, response.status);
}

Responses can contain multiple output items. Do not assume the first item is final text. Inspect item types, preserve tool-call evidence needed by the product, and use the SDK's text helper only when the workflow expects text. The tool-interface guide explains how to keep tool results reviewable.

An incomplete result needs a product decision. It may contain useful partial output, but partial output should not be presented as a complete result without an explicit policy. Record incomplete_details, decide whether the job is retryable, and show the user a truthful state.

Keep the endpoint fast and observable

Signature verification and one transactional write belong on the request path. Response retrieval, output validation, notifications, analytics, and third-party calls belong in workers. Set a short endpoint timeout and alert on verification failures, database errors, queue failures, and growing event-processing lag.

Useful measurements include accepted events by type, duplicate event IDs, time from event receipt to worker completion, jobs with no terminal event, and terminal events with no matching local job. Redact payloads and user content from logs. The Zero Data Retention audit explains why local queues and logs remain part of the data path even when provider-side retention is restricted.

Add a reconciliation task that queries local jobs stuck in starting or running, retrieves known Response IDs, and applies the same state transition used by the webhook worker. This repairs missed delivery and local processing failures without creating a second interpretation of status.

Test failures before enabling traffic

Exercise valid and invalid signatures, a parsed-body mistake, repeated event IDs, unknown verified event types, database outages, queue failures, worker retries, missing local jobs, and all four terminal Response states. Confirm that each accepted event produces at most one state transition and that reconciliation reaches the same result as normal delivery.

For applications that expose several model providers through ISH API, keep the provider event adapter thin and translate it into one internal job-state model. ISH chat can help compare foreground and background behavior across models without coupling the receiver to presentation code.

Run the raw-body verification test in the deployed framework before enabling the webhook endpoint. Check every middleware layer that can parse or rewrite a request. Release when signatures are verified, event IDs are unique, workers retrieve current Response state, all terminal statuses close the local job, and reconciliation repairs an interrupted path.

Sources

#OpenAI API#Responses API#webhooks#background jobs#reliability
Advertisement

Keep reading

Related stories

Browse the archive