Drıftstack docs
Docs

Task recipes

Each recipe is a whole program you can copy. They use the TypeScript SDK (npm install @driftstack/sdk) and run as ES modules on Node 18 or later. Every TypeScript sample on this page is compiled against the SDK whenever these docs change, so the calls and fields are real.

Every recipe here, and every sample on Driftstack for AI agents, reads its settings from the environment:

Variable What it holds
DRIFTSTACK_API_KEY An API key with read and write.
DRIFTSTACK_OWNER_KEY An API key with account_owner, to save a proxy.
DRIFTSTACK_WEBHOOK_SECRET The secret POST /v1/webhooks showed you once.
PROXY_ID The id of one of your saved proxies. Every session needs one.
PROXY_PASSWORD The password of the proxy you save.
PROFILE_ID A saved profile, for the recipes that reuse one.
SESSION_ID An agent session that is open and ready, where a recipe needs one.
PORTAL_PASSWORD The password the form recipe registers as a session secret.
SHOP_PASSWORD The password the session-secret sample registers.

Read Driftstack for AI agents first: it explains the rules these recipes follow.

Start a session and extract a fact

The simplest job: open a website and get one fact back. Ask in plain words and use answer.

import { randomUUID } from 'node:crypto';
import { Driftstack } from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });

const session = await client.agentSessions.create(
  { proxy_id: process.env.PROXY_ID! },
  { idempotencyKey: randomUUID() },
);
try {
  for (let i = 0; ; i++) {
    const s = await client.agentSessions.get(session.id);
    if (s.status === 'closed') throw new Error(`could not start: ${s.closed_reason}`);
    if (s.status === 'active' && s.ready !== false) break;
    if (i === 60) throw new Error('not ready after two minutes');
    await new Promise((resolve) => setTimeout(resolve, 2_000));
  }

  const reply = await client.agentSessions.message(
    session.id,
    'Open https://example.com/pricing and tell me the monthly price of the Pro plan.',
    { idempotencyKey: randomUUID() },
  );
  if (reply.kind === 'plan-executed' && reply.answer !== undefined && reply.notice === undefined) {
    console.log(reply.answer);
  } else if (reply.kind === 'plan-executed') {
    console.log('No answer:', reply.notice ?? reply.answer_unavailable ?? 'a step failed');
  } else {
    console.log('The agent did not run the task:', reply.kind);
  }
} finally {
  await client.agentSessions.close(session.id);
}

When you already know where the text is, skip the AI and send an extract step. The text comes back in the step’s summary, after the words extracted text from <selector>: .

import { randomUUID } from 'node:crypto';
import { Driftstack } from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });
const sessionId = process.env.SESSION_ID!;

const run = await client.agentSessions.runSteps(
  sessionId,
  [
    { kind: 'navigate', url: 'https://example.com/pricing' },
    { kind: 'wait', condition: 'selector_visible', selector: '#pro .price', timeoutMs: 10_000 },
    { kind: 'extract', selector: '#pro .price' },
  ],
  { idempotencyKey: randomUUID() },
);
if (run.kind === 'plan-executed') {
  const read = run.results.at(-1);
  console.log(read?.kind === 'success' ? read.summary : 'no text');
}

The same flow with curl. The task’s answer streams back; the last event, response, holds the result.

# Create the session. Keep the "id" it returns as $ID.
curl -sS -X POST https://api.driftstack.dev/v1/agent-sessions \
  -H "Authorization: Bearer $DRIFTSTACK_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{ "proxy_id": "<your proxy id>" }'

# Send the task once the session is active.
curl -sS -N --max-time 3000 -X POST https://api.driftstack.dev/v1/agent-sessions/$ID/message \
  -H "Authorization: Bearer $DRIFTSTACK_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{ "user_message": "Open https://example.com/pricing and tell me the monthly price of the Pro plan." }'

# Close it.
curl -sS -X DELETE https://api.driftstack.dev/v1/agent-sessions/$ID \
  -H "Authorization: Bearer $DRIFTSTACK_API_KEY"

Fill in fields and submit

For a page you know, send exact steps. Register the password as a session secret, so it never appears in your steps, the transcript or any answer.

import { randomUUID } from 'node:crypto';
import { Driftstack } from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });
const sessionId = process.env.SESSION_ID!;

const password = await client.agentSessions.registerSecret(sessionId, {
  label: 'Portal password',
  secret: process.env.PORTAL_PASSWORD!,
  sites: ['portal.example.com'],
});

const run = await client.agentSessions.runSteps(
  sessionId,
  [
    { kind: 'navigate', url: 'https://portal.example.com/login' },
    // An empty value clears the field; typing adds to what it holds.
    { kind: 'interact', action: 'type', selector: '#email', value: '' },
    { kind: 'interact', action: 'type', selector: '#email', value: '[email protected]' },
    {
      kind: 'interact',
      action: 'type',
      selector: '#password',
      value: `{{credential:${password.handle}}}`,
    },
    { kind: 'interact', action: 'tap', selector: 'button[type="submit"]' },
    { kind: 'wait', condition: 'idle', timeoutMs: 10_000 },
  ],
  { idempotencyKey: randomUUID() },
);

if (run.kind === 'plan-executed') {
  for (const step of run.results) {
    if (step.kind === 'failure') console.log('failed:', step.reason);
    if (step.kind === 'confirmation_required') {
      // A purchase, a payment or an account deletion. A person decides.
      console.log(`needs a person: ${step.category} "${step.matchedText}"`);
    }
  }
  console.log(run.ok ? 'sent' : 'not sent: see the failed step');
}

A session secret is kept for 1 hour unless you set another time (at most 24 hours), and only in the server’s memory: after that time, or after a service restart, register it again before you send steps that use it. Once its time has passed, a secret is never typed again, not even one a step has already typed: register it again and use the new handle.

For a website you have not seen, send a task instead: “Open https://example.com/contact, fill in the name Ana Silva, the email [email protected] and the message ‘Please call me back’, then send it.” Keep passwords and card numbers out of that sentence: use a placeholder as above.

Handle a webhook

A small server that checks every delivery’s signature, drops repeats and answers fast. Store the signing secret from POST /v1/webhooks in DRIFTSTACK_WEBHOOK_SECRET.

import { createServer } from 'node:http';
import { verifyWebhookSignature } from '@driftstack/sdk';

const secret = process.env.DRIFTSTACK_WEBHOOK_SECRET!;
const seen = new Set<string>(); // use your database in production

interface DriftstackEvent {
  id: string;
  type: string;
  created_at: string;
  data: Record<string, unknown>;
}

createServer((req, res) => {
  const chunks: Buffer[] = [];
  req.on('data', (chunk: Buffer) => {
    chunks.push(chunk);
    // No delivery is this large: stop reading.
    if (chunks.reduce((total, c) => total + c.length, 0) > 1_000_000) req.destroy();
  });
  req.on('end', () => {
    void (async () => {
      try {
        const body = Buffer.concat(chunks); // the raw body, before any parsing
        const ok = await verifyWebhookSignature({
          body,
          header: req.headers['x-driftstack-signature'],
          secret,
        });
        if (!ok) {
          res.writeHead(401).end();
          return;
        }
        const event = JSON.parse(body.toString('utf8')) as DriftstackEvent;
        res.writeHead(200).end(); // answer within 10 seconds, then do the work
        if (seen.has(event.id)) return;
        seen.add(event.id);
        if (event.type === 'session.challenge_detected') {
          console.log('A check needs a person on session', event.data.session_id);
        }
      } catch (err) {
        // An error here must not end the process: answer 500 if nothing was sent.
        console.error('webhook handler failed', err);
        if (!res.headersSent) res.writeHead(500).end();
      }
    })();
  });
}).listen(8080);
  • The signature covers t. plus the raw body, and is checked against every v1= value, so it keeps working while you rotate the secret.
  • A delivery older than 5 minutes fails the check: that stops replays.
  • Deliveries can repeat and arrive out of order. Drop repeats by id.

Save and reuse a profile

Create a profile once and sign it in by hand in the desktop app. Then every run starts a new session on it, and the session stays signed in.

import { Driftstack } from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });

// Once. Keep the id as PROFILE_ID.
const profile = await client.profiles.create({ name: 'supplier-portal' });
console.log(profile.id);
import { randomUUID } from 'node:crypto';
import { appendFileSync, existsSync, readFileSync } from 'node:fs';
import { Driftstack, ProfileInUseError } from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });
const body = { proxy_id: process.env.PROXY_ID!, profile_id: process.env.PROFILE_ID! };
const key = randomUUID();

// The ids this program created, one per line. One file per program: never share it.
const MINE = 'profile-job-sessions.txt';
const mine = new Set(existsSync(MINE) ? readFileSync(MINE, 'utf8').split('\n') : []);

// closeHolder: true closes the session that holds the profile, but only when
// its id is in this program's own file (an earlier run left it open). Any
// other holder is never closed: it may be a person working in the desktop app.
// Leave it false if this program hands its sessions to a person.
async function startOnProfile(closeHolder = false) {
  try {
    return await client.agentSessions.create(body, { idempotencyKey: key });
  } catch (err) {
    if (!(err instanceof ProfileInUseError) || err.activeSessionId === undefined) throw err;
    const holder = err.activeSessionId;
    if (closeHolder && mine.has(holder)) {
      await client.agentSessions.close(holder);
      return client.agentSessions.create(body, { idempotencyKey: randomUUID() });
    }
    // The holder is not a session this program created. It is not ours to close: stop and say so.
    throw new Error(
      `Profile ${body.profile_id} is in use by session ${holder}. ` +
        'It may be in use, or left open by an earlier run: close it yourself only if nobody is using it, then run this again.',
      { cause: err },
    );
  }
}

const session = await startOnProfile();
try {
  appendFileSync(MINE, `${session.id}\n`); // write it down before anything else
  // … tasks or steps …
} finally {
  // Closing saves the profile's cookies and site data for the next run.
  await client.agentSessions.close(session.id);
}

A profile runs one session at a time. When another session holds it, the create answers 409 profile-in-use with that session’s id. The recipe never closes a session it did not create: it may be a person’s, in the desktop app. It stops and names the session, so a person can decide. With closeHolder set to true it closes the holder in one case only: the id is in the program’s own id file, so an earlier run of this program left it open. Pass true only when no other copy of this program is running, and never when the program hands its sessions to a person: it would close the other copy’s session, or the one the person is working in. Give the program an id file of its own. One file per program; never share it.

To notice that a website signed the profile out, add to your task: “If you see a sign-in page, stop and tell me.” Then alert a person. Do not send a password in a message.

Use a proxy

Save a proxy once with a key that has the account_owner scope, test it, and give its id to your jobs. Use socks5, openvpn or wireguard: an http proxy cannot carry a session.

import { randomUUID } from 'node:crypto';
import { Driftstack } from '@driftstack/sdk';

const owner = new Driftstack({ apiKey: process.env.DRIFTSTACK_OWNER_KEY! });

const proxy = await owner.egress.createProxy(
  {
    scheme: 'socks5',
    label: 'Amsterdam residential',
    host: 'proxy.example.net',
    port: 1080,
    username: 'user-1',
    password: process.env.PROXY_PASSWORD!,
  },
  { idempotencyKey: randomUUID() },
);

const test = await owner.egress.testProxy(proxy.id);
if (test.ok) {
  console.log(`works, ${test.latency_ms} ms; PROXY_ID=${proxy.id}`);
} else {
  console.log(`does not work: ${test.reason}`);
}

Then pass it on every create. Add stop_on_exit_ip_change: true if the session must stop when the proxy’s exit IP changes mid-run:

import { randomUUID } from 'node:crypto';
import { Driftstack } from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });

const session = await client.agentSessions.create(
  { proxy_id: process.env.PROXY_ID!, stop_on_exit_ip_change: true },
  { idempotencyKey: randomUUID() },
);
console.log(session.id, session.status);

The same with curl:

curl -sS -X POST https://api.driftstack.dev/v1/account/me/proxies \
  -H "Authorization: Bearer $DRIFTSTACK_OWNER_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{ "scheme": "socks5", "label": "Amsterdam residential", "host": "proxy.example.net", "port": 1080, "username": "user-1", "password": "<proxy password>" }'

curl -sS -X POST https://api.driftstack.dev/v1/account/me/proxies/$PROXY_ID/test \
  -H "Authorization: Bearer $DRIFTSTACK_OWNER_KEY"

Page through a list

On a website, read a page of results, tap Next, and stop when there is no Next link. A tap that finds nothing to tap is a failed step.

import { randomUUID } from 'node:crypto';
import { Driftstack } from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });
const sessionId = process.env.SESSION_ID!;

await client.agentSessions.runSteps(
  sessionId,
  [{ kind: 'navigate', url: 'https://example.com/orders' }],
  { idempotencyKey: randomUUID() },
);

const pages: string[] = [];
for (let page = 1; page <= 20; page++) {
  const run = await client.agentSessions.runSteps(
    sessionId,
    [
      { kind: 'extract', selector: '#results' },
      { kind: 'interact', action: 'tap', selector: 'a[rel="next"]' },
      { kind: 'wait', condition: 'idle', timeoutMs: 10_000 },
    ],
    { idempotencyKey: randomUUID() },
  );
  if (run.kind !== 'plan-executed') break;
  const [read, next] = run.results;
  if (read?.kind === 'success') pages.push(read.summary);
  if (next?.kind !== 'success') break; // no Next link: that was the last page
}
console.log(`${pages.length} pages read`);

In the API, every list is paged with a cursor. The SDKs’ iterate() walks every page for you. This one finds sessions an earlier run of this program left open and closes them. The list holds every session on the account, also the ones a person runs by hand in the desktop app (manual) and the ones other programs started, and no field on a session says which program started it. So write each id to a file when you create a session, and close only ids from that file:

import { appendFileSync, existsSync, readFileSync } from 'node:fs';
import { randomUUID } from 'node:crypto';
import { Driftstack } from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });

// The ids this program created, one per line. One file per program: never share it.
const MINE = 'cleanup-job-sessions.txt';
const mine = new Set(existsSync(MINE) ? readFileSync(MINE, 'utf8').split('\n') : []);

// Close what an earlier run left open. Never a session this program did not create.
for await (const session of client.agentSessions.iterate()) {
  if (session.status !== 'closed' && mine.has(session.id)) {
    console.log('closing', session.id);
    await client.agentSessions.close(session.id);
  }
}

const session = await client.agentSessions.create(
  { proxy_id: process.env.PROXY_ID! },
  { idempotencyKey: randomUUID() },
);
try {
  appendFileSync(MINE, `${session.id}\n`); // write it down before anything else
  // … tasks or steps …
} finally {
  await client.agentSessions.close(session.id);
}

Do not run two copies of this program at once with the same file: one would close the other’s session. For the same reason, keep one file per program and never share it: a program that reads another program’s file closes that program’s sessions. The two recipes on this page use two file names.

By hand: pass limit, then send each answer’s next_cursor back as cursor until it is null. See pagination.

Recover from 429

A 429 is one of two things, and they need different answers.

  • rate-limited: too many requests, or your account already has 3 tasks or lists of steps running at once. Wait retry_after_seconds (also the Retry-After header when the answer is not streamed) and send the same request with the same Idempotency-Key. A running task can hold its place for minutes, so sendTask below keeps trying for 10 minutes rather than for a set number of tries. Full rules: Idempotency keys.
  • concurrency-limit: too many sessions open, at create. Close a session your program created and no longer needs, then try again. Never close a session you did not create: it may be a person’s live session. When none of the open sessions is yours, stop and tell the person. (With the SDK’s aiTasksInFlight flag set, this error is the 3-at-once limit again: depending on how your account’s AI is billed, it answers this way or as rate-limited. The recipe waits out both.)

The create can also answer 503 with code: "dispatch_no_live_node" when no machine is free just then. Nothing was created: wait retry_after_seconds and send it again with the same key.

import { randomUUID } from 'node:crypto';
import {
  ConcurrencyLimitError,
  Driftstack,
  FeatureUnavailableError,
  RateLimitError,
} from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });
const sleep = (seconds: number) => new Promise((resolve) => setTimeout(resolve, seconds * 1_000));

// The sessions this run created and has not closed. Nothing else is ours to close.
const mine = new Set<string>();

async function closeMine(): Promise<number> {
  const ids = [...mine];
  for (const id of ids) {
    await client.agentSessions.close(id);
    mine.delete(id);
  }
  return ids.length;
}

async function createSession() {
  const key = randomUUID(); // one create, one key, reused on every retry
  for (let attempt = 0; attempt < 5; attempt++) {
    try {
      const created = await client.agentSessions.create(
        { proxy_id: process.env.PROXY_ID! },
        { idempotencyKey: key },
      );
      mine.add(created.id);
      return created;
    } catch (err) {
      if (err instanceof RateLimitError) {
        await sleep(err.retryAfterSeconds);
      } else if (err instanceof ConcurrencyLimitError) {
        // Every slot is taken. Free our own; a session a person or another
        // program holds stays open. With none of ours open, stop and say so.
        if ((await closeMine()) === 0) {
          throw new Error(
            `All session slots are in use (limit ${err.limit ?? '?'}) and this run holds none. ` +
              'It closed no session it did not create. Close a session you no longer need, or wait for one to end, then run this again.',
            { cause: err },
          );
        }
      } else if (
        err instanceof FeatureUnavailableError &&
        err.extensions.code === 'dispatch_no_live_node'
      ) {
        await sleep(Number(err.extensions.retry_after_seconds ?? 60));
      } else {
        throw err;
      }
    }
  }
  throw new Error('could not start a session');
}

async function sendTask(sessionId: string, task: string) {
  const key = randomUUID(); // one task, one key
  const giveUpAt = Date.now() + 10 * 60_000; // a running task holds its place for minutes
  for (;;) {
    try {
      return await client.agentSessions.message(sessionId, task, { idempotencyKey: key });
    } catch (err) {
      // By how the account's AI is billed, 3 tasks at once answer either way: wait out both.
      const busy =
        err instanceof RateLimitError ||
        (err instanceof ConcurrencyLimitError && err.aiTasksInFlight === true);
      if (!busy || Date.now() > giveUpAt) throw err;
      // A task takes longer than the few seconds the answer names. Asking again
      // every 15 seconds keeps waiting tasks from using up your message rate.
      await sleep(err instanceof RateLimitError ? Math.max(err.retryAfterSeconds, 15) : 15);
    }
  }
}

const session = await createSession();
try {
  const reply = await sendTask(session.id, 'Open https://example.com and tell me the page title.');
  console.log(reply.kind);
} finally {
  await client.agentSessions.close(session.id);
  mine.delete(session.id);
}

The SDKs already retry a 429 on most calls, waiting at most 10 seconds each time. They never retry a task or a list of steps, so those are yours, as above.

Hand a check to a person

When a website shows a CAPTCHA or sends a code, the session pauses and the task comes back with a notice_reason that says so. Ask a person to solve it, then resume and carry on.

import { randomUUID } from 'node:crypto';
import { Driftstack } from '@driftstack/sdk';

const client = new Driftstack({ apiKey: process.env.DRIFTSTACK_API_KEY! });
const sessionId = process.env.SESSION_ID!;

const NEEDS_A_PERSON = new Set([
  'challenge_customer_handoff',
  'challenge_solver_pending',
  'verification_email',
  'verification_sms',
  'unexplained_pause',
]);

const first = await client.agentSessions.message(
  sessionId,
  'Open https://shop.example.com/orders and tell me the status of my latest order.',
  { idempotencyKey: randomUUID() },
);

async function handOver(notice: string) {
  // Tell a person: they open the session in the desktop app or the live view,
  // and solve the check or enter the code. Here we wait for them to say done.
  console.log(`Session ${sessionId} needs a person: ${notice}`);
  await new Promise((resolve) => process.stdin.once('data', resolve));
  process.stdin.pause(); // or the program never exits
  await client.agentSessions.resume(sessionId);
  return client.agentSessions.message(sessionId, 'continue', { idempotencyKey: randomUUID() });
}

const reply =
  first.kind === 'plan-executed' && NEEDS_A_PERSON.has(first.notice_reason ?? '')
    ? await handOver(first.notice ?? '')
    : first;
if (reply.kind === 'plan-executed') console.log(reply.answer ?? reply.notice ?? 'done');

resume() answers as soon as the request is accepted (resume_requested), not when the session has carried on. If the continue task comes back with one of those notice_reason values again, the check is still there: hand it over again.

No solver provider is offered yet, so every check is handed to you. Do not make your program tap at a check: hand it over. The session.challenge_detected webhook tells you the moment a check appears.