Talking to the model: a minimal BedrockModel

01One service, two providers

The LanguageModel service comes straight from effect/unstable/ai. You build a provider with LanguageModel.make({ generateText, streamText }) and wrap it as a Layer; the framework owns the generateText/streamText ergonomics on top. A provider's generateText receives normalized ProviderOptions (a Prompt.Prompt plus the offered Tools) and returns encoded response parts:

export const make: (params: {
  readonly generateText: (options: ProviderOptions) =>
    Effect.Effect<Array<Response.PartEncoded>, AiError, IdGenerator>;
  readonly streamText: (options: ProviderOptions) =>
    Stream.Stream<Response.StreamPartEncoded, AiError, IdGenerator>;
}) => Effect.Effect<LanguageModel.Service>;

interface ProviderOptions {
  readonly prompt: Prompt.Prompt;         // system / user / assistant / tool messages
  readonly tools: ReadonlyArray<Tool.Any>; // tools the model may call
  // ...responseFormat, toolChoice, span
}

A provider returns Response.PartEncoded values — { type: "text", text } for an answer, or { type: "tool-call", id, name, params } when the model wants a tool run. The agent never constructs ProviderOptions; it calls LanguageModel.generateText({ prompt, toolkit }) and the framework normalizes everything.

LanguageModel, Prompt, Response, Tool from effect/unstable/ai

02Why only the provider is hand-written

The upside: there is exactly one AI module to learn, the provider is ~80 lines you fully understand, and Converse is itself provider-agnostic across Bedrock's model catalogue.

03Mapping our contract onto Converse

Bedrock's Converse takes a modelId, a list of messages (each with content blocks), optional system blocks, and an optional toolConfig. Our job is two pure functions: Prompt.Prompt + tools → Converse request, and Converse response → Response.PartEncoded[]. The tools come in as real Tools, so their provider JSON schema is Tool.getJsonSchema(t):

import { Prompt, Response, Tool } from "effect/unstable/ai";
import * as bedrock from "@distilled.cloud/aws/bedrock-runtime";

const toConverse = (
  modelId: string,
  prompt: Prompt.Prompt,
  tools: ReadonlyArray<Tool.Any>,
): bedrock.ConverseRequest => {
  const system = prompt.content
    .filter((m) => m.role === "system")
    .map((m) => ({ text: textOfMessage(m) }));

  const messages: bedrock.Message[] = prompt.content
    .filter((m) => m.role !== "system")
    .map((m) => ({
      role: m.role === "assistant" ? "assistant" : "user",
      content: [{ text: textOfMessage(m) }],
    }));

  const base: bedrock.ConverseRequest = { modelId, system, messages };
  if (tools.length === 0) return base;

  return {
    ...base,
    toolConfig: {
      tools: tools.map((t) => ({
        toolSpec: { name: t.name, description: t.description ?? "", inputSchema: { json: Tool.getJsonSchema(t) } },
      })),
    },
  };
};

Reading the response is the mirror image. Converse returns output.message.content — an array of blocks. Text blocks carry text; tool requests carry a toolUse with a name and JSON input. We map them to encoded text and tool-call parts:

const fromConverse = (res: bedrock.ConverseResponse): Array<Response.PartEncoded> => {
  const parts: Array<Response.PartEncoded> = [];
  let i = 0;
  for (const b of res.output.message.content) {
    if (typeof b.text === "string" && b.text.length > 0) parts.push({ type: "text", text: b.text });
    if (b.toolUse !== undefined)
      parts.push({ type: "tool-call", id: `call-${++i}`, name: b.toolUse.name, params: b.toolUse.input ?? {} });
  }
  return parts;
};

shapes from src/model.ts, grounded in @distilled.cloud/aws bedrock-runtime Converse + effect 4 Response.PartEncoded.

04The layer — a provider over Converse

LanguageModel.make turns the two hooks into a LanguageModel.Service; we wrap it in Layer.effect(LanguageModel.LanguageModel, ...). Converse requires three services — Credentials, Region, and an HttpClient — which the provider resolves internally, normalising every failure to one AiError:

import { Effect, Layer, Stream } from "effect";
import { AiError, LanguageModel } from "effect/unstable/ai";
import { NodeHttpClient } from "@effect/platform-node";
import * as Credentials from "@distilled.cloud/aws/Credentials";
import * as Region from "@distilled.cloud/aws/Region";

const awsServices = (region?: string) =>
  Layer.mergeAll(
    Credentials.fromChain(),                                   // env / SSO / profile chain
    region ? Layer.succeed(Region.Region, region) : Region.fromEnv(),
    NodeHttpClient.layerUndici,
  );

export const BedrockModel = {
  layer: (options: { model: string; region?: string }): Layer.Layer<LanguageModel.LanguageModel> =>
    Layer.effect(
      LanguageModel.LanguageModel,
      LanguageModel.make({
        generateText: (o) =>
          bedrock.converse(toConverse(options.model, o.prompt, o.tools)).pipe(
            Effect.map(fromConverse),
            Effect.provide(awsServices(options.region)),
            Effect.mapError(toAiError),                          // any failure → one AiError
          ),
        streamText: () => Stream.empty,                          // this course uses generateText only
      }),
    ),
};

// Default to the offline stub; talk to real Bedrock when BEDROCK_MODEL is set.
// Config reads the env at the composition root. The selector is a Layer, but the
// choice lives inside an Effect (Config is yielded, not read synchronously) — so
// `Layer.unwrap` turns that "which Layer?" Effect into the Layer itself. The inner
// return annotation (`: typeof StubModel`) collapses the two branches to one type.
const ModelLive = Layer.unwrap(
  Effect.map(Config.option(Config.string("BEDROCK_MODEL")), (model): typeof StubModel =>
    Option.isSome(model)
      ? BedrockModel.layer({ model: model.value })   // region/creds from the env
      : StubModel,
  ),
);

export const AgentApiLayer = HttpApiBuilder.layer(AgentApi).pipe(
  Layer.provide(AgentHandlers.pipe(Layer.provide(Layer.mergeAll(ModelLive, LocalMemory)))),
);

# default: deterministic stub, runs offline
$ pnpm dev
$ curl -s localhost:8080/invocations -d '{"prompt":"what is a gateway?"}'
{"response":"「stub」 I read the explainers and here is what I found about: what is a gateway?"}

# the eu. inference profile needs EU credentials + an EU region
$ aws sso login --profile my-eu-profile
$ AWS_PROFILE=my-eu-profile AWS_REGION=eu-central-1 \
    BEDROCK_MODEL=eu.anthropic.claude-sonnet-4-5-20250929-v1:0 pnpm dev
# → same agent, now backed by Bedrock Converse

# in another shell — one real round trip through the respond loop:
$ curl -s localhost:8080/invocations -d '{"prompt":"what is a gateway?"}'
{"response":"A Gateway exposes the agent's tools as MCP endpoints. Here searchExplainers
runs as a Lambda behind an AgentCore Gateway, so the model can call it over the corpus."}

# the handler logs every answer (Effect.log, inside the agent.invoke span):
[12:00:14.015] INFO (#12) http.span=812ms: agent responded: A Gateway exposes the agent's…

BEDROCK_MODEL takes any Converse-capable Bedrock model id your account has enabled. Tested working here:

• openai.gpt-oss-20b-1:0 — an OSS model, no regional inference profile, so it runs without the eu./EU-region requirement.
• eu.anthropic.claude-sonnet-4-5-20250929-v1:0 — Claude Sonnet 4.5 via the EU cross-region profile (needs an EU region + EU credentials).

The eu. (and us./apac.) prefixes are cross-region inference profiles — they fan a request across that geo's regions, so the region you set must belong to it. A bare id like openai.gpt-oss-20b-1:0 has no such constraint. Anthropic models also require the account's one-time Anthropic use-case form (Bedrock console) before they answer; an OSS model is the quickest way to see a real response end to end.

05Proving it without spending tokens

Calling Bedrock needs AWS credentials and costs money, so the lesson test asserts the wiring, not a live call: that BedrockModel.layer provides the LanguageModel service and is a drop-in. The behavioural contract is pinned against the deterministic stub. The test lives with its checkpoint and runs as part of pnpm test:

it("BedrockModel.layer provides the same LanguageModel service", () =>
  Effect.gen(function* () {
    const model = yield* LanguageModel.LanguageModel;
    return typeof model.generateText === "function";   // it satisfies the service
  }).pipe(Effect.provide(BedrockModel.layer({ model: "eu.anthropic.claude-sonnet-4-5-20250929-v1:0" }))));

$ pnpm test
 Test Files  4 passed (4)
      Tests  13 passed (13)

The live model isn't exercised by a test — it's wired into the agent (section 04) behind BEDROCK_MODEL. Run the agent with that variable set and you hit real Bedrock through the very same respond loop the stub test pins; the handler logs each answer (Effect.log("agent responded: …")) so you can watch the model's output. Because the contract is identical, a green stub test is a meaningful spec for the real model.

06The tool loop, end to end

Recall respond from lesson 02. With a real model the loop is unchanged, but now the toolUse blocks come from Claude deciding it needs to look something up:

Locally the stub drives this deterministically; in the cloud Bedrock drives it. generateText({ toolkit }) does one round — generate, then auto-resolve the tools the model asked for; respond re-prompts with the results until text comes back. Lesson 06 takes the searchExplainers tool and exposes it through an AgentCore Gateway, so the tool runs as its own Lambda — but respond still just calls generateText.