Dev48
Language
  • About
  • Services
  • Industries
  • Technologies
  • Articles
  • Contacts
Book a call
    Home/Articles/Build a talking jack o lantern with a signalwire ai agent
Dev48
info@dev48.org+375 29 320-40-96WhatsApp@isky_by

© 2026 · All rights reserved.

Build a Talking Jack-o'-Lantern with a SignalWire AI Agent

Source: Signalwire•November 16, 2026 at 09:34 PM

Build a Talking Jack-o'-Lantern with a SignalWire AI Agent

A SignalWire AI agent in a jack-o'-lantern

Article

Nov 16, 2026

read

A SignalWire AI agent in a jack-o'-lantern

Adam Kalsey

EVP, Product

Build it free.

Create a space and ship your first call flow in minutes.

Voice AI

AI Agents

Developer Tutorials

Three kids walk up the driveway on Halloween night. A scarecrow with a burning jack-o'-lantern for a head watches them come.

"A witch," it says. "How daring. How utterly unprecedented."

The witch laughs. The vampire next to her asks the pumpkin how old it is. "I was carved on a Tuesday," it says, and goes back to ignoring them as attentively as it can.

The pumpkin is real, and it has a secret hidden inside: a SignalWire AI agent on a video call. A computer in the prop keeps the call open all night, with a webcam for eyes and a speaker for a mouth. The agent sees who's in front of it and talks to them in character. When a small child is in the group, it stays as grand as ever, but nobody in front of it is the joke.

This series shows how I built it. The excerpts in each post show the ideas, and the ai-jackolantern repository has the complete, working code.

What the series covers

The series has nine parts, and each one adds a working piece:

  • The system and the call (this post).

The system and the call (this post).

  • The agent: a personality and three modes, as contexts and steps.

The agent: a personality and three modes, as contexts and steps.

  • Tools: what the AI calls on my server, and why its material lives in code.

Tools: what the AI calls on my server, and why its material lives in code.

  • Vision: letting the agent see, and checking the camera before any decision that matters.

Vision: letting the agent see, and checking the camera before any decision that matters.

  • The stage: detecting who is standing in front of the pumpkin, in the browser.

The stage: detecting who is standing in front of the pumpkin, in the browser.

  • The voice: audio effects in the browser.

The voice: audio effects in the browser.

  • Pacing and state: the server keeps score.

Pacing and state: the server keeps score.

  • The operator: a person with a phone who can overrule the agent.

The operator: a person with a phone who can overrule the agent.

  • Testing it, and running it for a whole night.

Testing it, and running it for a whole night.

By the end of this post, a browser holds a video call to a SignalWire AI agent.

The pieces

The pumpkin runs in four places:

Piece

Runs on

Job

The pumpkin page

The computer in the prop

Holds the call, sends the camera and mic, plays the voice through audio effects, detects who is on stage

The AI agent

SignalWire

Listens, looks, talks, and calls tools

My server

Any host with a public HTTPS address

Serves the agent's configuration and the page, runs the tools, keeps the state of each visit

The phone remote

The operator's phone

Starts and stops the show, adjusts settings, switches modes

The server is one Python process built on our Python SDK (signalwire-sdk). It serves the agent, the pumpkin page, and the page's APIs from one port.

How a call flows

Every time the page dials, the call goes through the same steps:

  • The page asks my server for a guest token at /eyes-token.

The page asks my server for a guest token at /eyes-token.

  • The page dials the agent's address with that token, over video.

The page dials the agent's address with that token, over video.

  • SignalWire fetches the agent's configuration from /pumpkin on my server. The configuration is SWML, our markup for call flows, and the SDK writes it from the Python code.

SignalWire fetches the agent's configuration from /pumpkin on my server. The configuration is SWML, our markup for call flows, and the SDK writes it from the Python code.

  • SignalWire runs the ai verb from that SWML: speech recognition, the model, text to speech, and vision.

SignalWire runs the ai verb from that SWML: speech recognition, the model, text to speech, and vision.

  • When the model calls a tool, SignalWire posts the request to /pumpkin/swaig on my server, and my code answers.

When the model calls a tool, SignalWire posts the request to /pumpkin/swaig on my server, and my code answers.

Set up SignalWire from code

The page dials an address that belongs to a resource in my SignalWire space. My server creates that resource when it starts, so a new deployment needs no setup in the dashboard. The resource is an SWML webhook, which means SignalWire fetches the call's instructions from a URL when a call arrives. The setup code finds the resource by name, creates it if it's missing, and corrects its URL if it points somewhere else. Running it twice changes nothing.

The agent's URL carries a basic-auth user and password, so only SignalWire can fetch the configuration. I derive the password from the API token with an HMAC, so it comes out the same after every restart. That means a call in progress keeps working when the server reloads.

The server's .env holds SIGNALWIRE_SPACE, SIGNALWIRE_PROJECT_ID, and SIGNALWIRE_API_TOKEN, plus SWML_PROXY_URL_BASE, the server's public HTTPS address, so SignalWire knows where to fetch the agent.

Mint a guest token

Anyone can read what's in a browser page, so the page never gets the project credentials. My server mints a guest token for it instead, and that token can dial one address and expires in two hours.

The response also tells the page to dial the address's video channel, so the agent can see. The token route sits behind a password (EYES_PASSWORD), because each token can start a billed call.

Dial from the browser

The page uses our Browser SDK v4 (@signalwire/js), which takes a token provider, an object with authenticate() and refresh() methods. My provider fetches from /eyes-token, and the SDK calls refresh() before a token expires, so one call can outlive many tokens.

The dial sends the page's processed microphone stream and the camera, which I open at 640 by 480 and 10 frames a second. That's enough for the vision model, and it keeps the work light for the hidden computer. The pumpkin has no screen, so it receives audio only.

Keep the call up all night

A call that runs for hours will drop now and then, so the page dials in a loop if the call drops. A call that ran at least 30 seconds redials at once. One that failed right away backs off, from 5 seconds up to 60, so a broken agent can't spin the loop. If a dial hasn't started within 30 seconds, the page rebuilds the SDK client in place. A reload would need another click on the page before Chrome plays audio.

Traps to avoid

Don't put project credentials in the page. Anyone can open the page's source. Sure, you're the only one running the page and no one else knows the URL, but have good habits from the start. Mint a short-lived guest token on your server, limited to the one address the page may call, and put a password on the token route.

Don't hand the SDK only your processed audio. Once you pass inputAudioStream, the SDK sends only the streams you give it and ignores device settings. Pass the camera as inputVideoStream too, or the call goes out with no video and the agent is blind.

Next in this series

The call connects, but the agent on the other end has no personality yet. In part 2 I give it one, with a base prompt and three modes built as contexts and steps. All of it can be checked without placing a call.

← All articles

What this article says

More in Telecom & Networks

All →
Prediction market odds for Tesla-SpaceX merger by April 2027 top 60%Пресса
SpaceX

Prediction market odds for Tesla-SpaceX merger by April 2027 top 60%

Lumen Voice Alternative: No Dial Tone Lost
Infobip

Lumen Voice Alternative: No Dial Tone Lost

What an AI agent can do on Bird: CLI, MCP, terminal signup, and spend controls
MessageBird

What an AI agent can do on Bird: CLI, MCP, terminal signup, and spend controls

Vonage Video API 2.36: Retry Logic, Observability, and iOS 27 Support
Vonage API Platform

Vonage Video API 2.36: Retry Logic, Observability, and iOS 27 Support

Number Lookup Now Available for Sub-Organization Users
Telnyx

Number Lookup Now Available for Sub-Organization Users

Del B20 a Bruselas: redes, escala y certidumbre para una IA competitiva
Telefónica

Del B20 a Bruselas: redes, escala y certidumbre para una IA competitiva

More from Signalwire

Voice AI Is Growing Up
Signalwire

Voice AI Is Growing Up

What Your AI Agent Should Never Hear: Card Numbers and CVVs
Signalwire

What Your AI Agent Should Never Hear: Card Numbers and CVVs

Build a Voice AI Agent That Follows Code, Not Prompts
Signalwire

Build a Voice AI Agent That Follows Code, Not Prompts

What Is “Prompt and Pray”?
Signalwire

What Is “Prompt and Pray”?

ClueCon 2026 Wrap-up
Signalwire

ClueCon 2026 Wrap-up

Combatting Caller ID Spoofing with STIR/SHAKEN
Signalwire

Combatting Caller ID Spoofing with STIR/SHAKEN