Tutorial: Feed Hand-Drawn Sketches into an AI Workflow with a USB Camera

Abstract. This tutorial explains how to build a pipeline for getting paper sketches into an AI workflow using nothing but a USB webcam, ffmpeg, and a small bash script.

It explains how to:

Identify your camera
Capture a still
Build an interactive capture tool that shows a live preview
Trigger a screenshot with single keypress
Set up a filename convention that keeps captures sortable and machine-readable.
Hand the result to an AI model for description, cleanup, or redrawing.

Hand-drawn federal debt sketch captured by a USB webcam

Prerequisites

A USB camera. Any UVC-class webcam works; the example throughout uses a Logitech HD Pro Webcam C920 ¹. The built-in laptop camera also works.
The package manager used by your operating system. In this example, I use macOS with the Homebrew package manager. The capture commands use ffmpeg’s AVFoundation backend; the same ideas apply to Linux (/dev/videoN) and Windows (DirectShow).
ffmpeg installed (brew install ffmpeg). The Homebrew build includes ffplay, which provides the live preview.
Bash. macOS ships 3.2; the script below avoids bash 4-only features.

Step 1: Identify Your Camera

macOS exposes cameras to ffmpeg through AVFoundation. List the devices:

ffmpeg -f avfoundation -list_devices true -i "" 2>&1

Output looks like:

[AVFoundation indev] [0] HD Pro Webcam C920
[AVFoundation indev] [1] Immersed Camera
[AVFoundation indev] [2] FaceTime HD Camera
[AVFoundation indev] [3] Capture screen 0

Two things matter here. First, the USB webcam has a numeric index (0 in this listing). Second, the index can change between reboots or when other cameras appear, so a robust script should select the camera by name, not by hardcoded index. The script in Step 3 does exactly that: it parses this listing, skips virtual devices (FaceTime, Immersed, screen capture), and picks the first device whose name contains “Webcam” or “Camera”.

Step 2: Capture a Single Still

Capture one frame directly to a file:

ffmpeg -f avfoundation -framerate 30 -video_size 1920x1080 \
  -i "0" -frames:v 1 -update 1 -y sketch.jpg

The pieces:

-f avfoundation selects the macOS capture backend.
-framerate 30 -video_size 1920x1080 negotiates the stream parameters; the C920 supports 1080p at 30 fps.
-i "0" is the camera index from Step 1.
-frames:v 1 grabs a single frame instead of recording video.
-update 1 tells the image muxer to overwrite the file each run and silences the “image sequence pattern” warning.
-y overwrites without asking.

The first capture may trigger a macOS privacy prompt for your terminal app. Grant it in System Settings > Privacy & Security > Camera, or every capture will fail with “Input/output error.”

Verify the file is a real image:

sips -g pixelWidth -g pixelHeight sketch.jpg

Step 3: Build the Interactive Capture Script

A raw ffmpeg command works, but a loop that opens a preview, waits for a keypress, and writes a well-named file is far more usable. This is the script used to produce the sketch photo above. It lives at ~/av/bin/cam in the example setup.

#!/usr/bin/env bash
# cam -- interactive still capture from the USB camera.
#
# Default (interactive terminal): opens a live ffplay preview, captures a
# still on any single keypress, then opens the result in the default image
# viewer (Preview). Press q to abort. If the preview window is closed, the
# capture is aborted. A configurable timeout (default 5 min) auto-aborts
# so the command never hangs forever.
#
# Usage:
#   cam                 # interactive: preview -> any key captures
#   cam --shot          # capture one still immediately, no preview
#   cam --list          # show detected video devices and exit
#   cam --device 1      # force AVFoundation device index
#   cam -h              # help
set -euo pipefail

SNAP_DIR="$HOME/av/ast/img/illustrators/archer-ships/screenshots"
VIDEO_SIZE="${CAM_VIDEO_SIZE:-1920x1080}"
PREVIEW_W="960"
PREVIEW_H="540"
TIMEOUT="${CAM_TIMEOUT:-300}"
TMP_DIR=""
trap 'rm -rf "${TMP_DIR:-}"' EXIT

usage() {
    cat >&2 <<'EOF'
usage: cam [TITLE] [--timeout SECONDS] [--device N] [--shot] [--list]

Opens a live camera preview and waits for a keypress: any key captures a
still and opens it in the default image viewer; q aborts. If no key
arrives within the timeout (default 300s), the capture is aborted.
EOF
}

list_devices() {
    ffmpeg -f avfoundation -list_devices true -i "" 2>&1 || true
}

# detect_device <devlist-file> -- prints "<index>\t<name>" for the preferred
# video device: a USB webcam if present, else the first non-virtual device.
detect_device() {
    local file="$1" line rest idx name first="" first_name="" pref="" pref_name=""
    local in_video=0
    while IFS= read -r line; do
        case "$line" in
            *"video devices:"*) in_video=1; continue ;;
            *"audio devices:"*) in_video=0; continue ;;
        esac
        [[ $in_video -eq 1 ]] || continue
        case "$line" in
            *"] ["*) ;;
            *) continue ;;
        esac
        case "$line" in
            *FaceTime*|*Immersed*|*"Capture screen"*) continue ;;
        esac
        rest="${line#*] }"
        idx="${rest%%]*}"
        idx="${idx#*[}"
        name="${rest#*] }"
        [[ -n "$first" ]] || { first="$idx"; first_name="$name"; }
        case "$name" in
            *[Ww]ebcam*|*[Cc]amera*) pref="$idx"; pref_name="$name" ;;
        esac
    done < "$file"
    if [[ -n "$pref" ]]; then
        printf '%s\t%s\n' "$pref" "$pref_name"
    elif [[ -n "$first" ]]; then
        printf '%s\t%s\n' "$first" "$first_name"
    fi
}

DEVICE=""
SHOW_LIST=0
ONE_SHOT=0
TITLE="webcam"

while [[ $# -gt 0 ]]; do
    case "$1" in
        --list) SHOW_LIST=1 ;;
        --shot) ONE_SHOT=1 ;;
        --timeout)
            if [[ -z "${2:-}" ]] || [[ ! "$2" =~ ^[0-9]+$ ]]; then
                echo "cam: --timeout needs a positive integer (seconds)" >&2
                exit 2
            fi
            TIMEOUT="$2"
            shift
            ;;
        --device)
            if [[ -z "${2:-}" ]]; then
                echo "cam: --device needs an index (see cam --list)" >&2
                exit 2
            fi
            DEVICE="$2"
            shift
            ;;
        -h|--help) usage; exit 0 ;;
        -*) echo "cam: unknown option: $1" >&2; usage; exit 2 ;;
        *) TITLE="$1" ;;
    esac
    shift
done

if [[ $SHOW_LIST -eq 1 ]]; then
    list_devices
    exit 0
fi

TMP_DIR="$(mktemp -d /tmp/cam.XXXXXX)"

if [[ -z "$DEVICE" ]]; then
    list_devices > "$TMP_DIR/devlist"
    IFS=$'\t' read -r DEVICE CAM_NAME <<< "$(detect_device "$TMP_DIR/devlist")"
    [[ -n "$CAM_NAME" ]] || CAM_NAME="webcam"
fi
if [[ -z "$DEVICE" ]]; then
    echo "cam: no video device found. Run 'cam --list' to see what macOS sees." >&2
    exit 1
fi

mkdir -p "$SNAP_DIR"

if [[ $ONE_SHOT -eq 0 ]]; then
    echo "cam: opening live preview (device $DEVICE, $CAM_NAME) -- press any key to capture, q to abort (auto-abort after ${TIMEOUT}s)" >&2
    ffplay -loglevel error -f avfoundation -framerate 30 -video_size "$VIDEO_SIZE" \
        -x "$PREVIEW_W" -y "$PREVIEW_H" -window_title "cam preview" -i "$DEVICE" \
        </dev/null &
    FFPLAY_PID=$!
    sleep 1
    if ! kill -0 "$FFPLAY_PID" 2>/dev/null; then
        echo "cam: ffplay preview failed to start (device busy?)" >&2
        exit 1
    fi
    DEADLINE=$(( $(date +%s) + TIMEOUT ))
    KEY=""
    while :; do
        if ! kill -0 "$FFPLAY_PID" 2>/dev/null; then
            echo "cam: preview closed, aborting" >&2
            KEY="q"
            break
        fi
        if (( $(date +%s) >= DEADLINE )); then
            echo "cam: no keypress within ${TIMEOUT}s, aborting (no capture)" >&2
            KEY="q"
            break
        fi
        if IFS= read -r -n 1 -t 1 KEY; then
            break
        fi
        if [[ ! -t 0 ]]; then
            sleep 1
        fi
    done
    kill "$FFPLAY_PID" 2>/dev/null || true
    wait "$FFPLAY_PID" 2>/dev/null || true
    if [[ "$KEY" == "q" ]]; then
        echo "cam: aborted, no capture" >&2
        exit 0
    fi
    echo "cam: capturing..." >&2
    sleep 1
fi

# Filename per MEDIA FILENAME SPEC v1: TITLE-SOURCE-DETAIL-ID-DATE-QUALITY.
SOURCE_SLUG="$(echo "$CAM_NAME" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9' '-' | sed -E 's/-+/-/g; s/^-|-$//g')"
ID="$(date +%H%M%S)"
DATE="$(date +%Y-%m-%d)"
QUALITY="q${VIDEO_SIZE#*x}"
OUT="$SNAP_DIR/$TITLE-$SOURCE_SLUG-$ID-$DATE-$QUALITY.jpg"

if ! ffmpeg -loglevel error -f avfoundation -framerate 30 -video_size "$VIDEO_SIZE" -i "$DEVICE" -frames:v 1 -update 1 -y "$OUT" 2>"$TMP_DIR/ffmpeg.log"; then
    echo "cam: ffmpeg capture failed:" >&2
    cat "$TMP_DIR/ffmpeg.log" >&2
    if grep -qi "input/output error" "$TMP_DIR/ffmpeg.log"; then
        echo "cam: hint: macOS camera permission may be denied. Grant your terminal app access in System Settings > Privacy & Security > Camera." >&2
    fi
    exit 1
fi

if [[ ! -s "$OUT" ]]; then
    echo "cam: capture produced no file ($OUT missing or empty)" >&2
    exit 1
fi

echo "cam: opening $OUT" >&2
open "$OUT"

echo "$OUT"

Save it, make it executable, and check the syntax:

chmod +x ~/av/bin/cam
bash -n ~/av/bin/cam

The script is designed around three behaviors that matter for a sketch workflow:

Live preview. ffplay opens a 960x540 window showing the camera feed, so you can frame the paper before shooting.
Keypress capture. Any key takes the shot; q aborts. If you close the preview window instead, it aborts cleanly rather than hanging. The 300-second timeout is a safety net for sessions you forget about.
Non-interactive escape hatch. cam --shot captures immediately, which is what an agent or cron job should use. This matters because a bare cam from inside an agent terminal has no keyboard attached – never fake a keypress into the preview; launch it in a real terminal instead.

Step 4: Name Files for Machines

The script builds filenames like:

sketch-hd-pro-webcam-c920-131234-2026-08-14-q1080.jpg

That is the MEDIA FILENAME SPEC v1 convention: TITLE-SOURCE-DETAIL-ID-DATE-QUALITY, lowercase, dash-separated, web-safe characters only ². Every segment is optional except the title, and the detail segment is dropped when it duplicates the title. For a camera capture the mapping is:

TITLE: what you are photographing (default webcam, or a descriptive name like sketch-pasta-flow)
SOURCE: the camera name slug
ID: capture time (HHMMSS), so multiple captures never collide
DATE: capture date
QUALITY: resolution slug (q1080)

The reason to standardize: an AI tool or downstream script can parse the filename without asking. Sort by date, group by subject, or glob for a specific sketch – all of it works with plain string operations.

Step 5: Feed the Sketch into an AI Workflow

With the image on disk, the capture is only the input stage. Three patterns cover most sketch-to-AI uses:

Description and transcription. Point a vision-capable model at the file and ask for a structured read of the sketch: “Describe this hand-drawn diagram: labels, layout, arrows, annotations.” This is useful when the sketch is a chart or flowchart and you want the content in text form.
Image-to-image redraw. Use the capture as a reference image for a generative model to redraw the sketch as a clean vector-style graphic, a finished illustration, or a different medium. The camera capture is usually sufficient as a reference; deskewing and cropping in an editor (GIMP or similar) before generation improves results.
Cleanup and inpainting. When the sketch has pencil smudges, shadows, or the edge of the paper in frame, a mask-and-fill edit removes the artifacts before the image goes to the generator. The high resolution of a 1080p capture gives the editor room to work.

The capture directory in the example is ~/av/ast/img/illustrators/archer-ships/screenshots/ – a dedicated screenshots folder per project keeps sketches separate from generated images and lets an AI workflow watch the folder for new input.

Verification

Run cam --list and confirm your USB camera appears in the video device list.
Run cam from a real terminal. The preview window opens; press any key. The capture writes a file and opens it in the default viewer.
Confirm the filename follows the spec: sips -g pixelWidth -g pixelHeight ~/av/ast/img/illustrators/archer-ships/screenshots/*.jpg should report 1920x1080.
Run cam --shot from an agent or cron context and confirm it returns immediately with the file path on stdout.
Press q during a preview and confirm no file is written; let the timeout expire (or set --timeout 3) and confirm it aborts with no capture.

Notes

Logitech, “HD Pro Webcam C920.” Logitech product page, accessed 2026-08-14. https://www.logitech.com/en-us/products/webcams/c920-pro-hd-webcam.958-000055.html ↩︎
Archer Ships, “MEDIA FILENAME SPEC v1.” ~/av/pol/CODE.md section 16, 2026-08-14. https://github.com/archerships/av ↩︎

Want to stay in touch?

Join my Signal announce-only group to be notified when I have a new essay up and other important announcements.
For discussion, join the libertygardeners Signal group.
Subscribe to my mailing list.
Email: archerships@protonmail.ch
Signal: archerships.43
Website: archerships.com
Other social media: Substack | Twitter | Facebook | Nostr | Odysee

If you’d like to support my work:

Share my posts.
Become a subscriber to my newsletter.
Attend my live events (dinner parties, conferences, pop-up cities, etc).
Introduce me to like-minded people.
Make a one-time donation to support my work: Crypto | Fiat
Hire me for privacy / crypto / censorship consulting.

If there is a topic you’d like me to cover, please let me know!

Questions, comments, and suggestions are welcome.