skywalkskills

CatalogueAI-Era Code Health

F3Ready

LLM Integration Security & Cost Audit

Reads a shipped model feature for prompt injection, unsafe output, unbounded spend, and personal data leaving the system.

What it examines

  • Untrusted text placed into a prompt
  • Model output treated as HTML or as an instruction the system obeys
  • No cap on tokens or on spend
  • Personal data sent to the provider
  • No timeout or fallback when the provider fails
  • Prompts and responses that are not recorded
  • Model output stored with no filter

What you get back

A findings file. Each item has the evidence, how serious it is, and a fix.

Dependencies

  • An AI coding agent that reads SKILL.md files, such as Claude Code, Codex or Gemini CLI
  • Read access to the code of the AI feature
  • The prompt templates, including system prompts and examples. They may live outside the repository, in a config service or a database.
  • The list of model providers in use
  • The list of tools the feature can call, such as shell, database, files, outside APIs and email
  • Optional: production telemetry and billing data, for live checks of spend and logging
  • Python 3, for the report script. It uses only the standard library.
  • Python 3 with the jsonschema package, to check findings.json before fixing anything

How it works

The audit is a defensive review of your own system. It is read-only, and it never sends adversarial text to a live model unless you sign off on a controlled test.

It traces every route by which user or retrieved content reaches a prompt, then runs seven checks: prompt injection, unsafe model output, unbounded token spend, sensitive-data egress, provider fallback, interaction logging and output filtering.

Each finding says what backs it: code inspection or a live test. A check it could not test is recorded as insufficient evidence, never skipped.

You get findings.json and findings.md, ranked by risk. The remediate skill reads findings.json and adds the selected controls or plans them.

Setup

  1. Unzip the download, then copy the audit and remediate folders into your agent's skills folder (for Claude Code, .claude/skills/) in the repository you want to check.
  2. Collect the prompt templates, the provider list and the tool list before you start. The audit asks for them.
  3. Open the repository in your agent and ask it to run the f3-llm-integration-security-cost-audit skill. Add telemetry or billing exports if you have them.
  4. Check the result with the validator: pip install jsonschema, then python3 schema/validate.py findings.json.
  5. Ask your agent to run the f3-llm-integration-security-cost-remediate skill using findings.json.

Run it with your agent

Copy the prompt, fill in the parts in square brackets, and paste it into your agent.

Audit prompt

Run the f3-llm-integration-security-cost-audit skill on this repository. Prompt templates are at [paths]. Model providers: [list]. Tools the feature can call: [list]. Telemetry or billing data: [path, or none]. Do not send attack text to the live model. Mode: audit only. Do not change any code. Write findings.json and findings.md in the repository root.

Fix prompt, after you have findings.json

Run the f3-llm-integration-security-cost-remediate skill using findings.json. Work through the findings I choose, and leave the rest open.

Seen on ClaimsPortalThe claimant writes the prompt, and the model names the payoutRead the finding

What is inside

The files that make up this skill, as your agent reads them.

packages/f3-llm-integration-security-cost

audit › SKILL.md

Sign in to download

Use the email and password you signed up with.