ASO Audit Agent

ASO Audit Agent takes an Apple App Store URL, confirms the right app, and scores the listing against a 10-dimension App Store Optimization rubric - an overall score, a prioritized fix plan, rewrites, and a competitor comparison.

Every score cites evidence quoted from the listing, so a grade is checkable. A dimension with no usable signal is excluded and the score renormalized over what was actually evaluated - never a missing source posing as a zero. Built on Mastra: one conversational agent owns the chat, one deterministic workflow owns the audit.

ASO advice is easy to generate and hard to check - ask a model to review a listing and it returns ten confident paragraphs traceable to nothing on the page.

The interesting problem is the inverse of generation: constraining a model until its output is falsifiable. A score must attach to a quote; a dimension with no data must leave the denominator; the pipeline must run the same way on an app it has never seen. Apple makes it harder - the keyword field is private and promotional surfaces are not exposed - so deciding the honest ceiling, and reporting it instead of filling it, is most of the work.

Split along the axis that matters: conversation is variable, the audit is not. A Mastra agent owns the chat - URL parsing, the confirmation card, progress, follow-ups. A deterministic workflow owns the audit: scrape, a parallel fan-out (reviews, competitors, one vision call per screenshot), a single structured scoring call, then a pure assembly step. Named step events stream into the chat, so progress is a real signal.

Scope boundaries, chosen before the code and stated in the report:

• Keyword field is Apple-private - inferred from title, subtitle and description, and flagged as inferred.
• Preview-video presence is detected; its content is not evaluated - that needs decoding the asset for a 5%-weight dimension.
• Competitors come from a keyword search - framed as directional, not exact.
• No paid ASO data service for one dimension in ten.

A single Next.js app on the Node runtime with Mastra's chat route mounted as a handler - one process, one dev command. All network code lives in tools and nothing else does I/O; the rubric is a versioned prompt imported only by the scoring step, so swapping it never touches the agent.

Three providers, chosen per task: Gemini Flash-Lite for chat and vision, Groq's 70B llama for structured scoring, Firecrawl for the listing fields Apple's APIs do not expose. Model ids are env-only and the agent provider is selectable, so a swap is config, not infrastructure.

Run state is two keys in thread metadata - the confirmed app and the last report - written deterministically; follow-ups answer from conversation history. Testing is two evals rather than a unit suite: evidence faithfulness (every quote a literal substring of the collected data) and schema-validity variance across repeated runs. They run locally, since they hit live providers.

• Ten-dimension scorecard with one overall score, renormalized over the dimensions actually evaluated
• Verbatim cited evidence behind every score and recommendation
• Recommendations grouped into quick wins, high-impact and strategic
• Confirmation gate before any audit runs
• Live progress streamed from the workflow into the chat
• Inferred and excluded dimensions reported explicitly, never silently

• The cheapest guard against hallucinated evidence is a substring check - a mechanical eval caught more than any prompt tightening.
• Where correctness matters, enforce in code what the prompt asks for; a model that agrees with a rule still breaks it.
• A degraded report is a product; an exception is not. Every failure mode got a defined output before it got a retry.