---
title: "How to QA AI Creative Tags Before Performance Analysis"
description: "An eight-gate protocol for auditing AI creative tags: reconcile every asset and field, expose coverage, migrate labels safely, and test accuracy."
canonical: "https://lemon-ai.com/resources/ai-creative-tagging-qa"
markdown_url: "https://lemon-ai.com/resources/ai-creative-tagging-qa.md"
language: "en"
image: "https://lemon-ai.com/og/resource-ai-creative-tagging-qa.png"
image_alt: "An AI creative tagging ledger reconciles submitted operations, applied values, explicit failures, and per-attribute coverage."
date_published: "2026-08-25"
date_modified: "2026-08-25"
authors: ["Gregory Potemkin"]
schema_types: ["Article","BreadcrumbList","Organization","Person","WebApplication","WebPage","WebSite"]
---

# How to QA AI Creative Tags Before Performance Analysis

Creative tagging integrity protocol

An eight-gate protocol for auditing AI creative tags: reconcile every asset and field, expose coverage, migrate labels safely, and test accuracy.

![Gregory Potemkin](https://lemon-ai.com/images/join-us/gregory.webp) 

By [**Gregory Potemkin**](https://lemon-ai.com/authors/gregory-potemkin)  
Founder & CEO  
Published August 25, 2026 

**Do not let AI-generated creative tags into a performance analysis until every eligible asset, submitted operation, and requested field can be reconciled.** Then test the tags themselves on a documented, human-reviewed sample drawn from the creatives the system will actually see. Those are two different controls: pipeline integrity proves that nothing disappeared; accuracy testing asks whether the values that arrived are right.

A handful of plausible labels proves neither one. A system can tag every sampled ad correctly while silently omitting 15% of the library. It can also return a valid row for every asset while one decision-critical field fails far more often than the others. The dashboard will still look complete unless it exposes the accounting underneath it.

[ ![An AI creative tagging integrity ledger. One thousand submitted operations reconcile to 970 applied, 20 explicitly failed, and 10 still in progress. Within the 970 applied responses, 3,880 requested fields reconcile to 3,683 valid values and 197 failed fields. Per-attribute coverage ranges from 82.0 to 96.8 percent, showing why one overall completion rate is not enough.](https://lemon-ai.com/images/resources/creative-tagging-integrity-ledger.svg) ](https://lemon-ai.com/images/resources/creative-tagging-integrity-ledger.svg) 

Two ledgers, two questions. Operation accounting catches missing work. Field accounting exposes the attribute that failed inside an otherwise usable response.

## A correct-looking tag table can still be wrong

Creative tagging has two independent failure planes.

**Semantic failure:** the system returns `testimonial` for a product demo, misses the offer, or assigns a value that does not follow the team’s definition.

**Accounting failure:** an eligible asset never enters the run, an output row never arrives, a retry creates duplicate paid work, an invalid field disappears without a failure state, or a renamed label abandons its historical answers.

Human review addresses the first plane. A ledger addresses the second. Neither substitutes for the other.

This distinction matters because structured generation solves less of the problem than its name suggests. Google’s current [structured output guidance](https://ai.google.dev/gemini-api/docs/structured-output) supports typed fields and enumerated values, but it also tells implementers to validate values in the application and handle output that follows the schema while remaining semantically wrong. Valid JSON is a transport property. It is not proof that the creative was classified correctly.

Adobe’s [Smart Tags documentation](https://experienceleague.adobe.com/en/docs/experience-manager-cloud-service/content/assets/manage/smart-tags) makes the human side equally explicit: review automatically generated tags, remove inaccurate ones, and deliberately reprocess older assets that were not tagged automatically. Adobe is describing digital asset management rather than ad-performance analysis, but the control transfers directly. If historical assets are absent from the denominator, a performance comparison is biased before a metric is calculated.

## The eight gates

| Gate                     | What must be true                                                                                                                 | Failure it catches                                                    |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- |
| 1\. Population           | The eligible denominator is defined by account scope and compatible creative type                                                 | Old, unsupported, private, or newly arrived assets disappear silently |
| 2\. Schema               | Every attribute has a stable name, definition, type, allowed values, asset scope, and version                                     | Two people or two runs answer different questions under one label     |
| 3\. Value validation     | Every returned value is checked after parsing                                                                                     | Valid JSON carries an invalid enum, number, boolean, or null          |
| 4\. Work reconciliation  | Every submitted operation is applied, explicitly failed, or still in progress, and a retry cannot create ambiguous duplicate work | Missing output rows, partial jobs, and duplicate submissions          |
| 5\. Field reconciliation | Every requested field becomes a valid stored value or an explicit field failure                                                   | One bad field is hidden, or it destroys an otherwise usable response  |
| 6\. Coverage             | Coverage is published per attribute against the full eligible population                                                          | A healthy overall rate hides a weak or selectively missing label      |
| 7\. Migration            | Renames and value-set changes have a declared data-migration policy                                                               | Coverage collapses or historical values change meaning after an edit  |
| 8\. Evaluation           | A frozen, realistic test set is reviewed by people and monitored after changes                                                    | Schema-valid labels are consistently wrong in actual use              |

The gates are ordered because each one establishes the population for the next. Measuring tag accuracy on returned rows cannot detect assets that never returned. Computing coverage before invalid fields are separated from valid ones counts failure as success. Evaluating performance before taxonomy migration is complete compares labels that no longer mean the same thing.

## Gate 1: define the denominator before the run

Start with the set of assets that should be tagged, not the rows the model happened to return.

Write the eligibility rule in operational terms:

- account, app, and date scope;
- media types the attribute can describe;
- whether text, playable source, private video, or unsupported formats qualify;
- how duplicates and replacement files are identified;
- whether an existing non-null answer is preserved;
- what happens to assets that arrive during a run.

The denominator for a video attribute is every eligible video in that scope. It is not every video with an `attributes` object, every row returned by the provider, or every asset that spent money in the period. Those are different populations.

A stored null is missing, not tagged. Treating key presence as coverage lets a failed value masquerade as an answer. It also prevents the system from filling the field later if selection assumes the work is already done.

## Gates 2 and 3: constrain the answer, then distrust it

A useful attribute definition is executable. It needs a stable machine name, a description precise enough to decide borderline cases, a compatible creative type, and one of three basic output contracts:

- **Enum:** one value from a closed list, such as `testimonial`, `product_demo`, or `gameplay`.
- **Boolean:** a real true or false judgment, such as whether a spoken CTA is present.
- **Number:** a finite measurement, such as seconds until the product first appears.

Free-form strings feel flexible but destroy comparability. `UGC`, `ugc`, `creator video`, and `user generated` become four groups unless something normalizes them, and normalization after the fact introduces another model whose errors need their own audit.

The provider schema should constrain the type and allowed values, but the application still validates each field after parsing. Reject an enum member outside the list. Reject a boolean disguised as a number. Reject infinity, `NaN`, an empty value, and a null where the analysis requires an answer.

Do that per field. If `hook_type` is invalid but `cta_present` and `seconds_to_product` are usable, preserve the two valid values and record `hook_type` as failed. Throwing away the whole object wastes good evidence. Quietly dropping the bad field is worse because the missingness vanishes from the audit trail.

## Gates 4 and 5: reconcile work and fields separately

At any snapshot, the operation ledger should satisfy one plain invariant:

```
submitted operations = applied + explicitly failed + still in progress
```

No fourth bucket called “probably finished.” A provider job reaching a terminal state is not enough. Google Cloud’s current [BatchPredictionJob reference](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/reference/rpc/google.cloud.aiplatform.v1) exposes completion statistics and partial failures separately from overall job state. The consumer still has to reconcile the identifiers it submitted with the rows and failures it received.

Make the request identity durable before the provider call. A practical identity includes the processing phase, creative type, model, model configuration, input fingerprint, and sorted operation identifiers. If an orchestrator retries the same semantic work, it should attach to that record rather than create a new purchase.

The ugly case is an interrupted submission: the provider may have accepted the job while the caller lost the response. Blindly retrying can buy the same inference twice and leave two jobs racing to apply values. Quarantine the work, look for the first job using its immutable coordinates, and require manual reconciliation if identity is ambiguous.

Inside each applied response, use a second invariant:

```
requested fields = valid stored values + explicit field failures
```

This second ledger is what prevents one weak attribute from hiding inside an otherwise successful row.

## Gate 6: publish coverage by attribute

Consider a synthetic run over 1,000 eligible video assets. The numbers below demonstrate the accounting only. They are not a model benchmark and do not come from a customer account.

The operation ledger closes:

| Operation state                         | Assets    | Share of eligible assets |
| --------------------------------------- | --------- | ------------------------ |
| Applied response                        | 970       | 97.0%                    |
| Explicit provider or processing failure | 20        | 2.0%                     |
| Still in progress                       | 10        | 1.0%                     |
| **Submitted**                           | **1,000** | **100.0%**               |

Now inspect the four requested attributes inside the 970 applied responses:

| Attribute                | Valid values | Failed fields | Coverage against 1,000 eligible assets |
| ------------------------ | ------------ | ------------- | -------------------------------------- |
| hook\_type               | 940          | 30            | 94.0%                                  |
| cta\_present             | 968          | 2             | 96.8%                                  |
| creative\_format         | 955          | 15            | 95.5%                                  |
| offer\_type              | 820          | 150           | 82.0%                                  |
| **All requested fields** | **3,683**    | **197**       | **3,880 fields reconciled**            |

The provider-row story is “97% complete.” The `offer_type` story is “18% of the eligible library has no usable value yet.” Both are true. Only the second tells an analyst whether an offer comparison represents the library.

Coverage is not accuracy. A value can be present and wrong. Coverage is also not a reason to impute a label: filling missing tags from campaign name, filename, or the dominant value merely hides the failure and can manufacture a performance pattern. Show the missing group, the failure causes, and whether retry can help.

Do not open downstream analysis merely because coverage crossed one universal percentage. The acceptable gap depends on how missingness is distributed. Ninety percent coverage may be usable when failures are random across a broad exploratory table. Ninety-nine percent may still be unsafe if every playable, non-English ad, or long video is inside the missing one percent. Segment the failures before deciding.

## Gate 7: treat taxonomy edits as data migrations

A tag name is often both a display label and a storage key. Renaming it without moving the stored answers creates a new empty field, abandons the old values, collapses coverage, and can trigger a paid re-tag of the whole library. The screen shows an innocent text edit. The data system sees a schema migration.

Handle common changes explicitly:

- **Rename:** move the stored values to the new key in the same transaction as the definition change. Refuse collisions.
- **Merge enum values:** publish the old-to-new mapping and retain the prior schema version for reproducibility.
- **Split one value into several:** do not invent the new distinction from the old label. Re-review or re-tag the affected assets.
- **Delete a value:** define whether historical reports retain it, map it, or exclude it. Never let it silently become null.
- **Change asset scope or definition:** create a new version. The same name with a new question is not the same attribute.

Adobe’s [taxonomy guidance](https://experienceleague.adobe.com/en/perspectives/taxonomy-and-tagging-best-practices-for-aem-assets) notes that vocabularies evolve with the organization and should be governed across the asset lifecycle. For performance analysis, that lifecycle needs a reproducible record: attribute-definition identifier, schema version, model and configuration, input fingerprint, application time, and whether a person corrected the value.

Manual correction needs a policy too. A corrected field should not be overwritten merely because the next automated job ran. If a new creative file genuinely replaces the old one, its source fingerprint should change and the system can deliberately decide what must be reviewed again.

## Gate 8: measure accuracy in the operating population

Once the ledgers close, test the meaning of the values. NIST’s [AI Risk Management Framework](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf) says accuracy measurements should use clearly defined, realistic test sets representative of expected use, document the method, and disaggregate results across relevant segments. Its [Measure playbook](https://airc.nist.gov/airmf-resources/playbook/measure/) extends that into ongoing evaluation and production monitoring.

For creative tags, that means:

1. Freeze a sample from the real operating population. Include the media types, languages, durations, visual styles, and source-quality conditions the system actually receives.
2. Write the label rubric before reviewing output. Define borderline cases and the evidence needed for an answer.
3. Have a second qualified reviewer inspect disagreements, not merely accept the first review as truth. Record disagreement separately from model error.
4. Report the metric that fits the field. Use a confusion matrix and per-class precision and recall for enums, false-positive and false-negative rates for booleans, and an error distribution with a declared tolerance for numbers.
5. Segment the result. An average can hide that the model works on static English ads and fails on long non-English videos.
6. Re-run after a model, prompt, schema, media-preparation, or taxonomy change, and on a declared monitoring cadence. Do not wait for a performance table to look strange.

There is no honest universal pass rate. Mislabeling `background_color` and mislabeling `offer_type` do not have the same cost, and a discovery tool can tolerate errors that an automated budget workflow cannot. Set the acceptance rule from the decision the tag will influence, then publish the rule with the result.

A model confidence score does not replace this evaluation. Adobe sorts its Smart Tags by confidence and still advises human review. Unless a score has been calibrated against the same definition and operating population, it is a ranking signal, not the probability that the tag is correct.

## Eight questions for a vendor or implementation review

Before creative tags influence a brief or test plan, ask:

1. What exactly is the eligible asset denominator for each attribute?
2. Can I inspect the definition, type, allowed values, asset scope, and version?
3. What happens to a valid row containing one invalid field?
4. Can every submitted operation be reconciled to applied, failed, or still in progress?
5. How does the system prevent duplicate work after an ambiguous retry?
6. Does it publish per-attribute coverage and failure causes against all eligible assets?
7. What happens to historical answers when a label is renamed, merged, split, or deleted?
8. Where is the human-reviewed test set, its rubric, segment breakdown, change history, and next monitoring date?

If one answer is “the model usually gets it right,” the audit has not started.

## How Lemon applies the protocol

Lemon’s tagging path separates the same concerns described above. Attribute definitions are typed as enum, boolean, or number. Returned values are validated field by field, usable values survive a neighboring failure, and failed fields remain visible as missing work rather than being counted as tags. Batch operations carry durable identities, output rows are reconciled to the operations that requested them, and ambiguous submissions do not receive an automatic second purchase. Coverage is computed per attribute against compatible assets, with queued, active, failed, and unfillable states kept distinct. Renames move stored answers with the definition instead of abandoning them under the old key.

Those controls establish data integrity. They do not convert an AI label into causal evidence, and they do not manufacture an accuracy percentage. After the tags pass this protocol, the separate [creative attribute analysis method](https://lemon-ai.com/resources/creative-attribute-analysis) decides whether the performance difference between label values is stable enough to become a test hypothesis.

Use [Attribute Analysis](https://lemon-ai.com/attribute-analysis) to define the creative questions your team cares about, tag the compatible library, inspect coverage, and connect the resulting patterns to the ads behind them. The integrity protocol on this page is the entry requirement. The performance protocol is what comes next.

## Primary sources

- [Google Gemini API: Structured outputs](https://ai.google.dev/gemini-api/docs/structured-output)
- [Google Cloud: BatchPredictionJob reference](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/reference/rpc/google.cloud.aiplatform.v1)
- [Adobe Experience Manager: Auto-tag assets with Adobe AI smart service](https://experienceleague.adobe.com/en/docs/experience-manager-cloud-service/content/assets/manage/smart-tags)
- [Adobe Experience Manager: Taxonomy and tagging best practices](https://experienceleague.adobe.com/en/perspectives/taxonomy-and-tagging-best-practices-for-aem-assets)
- [NIST Artificial Intelligence Risk Management Framework 1.0](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf)
- [NIST AI RMF Playbook: Measure](https://airc.nist.gov/airmf-resources/playbook/measure/)

On this page

- [A correct-looking tag table can still be wrong](https://lemon-ai.com/resources/ai-creative-tagging-qa#a-correct-looking-tag-table-can-still-be-wrong)
- [The eight gates](https://lemon-ai.com/resources/ai-creative-tagging-qa#the-eight-gates)
- [Gate 1: define the denominator before the run](https://lemon-ai.com/resources/ai-creative-tagging-qa#gate-1-define-the-denominator-before-the-run)
- [Gates 2 and 3: constrain the answer, then distrust it](https://lemon-ai.com/resources/ai-creative-tagging-qa#gates-2-and-3-constrain-the-answer-then-distrust-it)
- [Gates 4 and 5: reconcile work and fields separately](https://lemon-ai.com/resources/ai-creative-tagging-qa#gates-4-and-5-reconcile-work-and-fields-separately)
- [Gate 6: publish coverage by attribute](https://lemon-ai.com/resources/ai-creative-tagging-qa#gate-6-publish-coverage-by-attribute)
- [Gate 7: treat taxonomy edits as data migrations](https://lemon-ai.com/resources/ai-creative-tagging-qa#gate-7-treat-taxonomy-edits-as-data-migrations)
- [Gate 8: measure accuracy in the operating population](https://lemon-ai.com/resources/ai-creative-tagging-qa#gate-8-measure-accuracy-in-the-operating-population)
- [Eight questions for a vendor or implementation review](https://lemon-ai.com/resources/ai-creative-tagging-qa#eight-questions-for-a-vendor-or-implementation-review)
- [How Lemon applies the protocol](https://lemon-ai.com/resources/ai-creative-tagging-qa#how-lemon-applies-the-protocol)

---

Related product

- [Attribute Analysis](https://lemon-ai.com/attribute-analysis)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@id":"https://lemon-ai.com/#organization","@type":"Organization","name":"Lemon AI","url":"https://lemon-ai.com/","logo":{"@type":"ImageObject","@id":"https://lemon-ai.com/#logo","url":"https://lemon-ai.com/brand/lemon-ai-mark-512.png","contentUrl":"https://lemon-ai.com/brand/lemon-ai-mark-512.png","width":512,"height":512,"caption":"Lemon AI"},"email":"hi@lemon-ai.com","sameAs":["https://cy.linkedin.com/company/lemon-ai","https://x.com/lemon_ai_adtech","https://www.g2.com/products/lemon-ai/reviews"]},{"@id":"https://lemon-ai.com/#website","@type":"WebSite","name":"Lemon AI","url":"https://lemon-ai.com/","publisher":{"@id":"https://lemon-ai.com/#organization"},"inLanguage":"en"},{"@id":"https://lemon-ai.com/resources/ai-creative-tagging-qa#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Lemon AI","item":"https://lemon-ai.com/"},{"@type":"ListItem","position":2,"name":"Resources","item":"https://lemon-ai.com/resources"},{"@type":"ListItem","position":3,"name":"How to QA AI Creative Tags Before Performance Analysis","item":"https://lemon-ai.com/resources/ai-creative-tagging-qa"}]},{"@id":"https://lemon-ai.com/resources/ai-creative-tagging-qa#webpage","@type":"WebPage","url":"https://lemon-ai.com/resources/ai-creative-tagging-qa","name":"How to QA AI Creative Tags Before Performance Analysis","description":"An eight-gate protocol for auditing AI creative tags: reconcile every asset and field, expose coverage, migrate labels safely, and test accuracy.","isPartOf":{"@id":"https://lemon-ai.com/#website"},"publisher":{"@id":"https://lemon-ai.com/#organization"},"inLanguage":"en","datePublished":"2026-08-25","dateModified":"2026-08-25","about":{"@id":"https://lemon-ai.com/#software"},"breadcrumb":{"@id":"https://lemon-ai.com/resources/ai-creative-tagging-qa#breadcrumb"}},{"@id":"https://lemon-ai.com/#software","@type":"WebApplication","name":"Lemon AI","url":"https://lemon-ai.com/","description":"Creative intelligence and cohort-revenue forecasting for mobile app and game user-acquisition teams.","applicationCategory":"BusinessApplication","operatingSystem":"Web","browserRequirements":"Requires a modern web browser","provider":{"@id":"https://lemon-ai.com/#organization"},"featureList":["Creative-level spend, installs, purchases, revenue, and ROAS","Creative attribute analysis","Evidence-led creative generation","Cohort revenue and ROAS forecasting through D365","Read-only reporting API and MCP access"],"offers":[{"@type":"Offer","name":"Creative Intelligence annual billing","price":"209","priceCurrency":"USD","url":"https://lemon-ai.com/#pricing","description":"Effective monthly price per app when billed annually.","availability":"https://schema.org/OnlineOnly"},{"@type":"Offer","name":"Cohort Prediction monthly subscription","price":"679","priceCurrency":"USD","url":"https://lemon-ai.com/#pricing","description":"Monthly price per app after a $1,399 one-time account setup.","availability":"https://schema.org/OnlineOnly"}]},{"@id":"https://lemon-ai.com/authors/gregory-potemkin#person","@type":"Person","name":"Gregory Potemkin","jobTitle":"Founder & CEO","url":"https://lemon-ai.com/authors/gregory-potemkin","image":"https://lemon-ai.com/images/join-us/gregory.webp","sameAs":["https://www.linkedin.com/in/gregory-potemkin/"],"worksFor":{"@id":"https://lemon-ai.com/#organization"}},{"@id":"https://lemon-ai.com/resources/ai-creative-tagging-qa#article","@type":"Article","headline":"How to QA AI Creative Tags Before Performance Analysis","description":"An eight-gate protocol for auditing AI creative tags: reconcile every asset and field, expose coverage, migrate labels safely, and test accuracy.","image":"https://lemon-ai.com/og/resource-ai-creative-tagging-qa.png","datePublished":"2026-08-25","dateModified":"2026-08-25","articleSection":"Creative tagging integrity protocol","mainEntityOfPage":{"@id":"https://lemon-ai.com/resources/ai-creative-tagging-qa#webpage"},"author":{"@id":"https://lemon-ai.com/authors/gregory-potemkin#person"},"publisher":{"@id":"https://lemon-ai.com/#organization"},"inLanguage":"en"}]}
```