Agency Selection Scorecard: A Weighted Evaluation Matrix That Separates Capability From Price

An agency selection scorecard should let you defend a decision, not decorate one. The version that works runs in two gates: a pass/fail capability threshold that every bidder must clear with inspectable evidence, then weighted scoring of the agencies that survive, with price assessed last and never traded against a missing capability. Most templates in circulation skip the first gate and blend chemistry, claimed skill and fee into a single 1-5 sum, which tends to reward the best presenter. Timelines make the shortcut tempting. Reporting on LinkedIn data in January 2026 put the share of B2B buying decisions closing inside a two-week window at 45%, up from 38% the prior quarter; the underlying dataset was not published, so treat it as directional. This guide gives you the gate logic, a seven-row template with evidence anchors, discipline presets and a method for reading the result.

A scorecard is a measurement instrument, and a measurement is only useful when it answers a concrete decision: which agency can we award, and why did the others lose. That means every score must trace to something a second person can inspect, whether a draft SOW, a redacted report or a reference call. A claim scored as if it were evidence produces a tidy total that means nothing. The same standard applies to pitch-deck case studies. A ROAS figure without a baseline, a comparison method and a stated limitation tells you what the agency wants you to believe, not what the media delivered. Weight what you can verify, and keep price out of the room until capability is settled.

Why most agency scorecards feel rigorous and still pick the wrong partner

Most scorecards fail in three predictable ways, and each one is visible before the first agency presents.

The single-sum problem: one total hides which criteria carried the win

Templates circulating from agencies and software vendors, including the setup.us agency scorecard, HubSpot's printable two-page checklist and a widely shared Scribd upload, ask you to rate a list of criteria and add the results. The total looks objective, but it cannot tell a CFO whether the winner led on measurement capability or on presentation polish. When procurement asks why the runner-up lost, a single sum has no answer beyond "lower number".

Chemistry scored alongside capability

When cultural fit sits in the same column as delivery capability, a likeable team can offset a missing skill through arithmetic. Longer lists do not fix this. Flightdeck's 20-criteria scorecard is more thorough than a one-page checklist, yet any list that averages rapport with attribution competence still lets the room's mood move the ranking.

Claims scored as if they were evidence

The deeper flaw is that a confident answer and a documented capability earn the same points. Mighty & True's B2B evaluation scorecard makes the point about AI specifically: every agency will say yes, so the question is "whether AI has restructured their delivery in a way you can inspect". That standard belongs on every row, not only the AI one. A scorecard is a decision instrument; it has to survive scrutiny from finance, procurement and the CEO.

Open your current template and mark every row where a score could be awarded from the pitch deck alone, with no artifact, SOW clause or reference behind it. Those are the rows to rebuild.

The two-gate structure: capability thresholds before commercial scoring

The remedy is to split the evaluation into two gates so that capability is settled before anyone sees a fee.

Gate 1: pass/fail capability thresholds

Gate 1 is binary. Write four to six yes/no statements, each of which requires a document or a named person as proof. A useful default set for a media, creative or PR hire:

1. A named senior lead is committed in writing, with minimum hours, in a draft SOW.
2. The agency shows relevant category or channel work completed within the last 24 months, with a client contact.
3. The client will hold administrative ownership of all ad accounts, analytics properties and data from day one.
4. Reporting is tied to a business outcome the brief defines, not only to platform metrics.
5. The agency agrees to provide a redlined draft SOW before final scoring.

Fail any one and the agency exits before price is disclosed. There is no partial credit and no fee-based override; agree that rule with procurement before the process opens, and state it in the RFP so bidders know what to submit. GPI's advertising agency RFP template is a useful place to embed the threshold questions and the required-evidence list.

Gate 2: weighted scoring of agencies that cleared Gate 1

Weights apply only to survivors, so the weighted total compares comparable options. This also settles the pain of comparing agencies that propose entirely different strategies: the threshold questions are identical regardless of the approach pitched, and strategy quality is scored later as one weighted row rather than as the whole contest.

The setup.us existing-partner scorecard shows how the same rows can later be reused for incumbent review.

Why price is scored last and never traded against a missing capability

Keeping fee out of Gate 1 prevents a low bid from rescuing a capability gap and stops a high bid from disqualifying the strongest team before value is assessed. Where the process allows, open pricing envelopes only after capability scoring is complete, which reduces anchoring. The compressed decision windows described in the Ecommerce Times report cut the other way too: teams that run every bidder through full weighted scoring run out of time and default to gut calls. A fast binary gate is what makes rigorous scoring of the finalists affordable.

The agency selection scorecard template: field by field

Copy the seven rows below into a spreadsheet, adjust the weights so they sum to 100, and attach the required-evidence column to the RFP so agencies know what to submit before scoring begins. Weights shown are defaults for a paid media hire, an editorial starting point rather than an industry norm.

Criterion 1: Strategic fit for the next twelve months

Score how specifically the proposal addresses your brief, your channel mix and the buyers you need to reach over the coming year, not general strategic sophistication. Ask for a written plan tied to the brief's stated business decision. A 1 is a generic credentials deck; a 5 is a plan that names your constraints and the tradeoffs it accepts. Include how the agency plans for buyers who research through generative tools: Forrester reported in November 2024 that 89% of B2B buyers had adopted generative AI as a top self-guided information source (report overview only; methodology gated).

Criterion 2: Senior time and named team

Score whether the people delivering are named, with minimum monthly senior hours, in the draft SOW. Bios of the pitch team score 1. Named delivery staff with hours committed in a contract document score higher; a reference confirming those people stayed on the account earns a 5.

Criterion 3: Measurement and attribution approach

Score whether the agency states what its attribution can and cannot prove. This follows GPI's view that attribution describes association, not causation. Ask for a redacted client report and a written measurement plan. An agency that presents platform-reported ROAS as proof scores 1; one that separates platform attribution, incrementality tests and their limitations, and shows a report doing so, scores 5.

Criterion 4: AI-restructured delivery and speed model

Score inspectable change to how work gets done, not tool adoption. Request a workflow walkthrough and a median brief-to-live turnaround with QA steps. Detailed evidence requests and the three-level rubric are in the modern capability section.

Criterion 5: Verifiable results and references

Score whether case studies include a baseline, a comparison method and a stated limitation, and whether references are contactable. A top-line result with no method scores 1. The high-stakes section covers what to ask references.

Criterion 6: Commercial structure and pricing shape

Score the shape, not the number: retainer, percentage of spend or performance fee, what is excluded, and how scope changes are priced and approved. Fee level is handled separately as a ratio in the adaptation section.

Criterion 7: Working model and communication

Score cadence, escalation path and tooling as written in the SOW. This row sits last and carries the lowest weight so that chemistry cannot carry the total.

Weights in the table are editorial defaults. For contrast, Crackle PR publishes a 100-point PR rubric and Column Five offers a content agency comparison template, both discipline-specific and neither built around an evidence gate.

The 1-5 rubric with evidence anchors

Use the same anchor ladder on every row so scorers interpret points consistently: 1 means a claim only; 3 means the claim plus a document the agency produced; 5 means the claim plus a document plus third-party confirmation, such as a reference, platform access or a public record. Scores of 2 and 4 sit between anchors when the artifact is partial. The ladder gives the inspectability standard a working definition each scorer can apply the same way.

Default weights and the 100-point normalization

The defaults above sum to 100: 15, 20, 15, 15, 15, 10, 10. Multiply each row's 1-5 score by its weight and divide by 5 to produce a 100-point total. Senior time carries the most weight because it is the row agencies most often overstate and the one that most directly predicts the first ninety days. Edit the weights to reflect the decision the agency is being hired to move, then lock them in writing before the RFP goes out.

Modern capability assessment: scoring AI-restructured delivery, speed and answer-engine visibility

Add a mandatory workshop step at the shortlist stage in which each agency demonstrates one live workflow, then score inspectability with the three-level rubric before any fee discussion.

What 'AI adoption proof' means in practice

Ask for a walkthrough of one recent workflow before and after AI tooling, with the change in hours, headcount and QA steps. You are scoring the visibility of the change, not the claim, following the Mighty & True standard that a yes is worth nothing until the restructured delivery can be inspected. A 5 requires a client who confirms the workflow is how their account actually runs.

The speed model: from brief to live asset

Request the agency's median turnaround from approved brief to live creative or campaign, the range around it and where QA sits. Speed without a QA step is a risk transfer to you. A 1 is an adjective ("fast"); a 3 is a stated median with a described process; a 5 is a median backed by a project log or a reference who confirms it.

Generative engine and answer-engine visibility as a capability, not a buzzword

Buyer behavior makes this scorable. The Forrester adoption figure cited in the template section justifies asking every agency, not only PR firms, how it measures and influences brand presence in AI-generated answers. Discovered Labs opens its AEO scorecard post with an anecdote in which a prospect asked ChatGPT for vendor recommendations, evaluated three named competitors and signed, while the VP's own company was never mentioned. That is a single story from an agency blog, not a trend statistic, but it shows why a scored row is warranted. For media and creative agencies, the relevant evidence is a method for tracking brand mentions in AI answers and a plan for influencing them. For PR agencies it is closer to core work, which is why discipline-specific rubrics such as the Everything-PR 2026 criteria treat it as an evaluation dimension in its own right.

The rubric applies the same claim, document, confirmation ladder used throughout the template; the data-handling row is cross-referenced to the contract section rather than scored twice.

Red flags: renamed process, no inspectable workflow

Three signals should lower a score regardless of how the answer sounds: "we use ChatGPT" with no change to process, hours or QA; AI positioned as a surcharge or a separate product line rather than a change to how the core retainer is delivered; and any refusal to demonstrate a workflow on the grounds of confidentiality. A confidentiality concern is legitimate, but an agency that cannot show a redacted version has asked you to score a claim.

Scoring the fields agencies most often overstate: team, case studies and price

Three rows carry most of the risk of overstatement, and they need a stricter evidence standard than the rest of the matrix.

Team: the people pitching versus the people delivering

The team row scores whether named individuals and minimum senior hours appear in the draft SOW. A verbal assurance that "the people in this room run the account" scores 1. Bios attached to the SOW with hours committed score 3. A reference who confirms those individuals stayed on their account through the first year scores 5. Ask the agency directly for the hours the named lead will spend on your account each month, then check the number against their other disclosed commitments.

Case studies: demand the counterfactual and the limitation

A case study earns points for its method, not its headline. Score whether the agency states the baseline, the comparison method (holdout, geo test, pre/post with confounders acknowledged, or platform attribution alone), the attribution approach and what the result cannot prove. A top-line ROAS without any of these scores 1. Agencies that publish their own comparison frameworks, such as whitehat's 2026 marketing agency scorecard, rarely apply this standard to their own case studies, so ask for it explicitly. On reference calls, ask what changed after the first 90 days, who left the account and how the agency reported a result that went the wrong way.

For a reference point on what documented agency evidence looks like, see how the Growth Partner Index methodology scores verifiable public proof and stated limitations. The same discipline applied to a private selection is what turns a case study into evidence.

Price: score the shape and the lowball risk, not the number

Retainers, percentage of spend and performance fees each shift risk differently. A retainer fixes cost but can reward under-delivery; percentage of spend aligns the agency with budget growth rather than efficiency; performance fees depend on metrics the agency may not control. Score the clarity of scope boundaries, exclusions and the change process. Treat a bid materially below the field as a scope-risk signal to probe, not as a bonus: ask which deliverables are excluded, what hours are assumed and how overages are priced. Hypothetical illustration: if four bids cluster near a monthly retainer of 30,000 and one comes in at 15,000, the right response is a written question about assumed senior hours, not a higher commercial score.

The table is not drawn from survey data; it restates the standards above in one view.

How to evaluate a completed row

Apply a two-scorer rule to these three rows. Any score above 3 must cite the specific artifact in a notes column, and a second scorer must be able to find that artifact and agree it supports the score. Disagreements above one point trigger a follow-up request to the agency rather than an average. Averaging hides missing evidence. A follow-up request brings it into the open.

Contract and SLA evaluation: the governance rows that belong in the matrix

Request a redlined draft SOW from every shortlisted agency before final scoring, and score the contract rows from the document rather than from what the pitch team said about the document. This section is procedural guidance, not legal advice; have counsel review jurisdiction-specific terms.

Ownership of ad accounts, data and creative IP

This row is a Gate 1 threshold. The client should hold administrative access on all ad accounts and analytics properties, receive raw data exports on request and own creative, audience and first-party data assets on exit. An agency-owned account structure with "full transparency" promised verbally fails the gate. The U.S. Department of Commerce maintains an overview of privacy laws and guidance that is a useful starting reference for the data-handling obligations your contract needs to reflect, though the specifics depend on your jurisdiction and category.

Initial term, exit clause and scope-change process

Score whether the initial term is proportional to the test period the brief defines. A twelve-month lock for a hire whose value should be evident in a quarter is a mismatch. Exit notice should be symmetric; an agency that requires 90 days from you and offers 30 to itself has written its own risk down. The scope-change process should name who approves a change, how it is priced and how it is documented. Brandactive's agency evaluation criteria from 2020 remain a reasonable checklist of commercial terms to look for, though it predates current AI and data-handling concerns.

Service levels: response times, reporting cadence and raw data access

Apply the inspectability standard directly. A service level described in the pitch scores 1. A written SLA with measurable response thresholds, a reporting cadence, a named escalation path and a commitment to raw data access scores 5. Between those anchors, a written clause without measurable thresholds scores 3.

Performance clauses and what they can realistically bind

Score performance clauses for realism. A clause tied to metrics the agency does not control, such as revenue when the agency runs only prospecting media, is a red flag for later disputes, not a sign of confidence. Prefer clauses tied to controllable inputs (senior hours, turnaround, test cadence) with outcome metrics used for review rather than penalty. Add AI-specific terms: disclosure of AI use in deliverables and a written statement of which third-party tools receive client data.

The table applies the claim, document, confirmation ladder to contract rows; no external contract statistics were available in the research ledger, so the guidance is procedural.

Adapting the weights by discipline and running the scoring session

Select the preset closest to the hire, adjust no more than two weights based on your specific brief, lock the weights in writing before the RFP goes out and circulate the reconciliation rule to all scorers.

Weight presets: paid media, creative and UGC, influencer, PR and communications

The presets below are editorial defaults derived from the criteria in this article, not survey-based norms. They shift emphasis without changing the rows: PR raises answer-engine and earned-media visibility inside the AI row; creative raises speed and production model; influencer raises measurement because creator vetting and attribution are where those engagements most often fail.

Each column sums to 100. The presets are the article's own defaults; discipline-locked scorecards such as Mighty & True's B2B version and setup.us's guide to choosing an agency show how much weighting varies by author, which is a reason to set your own rather than borrow one.

Normalizing cost against capability without letting it dominate

Score commercial structure inside the matrix, then present the fee as a separate ratio: annual fee divided by weighted capability points. Hypothetical illustration: an agency scoring 82 points at 360,000 a year costs about 4,390 per point; one scoring 68 at 240,000 costs about 3,530 per point. The ratio shows the committee what it pays for each unit of demonstrated capability without letting fee contaminate the capability scores, and it makes a lowball bid visible as an outlier rather than a winner.

Running the session: independent scoring, then reconciliation

Each scorer completes the matrix alone, with artifact notes on every row above 3. The facilitator then reconciles differences above one point by requesting evidence from the agency, not by averaging. The two-scorer rule from the previous section applies to the team, results and price rows. Decide in advance who owns which rows; GPI's guide to procurement's role in agency selection covers the split between marketing and procurement in more detail.

Handling agencies that propose different strategies

The matrix scores the same seven capabilities regardless of the strategy proposed. Strategy quality is scored inside strategic fit with a rubric for specificity to the brief, so a bold but generic plan and a cautious but tailored one are compared on the same terms. The compressed decision windows reported by Ecommerce Times are a further argument for fixing weights and the session format before the RFP issues; a team that decides weights after seeing proposals will drift back to consensus by conversation.

Reading the result: what a winning score does and does not tell you

Before the award meeting, produce a sensitivity table and a one-page evidence summary per finalist, and present the ranking alongside the conditions under which it would flip.

Sensitivity check: does the winner change if one weight moves five points?

Shift each weight by five points in turn, rebalancing another row so the total stays at 100, and record whether the ranking changes. A winner that survives every shift is robust. A winner that flips when senior time moves from 20 to 15 is telling you the decision rests on one row, and that row deserves a follow-up round with both finalists rather than a signature. The check takes minutes in a spreadsheet and gives finance a concrete answer to "how close was it".

Gaps between scorers as information

Large disagreement on a row usually means one of two things: the evidence was missing and scorers filled the gap with impression, or the criterion was ambiguous. Both are fixable before award. Apply the inspectability standard in reverse: where scorers cannot name the artifact behind a score, reset that row to the claim-only level before running the sensitivity check. If the winner changes after the reset, the original ranking was carried by impression.

The scorecard supports a decision; it does not make one

The total does not prove future performance. It records that the chosen agency showed the most inspectable evidence against criteria your team chose in advance, which is the most a selection process can honestly claim. Two habits extend its value. First, convert the matrix into the first-quarter review baseline, so the same criteria that scored the pitch measure delivery against it, in the spirit of setup.us's existing-partner scorecard but with your own rows. Second, file the losing agencies' scores and artifacts with the debrief notes; if the award is challenged internally or by a bidder, the documented trail is what protects the decision.

Building your agency selection scorecard with GPI's evidence standard

The argument reduces to three moves. Run a pass/fail capability gate first, backed by documents and named people, so that no fee can rescue a missing capability. Weight only the survivors, on rows where every score above a claim traces to an artifact a second scorer can inspect. Score price last, on its shape and clarity, and present the fee as a ratio against capability rather than a column that quietly steers the total.

This is the same discipline the Growth Partner Index applies in public. GPI's methodology evaluates agencies on verifiable public evidence and states the limitations of what that evidence can show; an agency selection scorecard applies the identical standard to a private decision with more artifacts available to you than any directory can see. GPI's ownership disclosure explains how the index is funded and governed, which is the kind of transparency worth asking bidders for as well.

If you are building a shortlist, the Growth Partner Index directory lists paid, creator, Amazon and creative agencies assessed against documented evidence, a useful starting pool for Gate 1. Start there or with your own longlist, run every agency through the threshold questions before any conversation about fee, and remember that the weights are your team's decision. Tie them to the concrete business decision the agency is being hired to move, and the scorecard will tell you something worth defending.

Frequently asked questions

How many agencies should reach Gate 2 scoring?

Three to five. Fewer than three removes the comparison that makes weighted scoring meaningful; more than five multiplies workshops, reference calls and artifact reviews past what a compressed decision window can absorb. If Gate 1 leaves you with more than five, tighten a threshold (for example, category work within 18 months rather than 24) rather than loosening the evidence standard in Gate 2.

Should procurement or marketing own the weights?

Split them. Marketing sets the weights on the capability rows because it owns the business decision the agency is hired to move. Procurement owns the commercial structure and contract rows and runs the reconciliation process. Both sign the locked weights before the RFP issues, and neither changes them after proposals arrive.

What if an agency will not put named senior hours in the SOW?

Treat it as a Gate 1 failure, not a negotiation point. An agency that will not commit names and hours in writing is asking you to score a claim, and the whole matrix depends on refusing that. If the agency offers a reasonable alternative, such as a named lead with a minimum-hours floor and a substitution clause requiring your approval, that is a written commitment and can pass.

How do we score an incumbent against new bidders fairly?

Score the incumbent from current delivery artifacts: the last two quarterly reports, the current SOW and a reference call with your own team members who work with them daily. Historic relationship and goodwill are not artifacts. Apply the same evidence ladder, and expect the incumbent to score well on the working-model row and to be tested hardest on the AI-restructured delivery and measurement rows, where long-running accounts most often lag.

Should the fee be visible to scorers during capability scoring?

No, where the process allows. Have procurement hold pricing until capability scores are locked, then release the fee and compute the fee-per-point ratio. If your organization requires full proposals to be opened together, assign capability scoring to people who did not read the commercial section and have procurement score the commercial row alone.

How often should the weights be revisited as AI tooling changes?

Per hire, not per calendar. The evidence standard is what stays constant: score inspectable change to delivery rather than specific tools, and the framework survives tool turnover. When you open a new selection, review the AI and speed row's evidence requests against what buyers are doing at that moment, using current research such as the Forrester adoption data referenced earlier as a prompt, and adjust the weight by a few points if the brief warrants it.