# What Should a Casehouse Pilot Measure Before a Full Rollout?

issues.house · September 27, 2026

> The Direct Answer A casehouse pilot should measure whether the system improves the operating performance of issue, case, complaint, or regulatory...

## The Direct Answer

A casehouse pilot should measure whether the system improves the operating performance of issue, case, complaint, or regulatory workflows—not whether the team merely adopted new software. The most defensible pilot begins with a small, representative group of users and a clearly defined baseline period, then compares outcomes across at least 4 to 8 weeks. Useful measures include time to intake, time to assignment, time to first substantive response, median and 90th-percentile resolution time, backlog age, reopen rate, escalation rate, data completeness, and user adoption. The exact mix depends on the business: support teams may emphasize first response and backlog aging, while compliance and public-affairs teams may emphasize evidence quality, decision deadlines, stakeholder coverage, and reporting readiness.

**Also worth reading:** [How Do Casehouse Implementation Metrics Measure Support, Compliance, and Public-Affairs Operations?](https://issues.house/knowledge/how_do_casehouse_implementation_metrics_measure_support_compliance_and_public-affairs_operations.php) · [How Should a Support Organization Plan a Casehouse Rollout in 2026?](https://issues.house/knowledge/how_should_a_support_organization_plan_a_casehouse_rollout_in_2026.php) · [What Is a CLM Pilot Scorecard and How Should B2B Teams Measure It?](https://issues.house/knowledge/what_is_a_clm_pilot_scorecard_and_how_should_b2b_teams_measure_it.php)

The pilot should also establish a decision rule before launch. For example, an organization might require a 15% reduction in median handling time, no increase above 5% in reopen rate, at least 85% required-field completion, and active usage by at least 80% of the pilot group. These are proposed governance thresholds rather than universal standards, so leaders should adjust them for case complexity, regulation, and baseline performance. By 27 September 2026, the central question is less whether a casehouse platform contains workflow features and more whether those features produce measurable, repeatable operational improvement.

## Metrics That Matter Most

The first metric category is speed, but speed should be separated into stages so management can identify where delay occurs. Measure elapsed time from submission to triage, triage to assignment, assignment to first response, first response to resolution, and resolution to final closure. Include both the median and the 90th percentile: the median shows the typical experience, while the 90th percentile reveals whether difficult cases are being stranded. A fall in average handling time can conceal worsening outcomes for the most delayed cases, particularly if easy cases become easier while complex work remains unresolved.

The second category is quality and control. Track reopen rate, rework rate, escaped-error rate, escalation rate, and the percentage of records containing required evidence. For compliance-heavy work, samples may be reviewed for approval history, rationale, attachments, jurisdiction, control owner, and final disposition. A practical starting rule is to review 25 to 50 closed cases per month during a pilot, stratified by case type and complexity rather than selected solely because they went well. Inter-rater checks may also be used when multiple reviewers classify the same case, with an agreement target above 85% before automation or scoring is expanded.

The third category is adoption and operating discipline. Useful measures include weekly active users, workflow completion inside the system, percentage of cases with current owners, stale-item rate, and the share of users who complete required training. Instrumentation matters: a login does not prove that a user found value, while completing a case through the intended workflow is stronger evidence. A pilot participation target of 70% to 85% among invited users can be reasonable, but the number is not meaningful without confirming that users can perform normal work without falling back to spreadsheets, email, or chat.

## Designing a Fair Baseline and Pilot

Start by selecting one process with enough volume and a credible owner. A pilot involving only 10 cases or 2 users is unlikely to distinguish product effects from normal variation. Where case volume permits, use 20 to 50 active pilot users, or select all cases in one defined queue if the group is smaller. Run a baseline period long enough to observe different arrival patterns; 4 weeks is often workable for steady queues, while 8 to 12 weeks may be necessary for monthly, quarterly, or seasonal workflows. Compare like with like by segmenting results by case type, priority, jurisdiction, channel, and complexity.

Set the pilot period and evaluation date in advance to prevent selective reporting. A useful minimum design is two to four weeks of baseline data followed by four to eight weeks of live use, with a final data-quality review after month-end or quarter-end processing. If the organization cannot support at least four post-launch weeks, it should treat the exercise as a limited usability test rather than claim proven ROI. Record operational events such as major staffing changes, policy revisions, outages, migration defects, and unusually large case batches, because each can distort comparisons.

Use a small number of primary metrics and a broader set of guardrails. For example, median resolution time might be the primary efficiency measure, while reopen rate, 90th-percentile age, and compliance exceptions serve as guardrails. Do not declare success from a survey alone, and do not count system-generated timestamps as verified improvements without checking when users actually entered or acted on information. A pilot is strongest when conclusions combine system telemetry, case-record review, and short interviews with operators.

## A Practical Scorecard

A scorecard keeps the evaluation tied to decisions rather than a general impression of success. Each metric should have an owner, definition, baseline value, pilot value, target, and interpretation rule. Numeric changes should be reported as absolute values and percentages because a reduction from 5 days to 4 days is 20%, while a reduction from 20 minutes to 19 minutes is only 5%. Ratios with different denominators should be labeled carefully, especially when teams compare different case mixes or exclude incomplete records.

| Feature | Baseline measurement | Pilot target example | Decision implication |
| --- | --- | --- | --- |
| Median time to assignment | 18 hours | 12 hours or lower | Process efficiency is improving |
| 90th-percentile case age | 14 days | 11 days or lower | Long-tail delay is reducing |
| Required-field completeness | 76% | 90% or higher | Records are more audit-ready |
| Reopen rate | 8% | 8.5% maximum | Faster closure is not creating excessive rework |
| Weekly active users | Not tracked | 80% of invited users | Adoption is operationally credible |
| Cases completed in system | 62% | 90% or higher | Off-platform work is limited |

These numbers are examples for governance, not benchmarks that every organization must meet. Teams should define handling time consistently and decide whether waiting time belongs in the calculation. A compliance queue may intentionally require a 30-day observation period, so a shorter target could encourage premature closure and weaken quality. The scorecard should therefore distinguish avoidable delay from mandatory review or legal timelines.
Include qualitative evidence alongside the numbers. Ask users whether routing is predictable, whether the case view contains enough context, whether mandatory fields make sense, and whether escalation reaches the right person. Conduct short interviews at the midpoint and end of the pilot, then compare complaints and workarounds with the baseline. A feature used by 90% of users can still be troublesome if it adds eight manual clicks, while a lower-use approval feature may be valuable because it enforces a control.

## Comparing Casehouse Alternatives

Casehouse platforms are not identical. General case-management tools can be highly configurable but require more implementation effort; specialist issue-operations products may offer stronger prebuilt routing, reporting, and collaboration; spreadsheet-based systems may be inexpensive and familiar but difficult to audit at scale. The correct alternative is not always another SaaS vendor. Sometimes the better comparison is the current process plus a focused workflow tool, particularly when requirements are stable and the budget is limited.

| Feature | Specialist casehouse SaaS | General workflow platform | Spreadsheet-based process |
| --- | --- | --- | --- |
| Time to initial configuration | Usually faster for standard issue workflows | Moderate to long | Immediate |
| Audit trail | Commonly structured | Available, but design-dependent | Often weak or manual |
| Complex routing and permissions | Often prebuilt | Highly configurable | Manual and inconsistent |
| Reporting | Operational dashboards are common | Requires configuration | Manual formulas and charts |
| Data governance | Stronger when access and retention are configured | Depends on implementation | Weak for sensitive data |
| Upfront cost | Subscription plus implementation | Subscription, services, and configuration | Low direct cost but high labor risk |

A spreadsheet can outperform a platform on a small, low-risk queue, but it should not be assumed to provide the same controls. Sensitive support, compliance, or public-affairs information may require controlled access, retention rules, audit logs, and defensible deletion. Vendors should be asked how they handle data residency, subprocessors, encryption, exports, service levels, and termination. Procurement should also verify whether a product supports the organization’s existing identity provider, security review, incident-response obligations, and accessibility requirements.

## Cost, Pricing, and Expected ROI

Pricing varies by edition, user count, workflow complexity, data volume, implementation needs, and contract term, so published list prices are not a reliable total-cost estimate. A low-cost entry tier may be suitable for a small pilot, while enterprise pricing commonly reflects SSO, advanced permissions, custom objects, API usage, premium support, hosting requirements, and implementation services. A practical pilot budget can include 10 to 30 named users, implementation or configuration fees, training, integration work, and an internal owner’s time; the largest hidden cost is often process redesign and data cleanup rather than the license alone.

Organizations should compare total cost over 12 months, not only the monthly subscription. Include migration, storage, integrations, reporting, security review, training, support, and the labor required to correct incomplete or duplicate records. For a rough business case, calculate the expected annual value from time saved multiplied by loaded hourly labor cost, then subtract recurring software and operating costs. Run sensitivity cases at 50%, 75%, and 100% of the expected efficiency gain, because adoption and case-volume assumptions can materially change the result.

For example, if 20 users save an average of 20 minutes per case and process 1,000 cases per year, the theoretical labor capacity is 6,667 hours. That is not automatically cash savings; it may appear as redeployed capacity rather than reduced headcount. A more credible case would apply a conservative realization factor, such as 50%, and then test whether the saved time is used for higher-value work. Avoid promising a 200% ROI unless the baseline, utilization assumptions, and measurement period are documented.

## Common Pilot Mistakes

One common error is choosing users who are unusually enthusiastic while excluding frontline operators who handle difficult cases. Another is declaring success after only a few days, before the team has completed month-end, audit, or escalation cycles. A third error is measuring activity instead of outcomes: more status changes, more dashboard views, and more logins may indicate engagement, but they do not prove better decisions or faster resolution.

Data migration is another frequent weakness. If historical cases are incomplete, duplicates remain, or statuses are mapped incorrectly, users may distrust the system and return to email or spreadsheets. Set a migration acceptance rate, such as 95% of eligible records imported and at least 98% of required fields mapped or explicitly marked unavailable. Do not silently convert unknown dates or invent case categories to make a dashboard look complete. Preserve source references and document transformation rules.

Finally, avoid expanding the pilot before fixing confusing requirements. Mandatory fields should have a compliance or operational reason; mandatory fields added merely to make a report look complete create friction. Reviews, approval gates, queues, and integrations should be tested with edge cases, including rejected submissions, duplicate matters, confidential records, and overdue escalations. If a feature creates more exceptions than it resolves, revise it before rollout.

## When to Expand, Revise, or Stop

A rollout is justified when the pilot reaches its predefined efficiency and quality thresholds, users can perform core work inside the platform, and the operating owner can maintain the configuration. Expansion should still be staged—perhaps from one queue to three or from 20 users to 50—rather than moving every team on the same day. Recheck metrics at 30, 60, and 90 days after expansion because new case types can change performance. A positive pilot result should not be treated as permanent proof of ROI.

Revise the implementation when results improve for some users but not others, when speed rises while rework or compliance exceptions increase, or when adoption is concentrated in managers rather than frontline staff. For example, median handling time might fall 20%, but 90th-percentile case age could rise 30% if specialists are overloaded. That pattern calls for workload balancing and routing changes, not immediate company-wide promotion.

Stop or select another option when data cannot be trusted, the required integrations are not viable, security or regulatory requirements cannot be met, or the measured benefit is below the cost of the program. A small team may rationally retain a spreadsheet with controlled access if the process is simple, but sensitive or cross-functional operations should usually demand stronger governance. The best decision is the one supported by evidence, explicit trade-offs, and a defined review date—not the platform with the longest feature list.

## Quick answers

### How long should a casehouse pilot run?

Most pilots should run for 4 to 8 weeks after a comparable 2- to 4-week baseline. Longer cycles may be needed for quarterly compliance reviews, low-volume queues, or workflows with long approval periods. A short usability test can run for a few days, but it should not be described as a full ROI evaluation.

### What is the most important casehouse pilot metric?

There is no universal single metric, but median case-resolution time is often a useful starting point when quality and backlog measures are also tracked. For compliance workflows, evidence completeness and deadline adherence may matter more. Always pair efficiency metrics with reopen, error, or exception measures.

### How many users should participate in a casehouse pilot?

A practical pilot often includes 20 to 50 representative users, although the appropriate number depends on volume and complexity. Include frontline operators, managers, reviewers, and administrators rather than only volunteers. The group should be large enough to expose workflow problems but small enough for hands-on support.

### Can a spreadsheet work instead of casehouse SaaS?

A spreadsheet can be adequate for a small, low-risk, simple queue, especially when strong access controls and validation are available. It becomes risky for sensitive information, complex permissions, audit trails, and cross-team reporting. Compare the spreadsheet’s labor, control, and scaling costs with the platform’s total cost.

### What ROI is realistic for a casehouse pilot?

ROI varies widely by baseline efficiency, case volume, implementation effort, and how much saved time can actually be redeployed. Calculate value from measured time savings, then apply conservative realization assumptions and subtract software, services, training, and internal labor costs. Avoid relying on vendor projections without validating them against the organization’s own pilot data.

Canonical: https://issues.house/knowledge/what_should_a_casehouse_pilot_measure_before_a_full_rollout.php
Markdown: https://issues.house/knowledge/what_should_a_casehouse_pilot_measure_before_a_full_rollout.php/index.md