[SME] The Failure of Deterministic Sprints in Stochastic AI: Rare Disease Patient Discovery

Question

Abstract:
Standard Scrum assumes a deterministic workflow: Effort ≈
≈ Output. In Rare Disease Patient Discovery, the workflow is stochastic. The "effort" spent on a feature (e.g., a phenotype-matching algorithm) has a non-linear relationship with the "output" (e.g., finding a statistically significant patient cohort***

When we force a stochastic process into a deterministic 2-week sprint, we introduce Technical Debt of Accuracy. Teams begin optimizing for "Sprint Completion" (Velocity) rather than "Model Sensitivity" (Patient Discovery).

The Technical Hurdle: The "Cold Start" Data Variance

In rare disease AI, we deal with the Small n
n Problem. Unlike standard ML where you iterate on a million rows, here you might be iterating on 50. A single "outlier" patient can shift the entire model's weight, rendering a "completed" sprint task obsolete instantly.
Technical Demo: The "Sprint Failure" Simulation

The following Python code demonstrates why a "committed" sprint fails in a rare disease context. It simulates the difference between a Predictable Software Task (deterministic) and a Patient Discovery Task (stochastic).

import random
import matplotlib.pyplot as plt

def simulate_work(task_type, estimated_days):
    """
    Simulates work progress. 
    'Software' is deterministic (variance is low).
    'RareDiseaseAI' is stochastic (high variance due to data anomalies).
    """
    if task_type == "Software":
        # Variance is tight around the estimate
        return estimated_days + random.uniform(-0.5, 0.5)
    else:
        # Rare Disease AI: High chance of a 'Discovery Rabbit Hole'
        # where 1 outlier changes the entire approach.
        if random.random() < 0.4: # 40% chance of a major data anomaly
            return estimated_days * random.uniform(2.0, 5.0) 
        return estimated_days + random.uniform(-1, 1)

# Scenario: 5 tasks committed to a 10-day Sprint
tasks = ["Data Ingestion", "Feature Engineering", "Model Training", "Sensitivity Analysis", "Clinical Validation"]
est = 2 # Each task estimated at 2 days

scrum_results = [simulate_work("RareDiseaseAI", est) for _ in tasks]
total_time = sum(scrum_results)
sprint_limit = 10

print(f"Total time required for 'committed' tasks: {total_time:.2f} days")
if total_time > sprint_limit:
    print(f"RESULT: Sprint Failed. Overrun by {total_time - sprint_limit:.2f} days.")
else:
    print("RESULT: Sprint Successful.")

# Visualizing the instability
plt.bar(tasks, scrum_results, color='skyblue')
plt.axhline(y=est, color='r', linestyle='--', label='Estimated Effort')
plt.ylabel('Actual Days Spent')
plt.title('Stochastic Nature of Rare Disease AI Tasks')
plt.legend()
plt.show()
enter code here

The Implementation Shift: Kanban for Stochastic Pipelines

To resolve this, we move from Sprints → Pipelines.

In a Kanban-based AI pipeline, we replace "Velocity" with Cycle Time and Lead Time. Instead of committing to a result by Friday, we commit to a process of exploration.

Proposed Technical Workflow for Rare Disease AI:

1. Backlog (Hypothesis): "Do EHR codes X,Y,ZX,Y,Z correlate with Disease AA?"
2. In-Progress (Exploration): Model training and noise reduction.
3. Clinical Review (Gate): A mandatory "Human-in-the-Loop" step where a clinician validates the findings. (This is the "Bottleneck" that Scrum fails to account for).
4. Validated (Done): The model is locked for regulatory submission.

Summary for Architects

If your AI project involves extreme data scarcity (Rare Diseases) and high regulatory risk (FDA/EMA), stop measuring Velocity. Start measuring Confidence Intervals.