Pattern-019: LLM Placeholder Instruction Pattern

Status

Proven

Context

Large Language Models (LLMs) have a tendency to hallucinate or fabricate technical details when information is missing or uncertain, leading to inaccurate documentation, misleading error messages, and false technical claims. Without explicit instructions to handle uncertainty, LLMs may generate plausible-sounding but incorrect technical details, version numbers, error messages, or test results. The LLM Placeholder Instruction Pattern addresses:

Pattern Description

The LLM Placeholder Instruction Pattern prevents LLM hallucination by providing explicit instructions to use structured placeholders when information is missing or uncertain, rather than fabricating technical details. The pattern establishes three types of placeholders for different uncertainty scenarios and provides clear examples of what should not be fabricated.

Implementation

Structure

# LLM placeholder instruction framework
class LLMPlaceholderInstructor:
    def __init__(self):
        self.placeholder_types = {
            'specific_example': 'SPECIFIC EXAMPLE NEEDED',
            'fact_check': 'FACT CHECK',
            'clarifying_question': 'QUESTION'
        }

    def generate_placeholder_instructions(self) -> str:
        """Generate comprehensive placeholder instructions for LLM prompts"""
        pass

    def validate_content(self, content: str) -> ContentValidationResult:
        """Validate content for proper placeholder usage"""
        pass

    def extract_placeholders(self, content: str) -> List[PlaceholderItem]:
        """Extract all placeholders for review and completion"""
        pass

Example (Comprehensive Placeholder Instructions)

# LLM prompt instructions for technical content generation
def get_llm_placeholder_instructions() -> str:
    """
    Generate comprehensive placeholder instructions to prevent LLM hallucination
    in technical content generation
    """
    return """
## PLACEHOLDER INSTRUCTIONS (CRITICAL FOR ACCURACY):

**NEVER fabricate specific technical details not provided in the user request.**

When information is missing or uncertain, use these explicit placeholders:

### 1. SPECIFIC EXAMPLE NEEDED
Use when concrete technical details are required but not provided:
- **[SPECIFIC EXAMPLE NEEDED: exact error message displayed]**
- **[SPECIFIC EXAMPLE NEEDED: actual API response format]**
- **[SPECIFIC EXAMPLE NEEDED: precise configuration values]**
- **[SPECIFIC EXAMPLE NEEDED: actual file paths and names]**

### 2. FACT CHECK
Use for unverified claims that need validation:
- **[FACT CHECK: browser versions where issue occurs]**
- **[FACT CHECK: performance metrics and benchmarks]**
- **[FACT CHECK: testing environments and results]**
- **[FACT CHECK: version compatibility matrix]**

### 3. QUESTION
Use when clarification is needed to avoid guessing:
- **[QUESTION: Which specific deployment environment?]**
- **[QUESTION: What authentication method is being used?]**
- **[QUESTION: Are there specific performance requirements?]**
- **[QUESTION: Which user roles need this functionality?]**

### EXAMPLES OF PROPER USAGE:

❌ WRONG (Fabricated):

Error message: “Connection timeout: Unable to reach server at localhost:3000” Tested on Chrome 89, Firefox 86, Safari 14 Performance: Reduces load time by 40%


✅ CORRECT (With Placeholders):

Error message: [SPECIFIC EXAMPLE NEEDED: exact error message displayed] Tested on [FACT CHECK: browser versions and environments where issue occurs] Performance: [FACT CHECK: actual performance impact measurements]


### CRITICAL AREAS REQUIRING PLACEHOLDERS:
- Error messages and stack traces
- Version numbers and compatibility details
- Performance metrics and benchmarks
- Test results and coverage statistics
- Configuration values and environment details
- API responses and data formats
- File paths and system-specific details
"""

class TechnicalContentGenerator:
    """Generate technical content with proper placeholder usage"""

    def __init__(self):
        self.placeholder_instructions = get_llm_placeholder_instructions()

    def generate_documentation(self, user_request: str, context: Dict[str, Any]) -> str:
        """Generate documentation with placeholder instructions"""

        prompt = f"""
{self.placeholder_instructions}

## USER REQUEST:
{user_request}

## AVAILABLE CONTEXT:
{json.dumps(context, indent=2)}

## TASK:
Generate comprehensive documentation following the placeholder instructions above.
Use placeholders for any information not explicitly provided in the context.
"""

        # Send to LLM with placeholder instructions
        return self._send_to_llm(prompt)

    def validate_generated_content(self, content: str) -> ContentValidationResult:
        """Validate content for proper placeholder usage"""

        # Check for potential fabrications
        fabrication_indicators = [
            r'Error message: ["\'][^"\']*["\']',  # Quoted error messages
            r'Version \d+\.\d+\.\d+',  # Specific version numbers
            r'\d+% (improvement|reduction|faster)',  # Specific percentages
            r'Tested on [A-Z][a-z]+ \d+',  # Browser versions
        ]

        validation_issues = []
        for pattern in fabrication_indicators:
            matches = re.findall(pattern, content)
            if matches:
                validation_issues.append({
                    'type': 'potential_fabrication',
                    'pattern': pattern,
                    'matches': matches,
                    'recommendation': 'Replace with appropriate placeholder'
                })

        # Check for proper placeholder usage
        placeholder_count = len(re.findall(r'\[(?:SPECIFIC EXAMPLE NEEDED|FACT CHECK|QUESTION):[^\]]+\]', content))

        return ContentValidationResult(
            has_fabrications=len(validation_issues) > 0,
            placeholder_count=placeholder_count,
            validation_issues=validation_issues,
            recommendation=self._generate_validation_recommendation(validation_issues, placeholder_count)
        )

Example (Content Review and Completion)

class PlaceholderReviewManager:
    """Manage placeholder review and completion workflow"""

    def __init__(self):
        self.placeholder_tracker = {}

    def extract_placeholders(self, content: str) -> List[PlaceholderItem]:
        """Extract all placeholders from generated content"""

        placeholder_pattern = r'\[(SPECIFIC EXAMPLE NEEDED|FACT CHECK|QUESTION):\s*([^\]]+)\]'
        matches = re.findall(placeholder_pattern, content)

        placeholders = []
        for i, (placeholder_type, description) in enumerate(matches):
            placeholders.append(PlaceholderItem(
                id=f"placeholder_{i+1}",
                type=placeholder_type.lower().replace(' ', '_'),
                description=description.strip(),
                status='pending',
                priority=self._determine_priority(placeholder_type, description)
            ))

        return placeholders

    def create_review_checklist(self, placeholders: List[PlaceholderItem]) -> ReviewChecklist:
        """Create systematic review checklist for placeholder completion"""

        checklist_items = []

        # Group by type for systematic review
        by_type = {}
        for placeholder in placeholders:
            if placeholder.type not in by_type:
                by_type[placeholder.type] = []
            by_type[placeholder.type].append(placeholder)

        # Create checklist sections
        for placeholder_type, items in by_type.items():
            section_title = {
                'specific_example_needed': 'Technical Details to Provide',
                'fact_check': 'Claims to Verify',
                'question': 'Clarifications Needed'
            }[placeholder_type]

            checklist_items.append(ChecklistSection(
                title=section_title,
                items=[ChecklistItem(
                    id=item.id,
                    description=item.description,
                    priority=item.priority,
                    completed=False
                ) for item in items]
            ))

        return ReviewChecklist(
            title="LLM Placeholder Review",
            sections=checklist_items,
            completion_percentage=0.0
        )

    def complete_placeholder(self, placeholder_id: str, replacement_content: str) -> bool:
        """Mark placeholder as completed with replacement content"""

        if placeholder_id in self.placeholder_tracker:
            self.placeholder_tracker[placeholder_id].update({
                'status': 'completed',
                'replacement': replacement_content,
                'completed_at': datetime.utcnow().isoformat()
            })
            return True
        return False

    def generate_completion_report(self) -> PlaceholderCompletionReport:
        """Generate report on placeholder completion status"""

        total_placeholders = len(self.placeholder_tracker)
        completed_placeholders = sum(1 for p in self.placeholder_tracker.values()
                                   if p.get('status') == 'completed')

        return PlaceholderCompletionReport(
            total_placeholders=total_placeholders,
            completed_placeholders=completed_placeholders,
            completion_percentage=(completed_placeholders / total_placeholders * 100) if total_placeholders > 0 else 0,
            remaining_placeholders=[
                p for p in self.placeholder_tracker.values()
                if p.get('status') != 'completed'
            ]
        )

Usage Guidelines

When to Apply LLM Placeholder Instructions

Placeholder Type Selection

Content Review Best Practices

Integration with Workflows

Anti-Patterns to Avoid

Benefits

Trade-offs

Migration Notes (for consolidation from legacy systems)

Quality Assurance Checklist

Agent Coordination Notes

References

Last updated: September 15, 2025