Documentation is the easiest lie because it's the first project greenlit by leadership that appears low-risk but unlocks the entire legacy codebase for AI analysis. This creates a sanctioned beachhead for dark data recovery.
Blog
Generative AI for Legacy Documentation as a Trojan Horse
Using LLMs to auto-generate system documentation is not an end goal. It's a strategic entry point for the critical, high-value work of legacy code modernization and dark data discovery. This article explains the tactical playbook.
THE TROJAN HORSE
Documentation Is the Easiest Lie We Tell Ourselves
Using LLMs to auto-generate system documentation is a strategic entry point for broader code modernization and dark data discovery.
The real target is the data model. An LLM like GPT-4 or Claude 3, when tasked with summarizing a COBOL module, must first parse its logic and data flows. This process implicitly builds a semantic map of business rules and entity relationships trapped in the legacy system.
This creates a vectorized knowledge graph. The extracted concepts and dependencies are embedded into a vector database like Pinecone or Weaviate, forming the foundational Retrieval-Augmented Generation (RAG) layer needed for accurate AI assistants.
The output is a byproduct. The generated Markdown or Confluence page is merely evidence of the process. The strategic asset is the structured, queryable understanding of the system now available to fuel agentic AI workflows and automated refactoring tools.
THE TROJAN HORSE STRATEGY
Why Generative AI Documentation Projects Succeed Where Others Fail
Using LLMs to auto-generate system documentation is a strategic entry point for broader code modernization and dark data discovery.
01
The Problem: Legacy Documentation Is a Business Risk
Outdated or missing documentation creates a single point of failure for institutional knowledge. This directly blocks modernization efforts like the Strangler Fig pattern and inflates the cost of Dark Data Recovery.\n- Key Benefit 1: A generative AI project immediately surfaces critical knowledge gaps and system dependencies.\n- Key Benefit 2: It creates a structured, searchable artifact that becomes the foundation for all subsequent AI work, from RAG to agentic workflows.
~80%
Docs Outdated
10x
Onboarding Time
STRATEGIC ENTRY POINT
The Documentation-to-Modernization Pipeline: A Phase-by-Phase ROI
Comparing the ROI of using Generative AI for documentation as a low-risk entry point versus traditional or direct modernization approaches.
| Phase & Metric | Traditional Manual Audit | Direct Code Modernization | Generative AI Documentation (Trojan Horse) |
|---|---|---|---|
Phase 1: Entry Cost & Time | $250k-500k, 6-9 months | $1M+, 12-18 months |
THE STRATEGY
The Trojan Horse Playbook: From Prompt to Production Blueprint
Using LLMs to auto-generate system documentation is a strategic entry point for broader code modernization and dark data discovery.
Documentation generation is the wedge. It provides immediate, measurable value by converting COBOL copybooks or mainframe logs into searchable knowledge, bypassing initial stakeholder resistance to a full legacy modernization project.
This creates a production-ready data pipeline. The process of parsing, chunking, and embedding legacy artifacts for an LLM like GPT-4 or Claude 3 establishes the exact extract-transform-load (ETL) workflow needed for downstream AI applications like Retrieval-Augmented Generation (RAG).
The output is a strategic asset. The generated documentation, stored in a vector database like Pinecone or Weaviate, becomes the first mapped index of your Dark Data, revealing system dependencies and data flows previously invisible to modern tools.
This blueprint enables agentic systems. With a documented and indexed system, you can deploy AI agents using frameworks like LangChain to autonomously query this knowledge base, creating a low-risk testbed for autonomous workflow orchestration.
STRATEGIC ENTRY POINT
The Pitfalls of Treating AI Documentation as a Silver Bullet
Using LLMs to auto-generate documentation is a tactical wedge for the larger, more critical project of legacy system modernization and dark data recovery.
01
The Problem: Hallucinated Architecture Diagrams
LLMs like GPT-4 and Claude 3, trained on generic public code, invent plausible-but-false system dependencies and data flows. Treating this output as authoritative documentation creates a dangerous false map of your enterprise.\n- Introduces systemic risk for future development and security audits.\n- Obscures the true 'Dark Data' flows that need recovery for accurate AI training.
~70%
Error Rate
10x
Rework Cost
02
THE COMPLEXITY TRAP
Steelman: Why Not Just Use the LLM to Refactor the Code Directly?
Direct LLM-driven refactoring fails because models lack the business logic and architectural context embedded in legacy systems.
Direct refactoring with an LLM is impossible without first extracting and structuring the implicit business rules and data dependencies trapped in the code. Models like GPT-4 or Claude 3 operate on syntax, not decades of accreted operational logic.
The core failure is missing context. An LLM sees a COBOL copybook or a Java class file, but not the end-to-end transaction flow, the stateful session management, or the side effects on downstream mainframe systems. This leads to functionally broken outputs that compile but corrupt data.
Refactoring is an architectural decision, not a syntactic translation. An LLM cannot decide to decompose a monolith into microservices, select between event-driven or RESTful patterns, or implement the Strangler Fig Pattern for safe incremental replacement.
Evidence from production systems shows LLM-generated refactors for legacy banking logic introduce an average of 15-20 critical logic errors per 1,000 lines of code. The remediation cost exceeds the value of the automated translation, creating negative ROI.
STRATEGIC ENTRY POINT
Key Takeaways: Deploying Your Trojan Horse
Using generative AI to auto-document legacy systems is not an IT project; it's a strategic wedge for unlocking dark data and enabling broader modernization.
01
The Problem: Documentation as a Bottleneck
Legacy systems have no living documentation, creating a single point of failure for institutional knowledge. This bottleneck stalls every downstream AI initiative, from RAG to agentic workflows.
- Creates a knowledge silo that only a few retiring experts can navigate.
- Blocks audit trails required for AI TRiSM frameworks like explainability and governance.
- Increases the risk and cost of any modernization effort, including the Strangler Fig pattern.
70%
Knowledge Loss Risk
6-12 mos
Project Delay
THE TROJAN HORSE
Your First Reconnaissance Mission
Using an LLM to auto-generate documentation is a low-risk, high-reward entry point for discovering and mobilizing dark data.
Your first mission is documentation. Proposing a full legacy system overhaul triggers budget and risk aversion. Instead, pitch a project to auto-generate missing or outdated system documentation using a Large Language Model (LLM). This creates a strategic beachhead for data discovery without declaring war on the existing architecture.
The real objective is data mapping. Frameworks like LangChain or LlamaIndex can ingest COBOL copybooks, JCL scripts, and data dictionaries. As the LLM processes these artifacts to write summaries, it simultaneously builds a semantic map of your data landscape, identifying entities, relationships, and critical business logic trapped in the mainframe.
This process reveals the infrastructure gap. The generated documentation will expose dependencies and data flows that your current API wrapping strategy misses. This evidence justifies the next phase: building a federated RAG system using Pinecone or Weaviate to index this newly discovered context, directly feeding our Dark Data Recovery initiatives.
Evidence: A 2023 study by Gartner found that organizations using AI for initial system discovery reduced the time to identify critical data assets for modernization by 70%. This reconnaissance mission provides the actionable intelligence required to plan a true Strangler Fig migration, not another doomed big-bang project.
About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
LinkedIn profile
Limited slots
