Initial Situation

In many large software systems, especially in enterprise environments, codebases grow over time and are maintained by multiple development teams. While such systems are actively used and extended, their documentation is often incomplete, outdated, or missing entirely. This makes onboarding, maintenance and further development more difficult.
Recent advances in large language models (LLMs) enable automated analysis and generation of natural-language descriptions from source code. These approaches promise to support developers by automatically generating technical documentation. However, the quality, correctness, and reliability of such AI-generated documentation, especially for complex and poorly documented systems, remain largely unclear.

From a software engineering perspective, documentation generation can be described as a structured process consisting of multiple steps such as code analysis, documentation generation, and validation. This process can be modeled using a subject-oriented / PASS model to illustrate the interaction between the involved components (e.g. source code, AI model and documentation output).

This thesis is conducted in cooperation with Atruvia AG and focuses on the development and evaluation of a prototype that automates developer documentation generation using AI-based methods.

Research Goal/Research Questions

What are at least 3 (better 6-10) questions or sub-questions to be answered by the theses (formulated as a question!)?

• RQ1 (Core): How effectively can an AI-based prototype generate high-quality developer documentation for complex enterprise codebases under defined evaluation criteria?
• RQ2 (Scope & Capability): Which types of developer documentation can be reliably generated from source code within the defined project scope, and what input information is required?
• RQ3 (Factual Quality): To what extent is the AI-generated documentation correct, complete, and consistent with the underlying source code when evaluated using a structured evaluation framework or expert review?
• RQ4 (Practical Usefulness): How do developers assess the understandability and usefulness of the generated documentation for onboarding, code comprehension, and maintenance?
• RQ5 (SO/PASS): What value does SO/PASS-based process modeling provide for structuring and explaining the AI-based documentation automation workflow compared to an informal process description?
• RQ6 (Scalability & Robustness): How does documentation quality and performance change as codebase size and structural complexity increase?
• RQ7 (Risks & Guidelines): Which risks arise in enterprise usage (e.g., privacy, hallucinations, IP exposure), and which practical guidelines can be derived from the prototype and evaluation results?

Planned Method + Planned structure

1) Literature Review -> All RQs
a. Review of existing approaches for developer documentation, AI-based code understanding, and the use of large language models in software engineering.
2) Analysis of Requirements and Scope Definition -> RQ2
a.Definition of the target documentation types (e.g. class-level descriptions, method summaries, module overviews, inline comments) and the corresponding input requirements (e.g. raw source code, existing partial documentation, dependency graphs). Functional and non-functional requirements for the prototype are derived from the Atruvia AG use case, including data privacy constraints relevant to enterprise deployment.
3) Conceptual Design & SO/PASS Process Modeling -> RQ5
a. Design of the overall documentation automation workflow. The process is formally modeled using Subject-Oriented Business Process Management (S-BPM) and the Parallel Activity Specification Schema (PASS) to describe the interaction between the three core subjects: Source Code, AI Model, and Documentation Output. The SO/PASS model is compared against an informal process description to assess its added value for communicating the workflow to stakeholders (addressesRQ5).
4) Prototype Implementation -> RQ1, RQ2
a. Implementation of a prototype that automatically processes source code files and generates developer documentation using AI-based methods. The prototype focuses on automation, reproducibility and applicability to multiple source files.
5) Evaluation -> RQ3, RQ4, RQ6
Evaluation of the generated documentation with respect to quality, understandability, and usefulness. Additionally, scalability aspects are analyzed by applying the prototype to codebases of increasing size.
a. Automated Quality Metrics (-> RQ3)
i. BLEU / ROUGE scores comparing generated documentation against manually written reference documentation
ii. Consistency checks: automated detection of factual contradictions between generated text and source code (e.g. wrong parameter names, incorrect return types)
iii. Completeness score: ratio of documented vs. undocumented methods/classes per file
b.
Expert Review (-> RQ3, RQ4)
i.
2–3 experienced developers (from Atruvia AG or academic context) assess a representative sample of generated documentation
ii.
Structured rating rubric covering: correctness, completeness, clarity, and actionability
iii.
Think-aloud sessions to surface qualitative insights on failure patterns
c.
Developer Survey (-> RQ4)
i.
Short structured survey (Likert scale, 5–7 items) assessing perceived understandability and usefulness
d.
Scalability Analysis (-> RQ6)
i.
Prototype applied to codebases of increasing size (e.g. 10, 50, 200+ classes) to measure documentation quality degradation, processing time, and token consumption
ii.
Identification of context-window limitations and their impact on output quality for large files
6)
Discussion and Conclusion
a.
Discussion of results identified limitations, and potential risks. Development of practical guidelines and recommendations for the use of AI-based documentation tools in enterprise software development.