Demo video: Document Understanding Solution (@NLP London)
Background
The Document Understanding Solution (DUS) is a web-based application developed at Amazon Web Services to demonstrate how machine learning can transform document workflows across industries.
It ingests and analyzes files, extracts structured information like key-value pairs, tables, and entities, and supports downstream use cases like redaction and search. DUS integrates AWS services including Amazon Textract, Amazon Comprehend, Amazon Kendra, Amazon S3, and Amazon DynamoDB.
This project was a redesign of an early internal prototype that had limited usability and was not suitable for customer-facing use. Through this redesign, I transformed it into a polished experience that could also be used by AWS sales teams as a demo tool to showcase document AI capabilities.
Strategic Context
The Document Understanding Solution serves two main purposes:
• Internally, it is used by AWS teams to process, annotate, and review document data for product development and automation research.
•Externally, it acts as a sales enablement tool. AWS field teams use the solution in customer conversations to demonstrate how AWS machine learning services can support use cases like enterprise search, document digitization, entity extraction, and compliance workflows.
The Problem:
The original prototype for document review was fragmented and unintuitive, making it difficult for internal users to inspect, correct, and annotate ML-extracted data efficiently.
At the same time, AWS sales teams lacked a reliable way to demonstrate services like enterprise search, document digitization, and compliance workflows. They often built custom demos from scratch—an approach that was time-consuming and inconsistent.
DUS was created to solve both challenges:
•Streamline internal workflows
•Provide a scalable, interactive demo tool to showcase AWS machine learning capabilities
My contribution:
As the sole UX designer on this 6-week project (3 sprints), I led the full design process—from initial discovery and workflow definition to final interface design and specification handoff.
I redesigned the existing tool from the ground up, addressing core UX issues while aligning it with real-world enterprise search and compliance use cases.
I worked cross-functionally with the product manager, engineers, and applied scientist to define the end-to-end user experience for reviewing, editing, and annotating machine-extracted document data. I identified key usability gaps in the existing tool and proposed design improvements to streamline the annotation process, reduce manual effort, and increase overall task clarity.
I delivered high-fidelity UI mockups with detailed interaction specs and supported feature prioritization to align the design with technical feasibility and business goals.
Team: 1 PM, 1 SDM, 3 Engineers, 1 Applied Scientist, and myself as the UX designer.
Discovery & Research:
To understand the complexity of document workflows and identify pain points, I collaborated with stakeholders across multiple internal teams. I conducted contextual interviews and reviewed legacy tooling to identify friction in the user journey.
Key user tasks included:
• Uploading and viewing raw documents
• Reviewing model-generated structure and entities
• Manually editing or annotating sections
• Verifying metadata accuracy
Research Methodology
To redesign the DUS experience effectively, I led a comprehensive, multi-phase research and validation process. This ensured all design decisions were grounded in user feedback and tested at multiple levels—from early wireframes through to live beta functionality.
Phase 1: Understand the Problem & Define Opportunities
In this foundational phase, I focused on understanding the user goals, system capabilities, and friction points in the existing document review experience.
Activities:
•Audited the existing UI and workflows
•Conducted stakeholder interviews to understand product goals and constraints
•Interviewed internal users to capture day-to-day pain points and identify workflow inefficiencies
•Created initial task flows and wireframes for core use cases
•Conducted early usability tests on wireframes and low-fidelity prototypes to validate concepts and interaction logic
Phase 2: Validate Core Concepts & Refine Design Direction
Building on Phase 1 findings, I developed mid- to high-fidelity design concepts for the primary workflows. These were tested with users to assess alignment with real-world tasks and usability expectations.
Activities:
•Developed interactive prototypes for core flows such as document navigation, entity editing, and annotation
•Conducted one-on-one usability sessions to gather qualitative and task-based feedback
•Facilitated prioritization discussions based on usability findings and technical feasibility
•Iterated on the interaction model for document and metadata views
Phase 3: Test Expanded Workflows & Improve Usability
Once the main interaction patterns were validated, I turned focus to improving secondary workflows and extending the design system to less common, but still critical, tasks.
Activities:
•Refined UI for edge cases and less frequent interactions (e.g., complex entity types, unusual document structures)
•Created prototypes to test alternate flows and configuration options
•Validated updated designs for the main workflows based on feedback from Phase 2
•Iteratively improved microcopy and contextual guidance within the UI
Phase 4: Beta Testing & End-to-End Validation
In the final phase, I collaborated with the engineering team to test the working instance through a beta release. This allowed for realistic, in-context validation of the full document review experience, including system feedback, UI messaging, and support content.
Activities:
•Conducted end-to-end testing of the live interface using the beta release
•Collected user feedback on flow, clarity, and overall task efficiency
•Tested in-product instructional text and contextual help for usability
•Developed and tested user-facing documentation to support onboarding and advanced usage
•Captured final refinements for engineering handoff and documentation updates
Outcome & Reflection
The redesigned Document Understanding Solution delivered a dual impact:
•It provided a streamlined, task-oriented interface that helped internal Amazon teams work more efficiently with ML-extracted document data.
•It became a strategic asset for AWS field teams, enabling pre-sales engineers and solution architects to showcase real-world applications of AWS AI/ML services in enterprise settings.
The tool continues to support both operational workflows and customer education, helping bridge the gap between machine learning capabilities and real-world document challenges.
Final Solution Highlights
The following animated previews illustrate the final user experience of the redesigned Document Understanding Solution (DUS). These moments capture how the tool enables efficient document processing, intelligent search, and sensitive data redaction—all in an interface tailored for high-complexity review tasks.
Document Upload & Search Experience
Users can upload custom documents or work with preloaded examples. The redesigned search functionality lets users locate documents by keyword or phrase, supporting faster navigation in document-heavy environments
This flow demonstrates the search and upload experience, emphasizing speed and discoverability.
AI-Powered Search Results with User Context
Search results are enhanced through semantic analysis using Amazon Kendra, alongside traditional search powered by Amazon ES. Users can filter results based on personas to explore how different roles would prioritize information—enabling more context-aware discovery.
Entity Extraction & Document Structure Detection
Once a document is opened, DUS extracts entities, key-value pairs, and table data. Medical entities and other domain-specific elements are also detected. Users can download structured data in CSV format for further analysis.
The system intelligently identifies key elements and enables users to interact with them directly.
PHI & Entity Redaction for Compliance
In compliance workflows, users can redact sensitive information—including PHI, entities, and key-value pairs—with a click. This functionality is designed for industries like healthcare or legal, where documents must be scrubbed before distribution.
