Scaling Custom AI Knowledge Apps at BlackRock

This article summarizes key insights from a presentation by BlackRock's engineering team on their AI application framework. Watch the original presentation.

How do you accelerate AI deployment in a regulated environment where compliance can't be compromised? BlackRock's investment operations faced this exact challenge: complex knowledge workflows that needed to ship fast while satisfying strict regulatory requirements.

Their answer was a modular, Kubernetes-native AI framework that lets domain experts—not just engineers—shape and deploy LLM-powered apps. By splitting work into a low-code "sandbox" for iteration and an automated "app factory" for deployment, they cut delivery times from 3–8 months to days. Having architected similar systems in regulated industries, I've seen this pattern emerge as critical—the bottleneck isn't the AI models themselves, but the governance and deployment infrastructure around them.

Main Story

Investment operations are the unseen machinery behind portfolio decisions. They involve data acquisition, compliance, and post-trade processes—each with its own domain-specific complexity. BlackRock identified four AI application domains where LLMs could add value: document extraction, workflow automation, Q&A/chat, and agentic systems.

One example is “new issue operations”: setting up securities for IPOs, stock splits, or other events. This requires parsing lengthy prospectuses or term sheets, extracting structured data, validating it, and integrating it into downstream systems. Historically, each such app could take months to build, with multiple handoffs between business and engineering teams.

The new architecture changes that. The sandbox is a UI-driven environment where domain experts—without coding—can define extraction templates, set field-level rules, manage documents, and experiment with prompt strategies like retrieval-augmented generation (RAG) or chain-of-thought reasoning.

If you can get that sandbox out into the hands of the domain experts then your iteration speed becomes really fast.

Once the extraction logic is finalized, the app factory—a cloud-native operator—automatically packages it into a production-ready app. This abstracts away infrastructure details, choosing the right compute profile (GPU vs. burstable) and managing deployment, access control, and scaling.

By integrating transformation and execution into the same environment, the system removes brittle CSV/JSON handoffs. The result is faster iteration, fewer errors, and smoother compliance checks.

Technical Considerations

For engineering leaders, the model surfaces several practical lessons:

• Prompt engineering at scale is a first-class problem. Financial documents are long and complex; prompts need version control, evaluation metrics, and collaborative editing
• LLM strategy selection matters. Factors like document size, complexity, and compliance needs determine whether to use RAG, in-context learning, or hybrid approaches
• Deployment automation must account for performance and cost. Matching workloads to compute profiles avoids over-provisioning while keeping latency within acceptable bounds
• Human-in-the-loop design is non-negotiable in regulated settings. The architecture supports validation checkpoints before outputs affect downstream systems
• Integration paths should eliminate manual post-processing. Embedding transformation logic into the same environment reduces operational friction
• Security and access control are built into the deployment layer, ensuring only authorized users can trigger sensitive workflows

This approach requires strong cross-functional skills: prompt engineering expertise for domain SMEs, infrastructure automation for platform teams, and governance frameworks for compliance officers.

Business Impact & Strategy

The shift from months-long to days-long delivery cycles changes the economics of AI in operations. Leaders can now:

• Accelerate time-to-value for internal tooling
• Reduce dependency on scarce engineering resources
• Standardize AI app development across multiple domains
• Improve ROI by reusing modular components across projects

For example, a process that once took up to 8 months now ships in a couple of days. This frees engineering capacity for higher-value work and allows operations teams to respond to market events in near real-time.

Risks include over-investing in custom tooling when off-the-shelf solutions suffice, and underestimating the training needed for domain experts to become effective prompt engineers. BlackRock mitigates this by assessing ROI before productionizing and by embedding education into the rollout.

Key Insights

• Empowering domain experts with low-code AI tools accelerates iteration and reduces engineering bottlenecks
• Clear frameworks for LLM strategy selection improve performance and compliance alignment
• Integrating transformation into extraction workflows eliminates brittle, manual post-processing
• Automated deployment profiles optimize cost-performance trade-offs
• Human-in-the-loop validation is essential in regulated industries

Why It Matters

For technical and business leaders, this case study shows that AI adoption in complex, regulated environments is not just about model choice—it’s about the surrounding architecture. Giving subject matter experts the ability to design and iterate on AI-powered workflows, while automating infrastructure concerns, creates a force multiplier for both speed and governance.

The sandbox–app factory pattern is transferable to other industries where document-heavy, compliance-bound processes are common. It aligns technical flexibility with business agility, without sacrificing control.

Conclusion

BlackRock's approach demonstrates that scaling AI in a regulated enterprise is as much about process and platform as it is about the models themselves. By pairing domain-expert sandboxes with automated deployment, they've compressed delivery cycles, reduced friction, and maintained compliance.

For leaders exploring similar transformations, the key takeaway is to design for human-in-the-loop first, standardize your LLM strategies, and embed transformation into the core workflow. The BlackRock team shares more implementation details in the full presentation.

Questions or feedback? Reach out!