In the GenAI Era, How Can We Quickly Increase Development Velocity? Create a GenOS

In the GenAI Era, How Can We Quickly Increase Development Velocity? Create a GenOS.

Generative AI allows developers to design systems that generate responses on their own without requiring explicit coding, as it can follow instructions and reason through problems.

Numerous generative AI applications have already been developed by Intuit, such as Intuit Assist, which helps small business owners with QuickBooks; generates launch announcements for marketers in Mailchimp; assists Credit Karma members across credit ranges in making wise financial decisions; and transforms tax preparation in TurboTax. In the background, we're also utilising generative AI to power our internal developer platforms, which will increase the productivity and effectiveness of our data workers and software engineers.

And this is only the start. We are working on numerous other projects to improve the consumer and employee experiences.

Our GenOS, a proprietary operating system for creating and deploying generative AI-powered solutions, is a growing suite of tools that enable us to roll out these solutions at an accelerating speed. We created GenOS to address a huge challenge: enabling all product teams to develop solutions that responsibly and safely include generative AI into apps running on our platform. With the intention of democratising AI, Intuit has been investing in platform and AI expertise for some time now. This is only the most recent in this series.

Several opportunities are made possible by generative AI.

We encourage technologists to pursue passion projects during our twice-yearly Spring Global Engineering Days, where the first GenOS component, GenStudio, was introduced. Our engineers may responsibly access, experiment with, and build using third-party and proprietary large language models (LLMs) in GenStudio, the development sandbox. Following its launch, thousands of engineers seized the chance to work on hundreds of projects aimed at using generative AI to solve client problems.

There are significant obstacles to large-scale generative AI development.

In order to increase the potential of generative AI application development at Intuit, three major obstacles have to be overcome:

1. Responsible Development: LLMs and generative AI are dynamic fields. We must constantly assess new threats to our users and customers as they arise and come up with mitigation plans. In line with our responsible AI and data governance standards, we also take into account the security, legal, compliance, and privacy concerns linked to handling user and employee data on a single system across the business.

2. The limitations of LLM domain knowledge: Off-the-shelf solutions for Intuit's clients' most urgent financial and business needs are generally unmet by commercial and open source LLMs. The industry uses a number of approaches to address these issues, such as retrieval augmented generation, or RAG, integrating with current capabilities, and providing LLMs with context based on domain-specific data.

3. Encouragement of quick experimentation: client expectations and generative AI technologies are evolving at a very fast pace. To stay ahead of the curve, we must enable our teams to rapidly experiment with a multitude of alternative solutions, enabling them to determine the most effective means of incorporating generative AI into their applications.

We developed a patented generative AI operating system (GenOS) to address these three major issues and give a clear route for Intuit to accelerate the development of GenAI. We will look at how GenOS's main components address these issues in the following section.

Essential elements of GenOS

GenOS unifies a number of elements, each of which offers essential capability for creating and implementing experiences that make large-scale use of generative AI possible. It is comparable to a standard operating system in that it provides application developers with an expandable framework for user interactions and rendering, a runtime that controls the resources needed to execute programs, a number of services like a scheduler-like planner, short- and long-term memory, the LLMs themselves, and a pluggable ecosystem of domain-specific features akin to device drivers. To guarantee that different system components have the appropriate amount of access, it also includes role-based and use case-based access restrictions. Use case teams do not need to develop observability, governance, or cost attribution because it offers these features out of the box.

GenStudio: the sandbox for developers

A developer's sandbox for quick engineering, assessment, and optimization is offered by GenStudio. It gives teams a framework to assess and improve their prompts and agents, enabling them to gauge accuracy, factualness, and relevance. Moreover, integrated controls assist teams in rapidly iterating their prompts by testing and identifying possible problems. Additionally, GenStudio offers a virtual playground where technologists can trade off performance and cost as they create their ideas.

GenUX: An assortment of tools for user interfaces

Known as Intuit Assist, GenUX offers a collection of reusable libraries, widgets, and components made especially for generative AI-based apps at Intuit. By including user/view context, action management, conversation management, and dynamic renderers for the proprietary rendering player, it expands on the present UI engineering capabilities.

GenRuntime: The transitional zone between

The generative AI tools for creating novel product experiences are hosted by GenRuntime. GenOrchestrator, LLMs, our agents and tools ecosystem, knowledge retrieval, memory, and several registries and regulations are among the complex components that make up this system. It is worthwhile to examine these elements more closely.

GenOrchestrator: GenOS's brain

• Using generative AI, GenOrchestrator coordinates the numerous processes needed to fulfil requests to GenOS. It consists of the following main elements:

• Planner, which first ascertains the purpose of the request before determining the agents and resources needed to carry it out, plans how to respond to incoming requests.

• Executor: This uses tools and agents to carry out the plan.

• Memory and Knowledge Retrieval: these improve planning and execution by supplying pertinent context, which guarantees accuracy of the facts.

Tools & Agents: A reservoir of domain-specific expertise

Because commercial or open-source LLMs that are off-the-shelf lack Intuit's domain expertise, GenOS offers a plugin mechanism using agents and tools that allows them to be grounded in Intuit-specific data. A plugin registry is used to register agents and tools and control their lifecycle.

Registries and Policies: Safety nets for confidentiality and security

In order to meet permission, data management, legal, security, privacy, and compliance needs, GenOS registries and rules aid in enforcing the relevant boundaries. We created an extendable and adaptable framework to incorporate applicable controls, removing unnecessary labour, speeding development, and guaranteeing conformity with Intuit's responsible AI principles in order to address the broad range of applications generated using GenOS.

Custom LLMs for Financial Large Language Models

Using the best LLMs to suit our customers' demands is our approach. This includes our in-house financial Intuit LLMs, who are experts in resolving tax, accounting, marketing, cash flow, and personal finance issues. They have been specially trained on Intuit's domain-specific data. The constraints of commercial or open-source off-the-shelf LLMs are addressed by these custom LLMs, giving our teams greater capacity to handle a range of problems, such as accuracy, cost, and latency challenges. We also employ a range of top-notch open-source and commercial LLMs from top suppliers, all in accordance with our responsible data governance and AI policies.

Responsible AI: A structure for moral implementation

Our AI-driven expert platform is operated and scaled ethically by adhering to our responsible AI standards. We include controls into the GenOS architecture in addition to our Responsible AI review procedures to not only handle compliance, privacy, and security issues but also to provide satisfying and customised client experiences.

Before allowing use-case teams to onboard GenOS, we took the effort to define the rules and practices we required in place with the partners in Legal, Privacy, Security, Authorization and Access Management, and Compliance.

The Operating System's Power

It is amazing how effective GenOS has been for such a young and developing system. With numerous more being developed, more than thirty Intuit Assist use cases powered by GenOS are currently being tested and developed with customers. Building and deploying applications on GenOS across Intuit's product portfolio has brought together dozens of enterprises in an interesting adventure. Numerous groups not included in the primary goal also made important contributions to the creation of GenOS. Intuit Assist experiences across all of our platforms and products are now powered by GenOS. With its extensive feature set, GenStudio enables non-technical staff members to harness the potential of GenAI for tasks like content creation and knowledge base searches.

Six months after we first launched GenStudio, at our recent Fall Global Engineering Days, generative AI remained the main focus, with hundreds of concepts still awaiting customer testing and iteration. The timely and effective completion of this task would not have been possible without the support of GenOS and other crucial organisational strengths.

Even with all of these successes, GenOS is only getting started. We are still making investments in fast experimentation, self-serve onboarding and automation, as well as higher level abstractions like personalization. By collaborating with many special interest groups that concentrate on work streams like Reusable Agents, Evaluation, Testing, and more, GenOS is able to further improve its capabilities. Use case teams can add their own changes to the UI, Agents, and Tools, as well as guardrails, thanks to its extensibility.

We've discovered that releasing generative AI as a full-fledged operating system for developers to expand upon is the most effective approach to speed up its capabilities across products. This guarantees that, as we continuously develop GenOS, product teams maintain their attention on generative AI applications while also gaining from the organisation's aggregate knowledge.