Sharing knowledge has always been a bit tricky. It requires time, effort and dedication. And let’s be honest, hidden knowledge too often stays hidden. But what if you could use AI to make knowledge transfer easier and more effective? That’s exactly what Databricks Genie is all about. Your tireless, always-on data analyst. By now, you’ve probably seen how AI chatbots have been overhyped. To actually deliver value, they need a clearly defined scope and domain. The same goes for Genies. Like any valuable team member, Genie needs ongoing training and fine-tuning to stay sharp. To truly elevate the user experience, an agentic Multi-Genie setup on Databricks makes all the difference. No more waiting on tickets or tracking down specialists - Genies work alongside your data team, uncovering insights the moment they’re needed.
A video is worth a thousand words
You may have explored earlier BETA versions, as outlined in this article: Creating a Multi-Genie-Agent Solution with Databricks Apps (code included), which dives deeper into the thought process logic behind the solution. This time, we’re taking things to the next level. The key new features include support for acting on-behalf-of-user, meaning the agent can now operate with the user’s permissions. Genie Spaces are now automatically available and you can activate or deactivate them on the fly for the agent to use. In addition, ReAct capabilities have been added, enabling the agent to perform significantly more complex analyses by using multiple Genie Spaces simultaneously. You can also dynamically update the LLM model used by the agent during runtime. But enough with the teasers, time to move from words to action
📦 Code repo can be found here (yes, it's fully open sourced):
Layout - all necessary with dynamic values
The entire solution runs on Databricks Apps, providing both secure access and ease of use. Permission management is handled directly at the Databricks Apps level, ensuring a streamlined experience. Also, user authentication is automatic, so each user is identified seamlessly. Every user receives their own individual session. When you restart the App, a brand new Python session is created for you, keeping user environments isolated and secure. The chat history is optimized for 25 messages, with automatic cleaning to keep the user experience friendly and prevent the conversation from becoming too heavy or confusing. You can monitor both the current message and token count in real-time via the Chat Stats feature. But please note that token count is only an approximation and containing current message history only, not tools tokens etc. It’s purpose is to give you a high-level understanding of the current token usage rather than an exact number.
Options
Activate Genie Spaces - this is quite fun feature! The solution now automatically detects the Genie Spaces that users can utilize. Moreover, it's possible to activate or deactivate Genie Spaces dynamically. You can have zero, one, or as many Genie Spaces activated as you’d like. Even better, the solution automatically converts all activated Genie Spaces into tools that the agent can use. But like with all tools, it's crucial thing to remember that tool descriptions vital. Same goes with Genie Spaces descriptions since those are being used automatically here. If the descriptions for your Genie Spaces are unclear or are totally missing, it becomes very difficult for the agent to comprehend what the Genie Space is all about.
New interactive model support - Now you can choose one of your own LLM model or use Claude Sonnet 3.7. - once again on the fly. But remember that any other LLM model you decide to use must be capable of utilizing tools - otherwise it will not function as intended and can't use Genie Spaces. It's worth noting that testing has been limited to Anthropic and OpenAI models, so broader compatibility hasn't been validated yet. Additionally, the temperature parameter has been deactivated, as many of the latest models no longer rely on this setting.
Max tokens - well you know this one already now.
Activate ReAct Agent – a cool update as well! When set to "no," the solution defaults to a simple tool-calling agent, suitable for handling straightforward tasks with ease. But when it comes to complex challenges, like using multiple Genie Spaces to analyze customer behavior based on profitability, feedback and buying patterns, while factoring in average values and company policies, the ReAct Agent truly excels.
Built on the simple ReAct pattern, the agent takes a methodical step-by-step approach to ensure accuracy, even if it’s slightly slower. If the Genie can’t provide an answer (like poorly framed prompt), the ReAct Agent doesn’t give up so easily. It continues refining prompts and interacting with the Genie to fine-tune the conversation, ensuring the Genie fully understands the request and delivers the right response. Watching agents collaborate like this feels almost magical - like having your very own data analyst tirelessly working on your behalf! So go ahead, activate ReAct and let the agent do its magic. Remember that there's a hard limit of six nodes, but you have the option to let it continue beyond this limit (just to avoid the possibility of infinite loops).
Chatbox
Chatbox now supports Markdown output, making agent replies far more engaging and visually appealing. While the agent is working, both scrolling up and sending a new message are temporarily blocked, ensuring a seamless experience without interruptions. Once the agent's reply is ready, both are automatically unblocked. The settings are a bit more flexible and you can change these options anytime you like. Just keep in mind that it takes about 1 second to activate the changes, a slight delay introduced purely for the fun visual effect (Hey who wouldnt like delightful touches like that?).
On-behalf-of-user authentication
This feature is currently in Public Preview and requires manual activation within your Databricks Workspace. Workspace admins can enable it easily by navigating to the top-right corner of the workspace and selecting Previews. If it's not activated, the solution won't function. Authentication has been streamlined by switching from Apps Service Principal (SP) authentication to OAuth 2.0 On-Behalf-Of (OBO). This approach offers a more seamless, out-of-the-box solution without the hassle of managing SP permissions. When you use the app for the first time, you'll be prompted to approve OBO usage. Simply accept and the agent will then be able to communicate with Genie Spaces using your credentials. Rest assured, the API scope is strictly limited to Genie Spaces. This ensures that your usage is appropriately logged within Genie Spaces, enabling the Genie to improve over time by leveraging better historical data. Furthermore, when the ReAct Agent is activated, it will reuse the same Genie conversation in an iterative loop to continue enhancing Genie’s performance and quality.
Model deployment? How's that
The solution features two distinct agents now: ReAct and simple one. At the moment, Mosaic AI model deployment is not utilized due to the lack of support for on-behalf-of-user authentication for deployed agents (at least for now). However, the agents have been designed with future adaptability in mind, making the switch to Mosaic AI seamless once this functionality becomes available. Additionally, MLflow tracing has been temporarily disabled (commented out) for the same reason. To ensure secure usage, the agents are implemented as classes, rather than packaged agents. Packaged agents share common configurations across all app users, which can lead to value leakage and that's a significant security concern. This design choice ensures the solution meets security standards. It might not follow the typical "textbook example" of agent deployment, but it’s specifically designed to keep user data safe and secure.
Now deployment process is even too easy
Since almost everything is fetched and handled automatically based on user permissions, deploying this solution is now incredibly easy. All you need to do is to choose LLM model you want to use and run deployment notebook. Actually, you don’t even have to do that. By default, the solution uses Claude 3.7 Sonnet, which is readily available for all Databricks users. In other words, this setup works straight out of the box. If you are a fan of DAB (Databricks Asset Bundles), the support for it has been added as well.
Once again a reminder that you need to use LLM models which supports tool usage in Databricks. This includes nearly all OpenAI and Anthropic models. However, it’s worth noting that the newly added Google Gemma 3 12B model doesn’t support tools (at least not yet).
Tested LLM models:
Claude 3.7: Works really well while being thorough
GPT-4o: Performs well
GPT-4.1 series: Performs well
*OpenAI's reasoning models: Performed decently after disabling temperature settings, though still prefer the GPT-4.1 series.
Recommended models: For the best user-experience, I'd say GPT-4.1 series models or GPT-4o. But Claude 3.7 continues to deliver the best results overall, easily.
Architecture, one more time
As you can see, not much has changed here. This highlights an important tip - always aim to design your initial architecture to be as "future-proof" as possible. This ensures that you can seamlessly update it as new features and capabilities are introduced.
Agentic Databricks Solutions - Have your own Multi-Genie assistant!
As you can see, this is a powerful, end-to-end agentic solution. Genie Spaces works exceptionally well as "data analysts", though it does require some initial teaching and fine-tuning - just like onboarding a new employee. Once you teach the right tricks and best practices, the rest falls into place. Extensive testing has been done on this solution, but if you encounter any issues, please don’t hesitate to let me know. One of the key design choices is that text output isn’t streamed. This is because each LLM model follows slightly different streaming logic and the goal was to provide a simple and consistent way to integrate various LLM models. Additionally, streaming was found to cause some unreliability in the solution, so sticking to batch text output was the more stable option.
The dynamic tool usage in this solution is a clever way to optimize agent performance and token efficiency. Adding a long, static list of tools consumes tokens heavily and can reduce agent quality. It's similar to human decision-making. The more options you have, the harder it becomes to make a choice. Think about being in a grocery store - picking something to eat from 10 options takes much longer than choosing from just 2. With this solution, users have full control to dynamically use Genie Spaces, which offers a balance of flexibility and automation. Thanks to the dynamic design, users can activate only the Genie Spaces they actually need and simply deactivate them when they're no longer required. If you're uncertain which Genie Spaces to activate, you can always enable them all and let the agent sort things out for you.
Having your own chatbot has never been easier, so start your agentic journey with this solution! Take it, use it and improve it : )
📦 Here is the repo to the solution one more time:
Written by Aarni Sillanpää
Databricks is the agentic future
