Talk to Your Graph Client for GraphDB

Learn how to develop a Talk to Your Graph (TTYG) client by integrating GraphDB and the OpenAI Assistants API. Step-by-step guide for smarter, customizable AI-driven interactions.

Introduction

Since I became the product manager of GraphDB, I was expected to stop writing code but I couldn’t help it. It’s certainly unorthodox but I strongly believe this makes the product better. The first version of Talk to Your Graph (or TTYG for short) was released in 2023 and it was my baby. It was certainly very promising but also a bit of a Frankenstein, put together from bits meant to do somewhat different things, so I needed to pre-chew the bigger bytes for the AI toddler. It was never meant as something ready for production.

Fast forward about a year later and TTYG v2 was born. It turned out to be a very confident teenager that still needs guidance but is way more capable than the toddler. I spent less work on chewing and I could focus on using the existing LLM technologies better. What a human and an AI model can do together is truly impressive.

Other than being more powerful, TTYG v2 pushes firmly over the production-ready threshold. One of the most frequent questions customers asked was invariably “How can we integrate this into our own products?”. So, let’s create a TTYG client together!

Core concepts

Before we get to that, let’s clarify some concepts first. TTYG v2 is based on the OpenAI Assistants API, which allows you to create assistants, threads, and messages. Each assistant can use several predefined tools to fetch additional information to answer questions that go beyond the model’s knowledge. Each thread is essentially a conversation that maintains the context of messages within. Imagine tools as different information desks where specific kinds of questions can be answered. When you ask something, the model will either reply directly (based on general model knowledge or context) or call one or more tools to get relevant information that helps it answer. Within TTYG, we call assistants – agents, threads – chats, and tools – query methods. This difference in terminology matters only in certain contexts, while in general when talking about an assistant/agent, thread/chat, or tool/query method we mean the same thing.

Before you delve further, I strongly recommend you get familiar with how TTYG works in GraphDB Workbench. We have two great examples to get you started.

Implementing a TTYG Client

There are various ways to approach a third-party integration with TTYG. One way would be a high-level API that provides more or less the same functionality as the OpenAI Assistants API – creating agents and chats, sending and receiving messages to and from those chats, all while hiding the query method calls. The other approach would be calling the OpenAI Assistants API directly and handling the query method calls yourself. The second approach is much more powerful as it allows you to maintain full control over how the chat interaction works and even modify the query method input or output to suit your needs better.

Communication with OpenAI

First, we need to establish the base communication with the OpenAI Assistants API. We’ll use the OpenAI Python library since it is a very convenient way to achieve this (other ways include the Azure OpenAI client library for Java or using the REST API directly over HTTP). It is important to note that all solutions work equally well with stock OpenAI and Azure OpenAI with only a minor difference in initialization.

Creating an OpenAI client is the first thing we need to do. Let’s create a class TTYGClient that stores the client as a field:

OPENAI_KEY = "xxx"
OPENAI_URL =
OPENAI_AZURE_VERSION =


class TTYGClient:
    def __init__(self):
        if OPENAI_URL is not None and "azure.com" in OPENAI_URL:
            self._client = AzureOpenAI(api_key=OPENAI_KEY, azure_endpoint=OPENAI_URL,
                                       api_version=OPENAI_AZURE_VERSION)
        else:
            self._client = OpenAI(api_key=OPENAI_KEY)

As you can see, we create an OpenAI or AzureOpenAI instance based on whether the URL (if provided) contains “azure.com”. Other than the creation, both types of clients are used identically.

Then, we’ll need an OpenAI thread to represent our chat and store our messages. Threads are referenced by thread ID. Let’s add a method that creates a new thread, invokes the chat interface, and then deletes the thread once the chat is over:

class TTYGClient:
    # rest of TTYGClient ...

    def run_disposable_chat(self, assistant_id):
        thread = self._client.beta.threads.create()
        print(f">>> Created thread: {thread.id}")

        try:
            self.run_chat(assistant_id, thread.id)
        finally:
            self._client.beta.threads.delete(thread.id)
            print(f">>> Deleted thread: {thread.id}")

In a real application, you will need to store the thread IDs locally and not delete the thread, if you want to reuse it later. Unfortunately, the OpenAI Assistants API does not provide a mechanism to retrieve a list of previously created threads.

Now, let’s see how a chat is handled in the run_chat() method:

class TTYGClient:
    # rest of TTYGClient ...

    def run_chat(self, assistant_id, thread_id):
        while True:
            msg = input("> ").strip()

            if msg == "":
                break

            self._ask(assistant_id, thread_id, msg)

Essentially, we run a loop asking the user for input (with a prompt that looks like > ) until an empty message is received, and then for each message (question) we run the _ask() method:

class TTYGClient:
    # rest of TTYGClient ...

    def _ask(self, assistant_id, thread_id, message):
        self._client.beta.threads.messages.create(
            thread_id=thread_id,
            role="user",
            content=message
        )

        handler = TTYGEventHandler()
        with self._client.beta.threads.runs.stream(
                thread_id=thread_id,
                assistant_id=assistant_id,
                event_handler=handler
        ) as stream:
            stream.until_done()

The first thing we do is create a new message with the “user” role. The message is created directly on the OpenAI servers and becomes a part of the thread. To process the message, we need to run the stream by letting OpenAI know which assistant we want to handle the message. Just like threads, assistants are referenced by ID. To consume the response, we need to handle the stream. This is accomplished by creating an event handler and processing the stream. The basic TTYGEventHandler is very simple:

class TTYGEventHandler(AssistantEventHandler):
    @override
    def on_text_delta(self, delta, snapshot):
        print(delta.value, end="", flush=True)

    @override
    def on_text_done(self, text):
        print()

The on_text_delta() method will be called repeatedly when a chunk of text (response) is available. This is very similar to how the UI on chatgpt.com displays the text of the response bit by bit. The on_text_done() method will be called when the entire text has been sent.

Finally, we need some code that allows us to run the program:

def main():
    ttyg = TTYGClient()

    if len(sys.argv) != 2:
        print("Usage: chat.py <assistant-id>")
        sys.exit(1)
    else:
        ttyg.run_disposable_chat(sys.argv[1])


if __name__ == '__main__':
    main()

This code expects a single argument that represents the ID of an existing assistant. If we run this, it will be able to answer only questions that don’t require any additional information fetched from GraphDB. For example, greetings and general questions about the assistant’s capabilities:

> hey!
Hello! How can I assist you today?
> what can you do?
I can help you with information about characters, space vessels, and planets from the Star Wars universe, using data stored in GraphDB. Here are some things I can do:

1. Provide details about specific characters, planets, or species.
2. Retrieve information about films, including release dates, characters, and box office data.
3. Find relationships between different entities in the Star Wars universe.
4. Answer specific questions or queries using SPARQL queries.

If you have any specific question or need information about something, just let me know!

Calling tools

As you can probably figure out at this point, there is no connection to GraphDB and thus the program won’t be able to respond to any questions about the data stored there. The OpenAI Assistants API is designed to handle tools via the same stream run/event handling mechanism as for receiving the response. We just need to handle another event by extending our TTYGEventHandler class:

class TTYGEventHandler(AssistantEventHandler):
    # rest of TTYGEventHandler code ...

    @override
    def on_event(self, event):
        if event.event == "thread.run.requires_action":
            self._handle_requires_action(event.data)

The event we are interested in is called “thread.run.requires_action”. We delegate the handling to another method and pass the event’s data:

class TTYGEventHandler(AssistantEventHandler):
    # rest of TTYGEventHandler code ...

    def _handle_requires_action(self, data):
        tool_outputs = []

        for tool in data.required_action.submit_tool_outputs.tool_calls:
            tool_name = tool.function.name
            tool_args = tool.function.arguments
            try:
                output = self._call_tool(data.assistant_id, tool_name, tool_args)
            except ValueError as ve:
                output = ve.args[0]

            if output is not None:
                print(f">>> Called tool: {tool_name}")
                tool_outputs.append({"tool_call_id": tool.id, "output": output})

        self._submit_tool_outputs(tool_outputs)

The event’s data will contain a list of tool calls, where each tool call is composed of a tool name and tool arguments. Both of these are strings, and so is the raw output of each tool call. Let’s see how the actual tool calling is done inside the _call_tool() method:

class TTYGEventHandler(AssistantEventHandler):
    # rest of TTYGEventHandler code ...

    def _call_tool(self, assistant_id, tool_name, tool_args):
        try:
            response = requests.post(
                f"{GRAPHDB_URL}/rest/ttyg/agents/{assistant_id}/{tool_name}",
                data=tool_args,
                headers={"content-type": "text/plain;charset=UTF-8", "accept": "text/plain;charset=UTF-8"},
                timeout=60)
            if response.status_code == 200:
                return response.text
            print(f">>> HTTP error: {response.status_code}")
        except requests.exceptions.ConnectionError as e:
            print(f">>> Connection error: {e}")

        return f"Fatal error calling tool {tool_name}. Do not retry and inform the user."

For each tool call, we perform a POST request to the GraphDB TTYG low-level API, where the assistant ID and tool name are passed in the URL, and the tool call arguments are passed in the request body. Remember that both the arguments and the output are simple strings so we also need to pass the appropriate content type and accept headers. Note that errors are handled by returning a descriptive error message as the tool output. This will allow the model to generate a user-friendly error message.

Finally, back in the _handle_requires_action() method, we need to submit all tool call outputs by calling the _submit_tool_outputs() method:

class TTYGEventHandler(AssistantEventHandler):
    def __init__(self, client: OpenAI):
        super().__init__()
        self._client = client

    # rest of TTYGEventHandler code ...

    def _handle_requires_action(self, data):
        tool_outputs = []

        # some code omitted for brevity …

        self._submit_tool_outputs(tool_outputs)

    def _submit_tool_outputs(self, tool_outputs):
        with self._client.beta.threads.runs.submit_tool_outputs_stream(
                tool_outputs=tool_outputs,
                run_id=self.current_run.id,
                thread_id=self.current_run.thread_id,
                event_handler=TTYGEventHandler(self._client)) as stream:
            stream.until_done()

Submitting the tool outputs is very similar to running the initial stream and we need to create a new TTYGEventHandler instance. At this point, the model will either answer the question (handled by the on_text_delta() and on_text_done() methods) or call more tools (handled by the on_event() and _handle_requires_action() methods). Note that we also added an __init__() method so we can pass the client instance, store it, and then use it in the _submit_tool_outputs() method.

That’s pretty much everything you need to get started. The basic operation of the client is illustrated in the following diagram:

The full example

The full code, hosted in a dedicated Github project, is a bit more complex and offers additional functionality:

• Thread management – storing thread IDs locally so chats can be reused later.
• Detailed debugging messages – printing tool outputs to help you understand what happens.
• Built-in commands that allow you to switch between assistants and threads, and explain how (what tools were used) a particular response was generated.
• Support for connecting to a secured GraphDB.
• Use of colors – different messages are printed using different colors for better visual separation.
• Use of assistant and thread metadata.

So what is that metadata?

The OpenAI Assistants API provides a set of custom metadata fields for both assistants and threads. Each assistant and thread can have up to 16 key-value pairs, where the key is at most 64 characters and the value is at most 512 characters. Both must be strings. In TTYG, we use the metadata to keep track of some settings, the names of threads, and so on.

Assistant metadata:

Of these, only the TTYG installation ID is potentially interesting when you implement a third-party client. The logic in GraphDB uses the TTYG installation ID to provide isolation when multiple GraphDB instances share the same OpenAI project/AzureOpenAI deployment. The full client checks the installation ID to determine if a particular assistant is visible to the application but you can skip that part when you develop your application if it doesn’t make sense for your use case. You can read more about the installation ID in the GraphDB documentation.

Thread metadata:

Even though at present the threads you create directly via the OpenAI Assistants API won’t be visible in GraphDB Workbench, it might make sense to stick to this schema in order to benefit from any future improvements that might bridge the gap between threads created in GraphDB Workbench and those created externally. The full client creates threads like this and checks the installation ID and username when reusing an existing thread. Again, that is something you can skip when you develop your application.

Example run

The example run shows how chatting and some advanced features work in the full client. Before running in, I created the two example agents from the TTYG documentation. By starting python chat.py without any arguments, I get a list of assistants (remember agents are assistants) and no threads because this is the first time I run the application:

(venv) ttyg-client % python chat.py
GraphDB Talk to Your Graph Client
Usage: chat.py <assistant-id> (<thread-id>|new)

You can provide an existing thread ID, or the special value 'new' to create a new thread.

>>> The available assistants are:

	asst_FbakX5WHK5g1lxOiKpvfmzvL (Star Wars demo)
	asst_mEGbPatYE3JF0aWOfAl0IOmc (Acme demo)

>>> There are no persisted threads.

I want to use the Star Wars demo and, since I don’t have any existing threads, I create a new thread. I need to run the program with the appropriate assistant ID and “new” as the thread ID:

(venv) ttyg-client % python chat.py asst_FbakX5WHK5g1lxOiKpvfmzvL new                            
>>> Using assistant: asst_FbakX5WHK5g1lxOiKpvfmzvL (Star Wars demo)
>>> Created thread: thread_lIYqdySLyUU8wrU6FKhXH2fa
>>> Start conversation by asking something. Press Enter (empty input) to quit.
>>> Type !help and press Enter to get a list of !-prefixed commands.
> 

Then I simply ask a question:

> who is taller, luke or han solo?
...
>>>>>> Called sparql_query, result (159 characters):
    Error: org.eclipse.rdf4j.query.MalformedQueryException: org.eclipse.rdf4j.query.parser.sparql.ast.VisitorException: QName 'voc:height' uses an undefined prefix
>>>>>> Called sparql_query, result (159 characters):
    Error: org.eclipse.rdf4j.query.MalformedQueryException: org.eclipse.rdf4j.query.parser.sparql.ast.VisitorException: QName 'voc:height' uses an undefined prefix
It seems there was an error with the query due to an undefined prefix. Let me correct that and try again.
```sparql
PREFIX voc: <https://swapi.co/vocabulary/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?character ?name ?height WHERE {
  ?character voc:height ?height;
             rdfs:label ?name.
  FILTER (?name = "Luke Skywalker" || ?name = "Han Solo")
}
```
I'll run this corrected query now.
...
>>>>>> Called sparql_query, result (133 characters):
    ?character	?name	?height
    <https://swapi.co/resource/human/1>	Luke Skywalker	172.0
    <https://swapi.co/resource/human/14>	Han Solo	180.0
Han Solo is taller than Luke Skywalker. Han Solo's height is 180.0 cm, whereas Luke Skywalker's height is 172.0 cm.
...
>

Based on the debugging information printed with the >>>>>> prefix, we see that the model tried two SPARQL queries first but it forgot to include the prefixes. Then the model corrected itself, told us (outside of the debugging information), and ran a single SPARQL query that extracted the heights of Luke Skywalker and Han Solo. The debugging information also contains the output of that query as sent to the model. Finally, we see the text response “Han Solo is taller…”.

We continue asking questions about the Star Wars world:

> where is han solo from?
...
>>>>>> Called sparql_query, result (87 characters):
    ?character	?name	?homeworld_name
    <https://swapi.co/resource/human/14>	Han Solo	Corellia
Han Solo is from the planet Corellia.
...
> who are the pilots of the millennium falcon?
...
>>>>>> Called sparql_query, result (2,323 characters):
    ?pilot_name
    "Чүбәкка"@tt-Cyrl
    "Чүбәкка"@tt
    "Чубакка"@uk
    "Чубакка"@ru
    "Τσουμπάκα"@el
    "ชิวแบคคา"@th
    "チューバッカ"@ja
    "丘巴卡"@zh-Hant
    "Chewbacca"@cs
    ...skipped some output for brevity...
    "Han Solo"@... (output truncated at 1,000)
The pilots of the Millennium Falcon are:
1. Han Solo
2. Chewbacca
3. Lando Calrissian
4. Nien Nunb
...
>

Note how the program truncates the tool output at 1,000 characters to prevent large pieces of text on the screen.

Finally, let’s see how some of the commands work:

> !help
	!help                      - display the list of commands
	!explain                   - show the tools used to answer the last question
	!list                      - show the available assistants and threads
	!assistant <assistant-id>  - switch to a different assistant
	!thread <thread-id>|new    - switch to a different thread
	!rename <name>             - rename the current thread
	!delete                    - delete the current thread

> !explain
>>> Called tool: sparql_query
    {"query":"PREFIX voc: <https://swapi.co/vocabulary/>nPREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>nnSELECT ?pilot_name WHERE {n  ?starship rdfs:label "Millennium Falcon";n            voc:pilot ?pilot.n  ?pilot rdfs:label ?pilot_name.n}"}
>

Here I used the !help command to get the list of available commands and then the !explain command to understand how the last model response was generated.

I exited the program by pressing Enter (that is, sending an empty message) and then reran it again without arguments:

(venv) ttyg-client % python chat.py
GraphDB Talk to Your Graph Client
Usage: chat.py <assistant-id> (<thread-id>|new)

You can provide an existing thread ID, or the special value 'new' to create a new thread.

>>> The available assistants are:

	asst_FbakX5WHK5g1lxOiKpvfmzvL (Star Wars demo)
	asst_mEGbPatYE3JF0aWOfAl0IOmc (Acme demo)

>>> The persisted threads are:

	thread_lIYqdySLyUU8wrU6FKhXH2fa ([Unnamed chat@2025-01-13T18:06:20])

As you can see, this time it shows one available thread, and the program assigned a generic name for it, “[Unnamed chat…]”. This is the same thread as created by the previous run and I can go back to it by passing it as the second argument when I run the program:

(venv) ttyg-client % python chat.py asst_FbakX5WHK5g1lxOiKpvfmzvL thread_lIYqdySLyUU8wrU6FKhXH2fa
>>> Using assistant: asst_FbakX5WHK5g1lxOiKpvfmzvL (Star Wars demo)
>>> Using existing thread: thread_lIYqdySLyUU8wrU6FKhXH2fa ([Unnamed chat@2025-01-13T18:06:20])
> who is taller, luke or han solo?
...
It seems there was an error with the query due to an undefined prefix. Let me correct that and try again.
```sparql
PREFIX voc: <https://swapi.co/vocabulary/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?character ?name ?height WHERE {
  ?character voc:height ?height;
             rdfs:label ?name.
  FILTER (?name = "Luke Skywalker" || ?name = "Han Solo")
}
```
I'll run this corrected query now.
...
Han Solo is taller than Luke Skywalker. Han Solo's height is 180.0 cm, whereas Luke Skywalker's height is 172.0 cm.
...
> where is han solo from?
...
Han Solo is from Corellia.
...
> who are the pilots of the millennium falcon?
...
The pilots of the Millennium Falcon are:
1. Han Solo
2. Chewbacca
3. Lando Calrissian
4. Nien Nunb
...
>>> Start conversation by asking something. Press Enter (empty input) to quit.
>>> Type !help and press Enter to get a list of !-prefixed commands.
>

This time, the program reuses the thread and prints the last couple of questions and answers to give me some context. I can continue the chat as if I never quit. Let’s explore some other !-commands instead:

> !rename These aren't the droids you're looking for
> !list
>>> The available assistants are:

	asst_FbakX5WHK5g1lxOiKpvfmzvL (Star Wars demo)
	asst_mEGbPatYE3JF0aWOfAl0IOmc (Acme demo)

>>> The persisted threads are:

	thread_lIYqdySLyUU8wrU6FKhXH2fa (These aren't the droids you're looking for)

The !rename command renamed the current chat to “These aren’t the droids you’re looking for”, while the !list command provided an overview of the available assistants and threads.

FAQs

Conclusion

With TTYG v2, we’ve transitioned from a promising prototype to a production-ready system that empowers users to bridge the gap between AI and their own applications. The TTYG client demonstrates how GraphDB can integrate seamlessly with modern AI technologies and third-party applications. By leveraging the OpenAI Assistants API and handling query methods directly, the client offers unparalleled flexibility for customization and advanced use cases that meet your unique needs.

We invite you to try out the TTYG client and explore its capabilities. Clone the repository, configure your setup, and start building your own TTYG-powered application today. If you have feedback, questions, or ideas for improvement, don’t hesitate to share them. Together, we can make TTYG even better.

Let’s take the next step in redefining how humans and machines interact with knowledge graphs!