Precise Semantic Retrieval: Implementing Chunk-level Vector Search with GraphDB and Elasticsearch

When working with large documents, semantic search at the document level is often insufficient. Large texts frequently contain multiple distinct topics, and indexing them as a single unit can cause truncation. This may happen either because the embeddings sent to the Graphwise Transformer client exceed 4 MB (larger documents cannot fit within gRPC limitations) or due to size restrictions of the embedding model used by the transformer.

A more precise approach is to split documents into smaller logical segments (chunks) and generate embeddings per chunk.

With the introduction of vector search within nested fields in GraphDB 11.3, it is now possible to model and index these chunks as structured nested objects in Elasticsearch while preserving their semantic integrity. This enables accurate chunk-level vector similarity search without losing the relationship between different object fields.

How to set it up

The following steps describe the complete process — from preparing chunked RDF data, through configuring the Graphwise Transformer client and GraphDB, to creating the Elasticsearch connector and executing nested vector search queries.

Step 1: Generate chunked RDF data

Use the chunker.py script to generate an RDF dataset from a TSV file. The script can be found at https://github.com/Ontotext-AD/document-chunker-rdf.

As mentioned earlier, if the source documents are very long, Elasticsearch may index only part of the content. To avoid this behavior, split each long document into smaller chunks and index each chunk separately.

Step 2: Start the Graphwise Transformer client and Elasticsearch

The Graphwise Transformer is a Python gRPC server that serves sentence embeddings via gRPC. By default, it listens on port 5050 and is started as follows:

docker build -t graphwise-transformer:$(git rev-parse –short HEAD) .

docker run –rm -p 5050:5050 -e GRAPHWISE_CONFIG=/app/config.properties graphwise-transformer:$(git rev-parse –short HEAD)

This uses the default configuration for port and sentence embeddings model – port 5050 and the sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 embedding model. Both can be changed by modifying the config.properties. Further information about the client is available in the Graphwise documentation.

Step 3: Start GraphDB with the required properties configured

GraphDB can be started using the official dockerhub image or the provided distributions for Windows, Linux, and MacOS.

Start GraphDB with the following properties configured (see the configuration reference in our documentation):

• graphwise.transformer.address=localhost:5050
• graphwise.transformer.batch.size=128
• graphwise.transformer.embedding.model.name=<model_name> if the default model has been changed 

The third property is only required if the default model has been changed. More detail on embedding model configuration is available in the documentation.

Step 4: Create a general repository in GraphDB

Open GraphDB Workbench and navigate to the repository management section. Create a new general repository. No additional configuration is required at this stage — the default settings are sufficient to proceed.

Step 5: Import the generated RDF data into the repository

Once the repository is set up, navigate to the Import menu within GraphDB Workbench. Select the RDF file generated in the previous steps and import it into the newly created repository. After the import completes, verify that the data has been loaded correctly by checking the Explore menu.

Step 6: Create an Elasticsearch index configured for vector embeddings

Each chunk will be indexed as an individual document with its own embedding, using the native:nested datatype. This enables semantic similarity search across all chunks while keeping each chunk’s fields correctly associated with one another.

 
PREFIX :
PREFIX inst:

INSERT DATA {
	inst:chunks :createConnector '''
{
  "fields": [
	{
      "fieldName": "chunks",
      "propertyChain": ["http://schema.org/hasPart"],
      "datatype": "native:nested",
      "objectFields": [
        {
          "fieldName": "text_content",
          "propertyChain": ["http://schema.org/text"]
        },
		{
          "fieldName": "index",
          "propertyChain": ["http://schema.org/position"]
        },
        {
          "fieldName": "vector",
          "propertyChain": ["http://schema.org/text"],
          "datatype": "vector"
        }
      ]
    }
  ],
  "languages": [],
  "types": ["http://schema.org/CreativeWork"],
  "readonly": false,
  "detectFields": false,
  "importGraph": false,
  "skipInitialIndexing": false,
  "elasticsearchClusterSniff": false,
  "elasticsearchNode": "http://localhost:9200",
  "manageIndex": true,
  "manageMapping": true,
  "bulkUpdateBatchSize": 5000,
  "embeddingModel" : "com.ontotext.embeddings.GraphwiseTransformerClient"
}
''' .
}

This SPARQL INSERT DATA query creates an Elasticsearch connector (index in Elasticsearch) inside GraphDB named “chunks”. It indexes RDF resources of type schema:CreativeWork as Elasticsearch documents, each containing nested chunks. (“Chunks” field is created with datatype native:nested).

For each CreativeWork, the connector follows schema:hasPart to extract chunk resources and maps their schema:text and schema:position properties into nested object fields (text_content, index). The vector field is also derived from schema:text, but because its datatype is “vector”, GraphDB generates an embedding using the configured model com.ontotext.embeddings.GraphwiseTransformerClient, which connects to the graphwise-transformer service.

With manageIndex and manageMapping enabled, GraphDB automatically creates and maintains the Elasticsearch index and mapping, performs initial indexing of existing data, and pushes updates in bulk batches.  The result is a chunk-level nested vector index that supports semantic similarity search.

Step 7: Search for documents containing the most similar chunk

The following query searches for documents containing the chunk most semantically similar to a given input:

PREFIX inst: 
PREFIX conn: 

SELECT ?doc ?score WHERE {
    ?search a inst:chunks ;
            conn:query '''{
                "size": 5,
                "query": {
	         "nested": {
                        "path": "chunks",
                        "query": {
                            "knn": {
                                "field": "chunks.vector",
                                "query_vector": "Graph Database, then import  query data with the (OpenRDF) GraphDB Workbench, and finally explore and visualise data with the build in visualisation tools.",
                                "k": 5,
                                "num_candidates": 50
                            }
                        }
                    }
                }
            }''' ;
            conn:entities ?doc .
    ?doc conn:score ?score .
}

The query executes a vector similarity search through the Elasticsearch connector inst:chunks defined in GraphDB and returns the top 5 RDF documents ranked by semantic relevance. The conn:query JSON block is passed directly to Elasticsearch and performs a nested k-nearest neighbors (kNN) search on the chunks.vector field. This means that it searches inside the nested chunks objects created by the connector.

The query_vector is supplied as raw text rather than a numeric vector. GraphDB automatically converts this text into an embedding using the configured embedding model (via the Graphwise Transformer client) before sending the kNN request to Elasticsearch. The search retrieves the 5 most similar chunk vectors (k: 5) while evaluating 50 candidates for better recall (num_candidates: 50).

The connector then maps the matching Elasticsearch documents back to their corresponding RDF resources (conn:entities ?doc) and exposes their relevance score via conn:score ?score. In this way, the SPARQL result contains each matching RDF document along with its vector similarity score. Note that while each individual chunk can only be returned once, the same parent document, for example, CreativeWork, may appear multiple times in the results if several of its chunks independently match the query.

Why nested fields matter

Each schema:CreativeWork instance contains multiple logical sub-units (chunks), and each chunk carries its own text, position, and embedding vector. Declaring chunks as a nested field in Elasticsearch ensures that each chunk is stored and queried as an independent inner document, rather than being flattened into the parent document.

Without nesting, Elasticsearch flattens arrays of objects into parallel arrays of fields, breaking field associations. For example, chunks.text_content, chunks.index, and chunks.vector would lose their one-to-one relationship. A vector match from one chunk could then incorrectly combine with metadata from a different chunk inside the same parent document. This leads to false positives and incorrect scoring. The official Elasticsearch documentation provides a good explanation of this topic.

By using nested objects, Elasticsearch internally creates hidden child documents for each chunk. The nested query evaluates similarity per chunk, keeping scoring isolated within each specific chunk context. In the kNN query above, similarity is computed against each individual chunks.vector — not against the parent document as a whole — which is what makes chunk retrieval both precise and reliable.

Want to get some hands-on experience?