Skip to content

Conversation

@labrenbe
Copy link
Member

@labrenbe labrenbe commented Jan 15, 2026

Adds a new demo that shows Retrieval Augmented Generation with OpenSearch

Stack:

  • OpenSearch 3.1.0 with k-NN plugin
  • Ollama running two models: nomic-embed-text:v1.5 (embeddings) & llama3.1:8b (response generation)
  • JupyterLab with interactive RAG pipeline notebook

Data

  • Documentation for the Stackable Data Platform crawled from docs.stackable.tech
  • Pre-generated json file containing ~4200 documentation chunks (93MB, stored in git)
  • Loaded into OpenSearch with a Kubernetes Job

The data file should be regenerated after each Stackable release to ensure no outdated documentation is used in the demo. This process can take more than 1 hour and is also the reason why the data is commited to git and not generated during demo installation.

Note for reviewers: You need to change the branch parameter to feat/opensearch-rag-demo in the stacks and demos files to deploy the demo. Depending on the hardware the demo should run on your local machine, but a kind cluster on Replicated with a single r1.xlarge node works best.

@labrenbe labrenbe self-assigned this Jan 15, 2026
@labrenbe labrenbe linked an issue Jan 15, 2026 that may be closed by this pull request
@labrenbe labrenbe force-pushed the feat/opensearch-rag-demo branch from 6f40f70 to 8c1beee Compare January 19, 2026 14:51
@labrenbe labrenbe marked this pull request as ready for review January 19, 2026 15:07
@labrenbe labrenbe requested a review from razvan January 23, 2026 10:08
@siegfriedweber siegfriedweber requested review from siegfriedweber and removed request for razvan January 23, 2026 10:52
@siegfriedweber siegfriedweber moved this to Development: In Progress in Stackable Engineering Jan 23, 2026
@siegfriedweber siegfriedweber requested review from razvan and removed request for siegfriedweber January 23, 2026 11:32
@razvan razvan moved this from Development: In Progress to Development: In Review in Stackable Engineering Jan 23, 2026
Copy link
Member

@razvan razvan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice. I didn't run the script to generate embeddings to the end but it ran fine. The demo also works.

Technically we could have a custom image with ollama, the models and our embeddings already prepared. This would reduce the amount of additional artifacts that need to be downloaded when running the demo and also simplify things.

@razvan razvan enabled auto-merge (squash) January 27, 2026 08:52
@razvan razvan merged commit d9119fc into main Jan 27, 2026
2 checks passed
@razvan razvan deleted the feat/opensearch-rag-demo branch January 27, 2026 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Development: In Review

Development

Successfully merging this pull request may close these issues.

Add OpenSearch AI demo

3 participants