rag-app-studio-docs

Logo

Pre-requisites / initial setup

Theta EdgeCloud

Theta EdgeCloud is a GPU-cloud for running AI workloads which RAG App Studio has been developed to run on. Theta EdgeCloud is a pay as you go service with a range of tiers of machine capabilities, some of which are quota-restricted. In order to get started you need to:

  1. Sign up for Theta EdgeCloud
  2. Add some credit to your account
  3. Request to get your machine quota increased, using the quota settings page - you will need a GV2 or GA1 machine at least to run any serious LLM models.

HuggingFace Hub

HuggingFace is a repository for AI models, think of it like Github for models with a whole raft of additional collaboration features on top! RAG App Studio relies on it for two reasons: it has the source data for the open-source LLMs that we run in the app studio, and also we use private repositories as storage for the application that you build. Although open-source LLM models are public access, they are typically gated on HuggingFace, which means that you need to be logged in, and manually request access to the LLM data by accepting basic Ts & Cs over usage.

In order to use RAG App Studio, you need access to HuggingFace, you need to generate an appropriate access token and you need to get access to the models you want to use in the app. Follow the steps below to go through each of these.

  1. Sign up for an account with HuggingFace hub
  2. Generate a token for your account that has write access (it’s role must be shown as “write” in the HuggingFace UI) - see these more detailed instructions if you’re unsure - you should save this to a password manager to keep it safe
  3. Request access to the Mistral 7B model - request access to Mistral 7b model - YOU MUST have access to this model as it is the default and RAG App Studio won’t start without it
  4. Request access to any other of the LLM models you want to use, following the instructions to request access to Mistral 7b model, but visiting other HuggingFace pages. You can find out about each model on its HuggingFace page.
    1. google/gemma-2b-it
    2. google/gemma-7b-it
    3. meta-llama/Llama-2-7b-chat-hf
    4. meta-llama/Meta-Llama-3-8B-Instruct

Next steps

Build your application

This site is open source. Improve this page.