hosted-models.mdx

title	description
Hosted Models	Iterate quickly with seamless access to the most popular models

Hypermode includes a set of shared models available for integration into your app on a pay-per-token basis.

Need a bespoke model? You can include a model from Hugging Face in your app manifest and Hypermode runs and manages it for you.

Setup

To use a Hypermode-hosted model, set connection: "hypermode", provider: "hugging-face", and set sourceModel to be the model name as specified on Hugging Face.

{
  ...
  "models": {
    "text-generator": {
      "sourceModel": "meta-llama/Meta-Llama-3.1-8B-Instruct",
      "provider": "hugging-face",
      "connection": "hypermode"
    }
  }
  ...
}

Deployment mode

We run our most popular models as multi-tenant, shared instances across projects and customers.

By default, if the model you use is available as a shared model, your app uses these shared models at runtime. If the model you use isn't available as a shared model, Hypermode automatically spins up a dedicated instance of the model for your project.

Shared models

These are the models available currently with shared instances:

• meta-llama/Meta-Llama-3.1-8B-Instruct
• sentence-transformers/all-MiniLM-L6-v2
• AntoineMC/distilbart-mnli-github-issues
• distilbert/distilbert-base-uncased-finetuned-sst-2-english

We're constantly evaluating model usage in determining new models to add to our shared catalog. Interested in consuming an open source model not listed here by the token? Let us know at [[email protected]](mailto:[email protected]).