Get up and running with large language models.

Download

Download

Manual install instructions

The official Ollama Docker image ollama/ollama is available on Docker Hub.

To run and chat with Llama 3:

Ollama supports a list of models available on ollama.com/library

Here are some example models that can be downloaded:

Note: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.

Ollama supports importing GGUF models in the Modelfile:

Create a file named Modelfile, with a FROM instruction with the local filepath to the model you want to import.

Create the model in Ollama

Run the model

See the guide on importing models for more information.

Models from the Ollama library can be customized with a prompt. For example, to customize the llama3 model:

Create a Modelfile:

Next, create and run the model:

For more examples, see the examples directory. For more information on working with a Modelfile, see the Modelfile documentation.

ollama create is used to create a model from a Modelfile.

This command can also be used to update a local model. Only the diff will be pulled.

For multiline input, you can wrap text with """:

ollama serve is used when you want to start ollama without running the desktop application.

See the developer guide

Next, start the server:

Finally, in a separate shell, run a model:

Ollama has a REST API for running and managing models.

See the API documentation for all endpoints.