HackyHour Giessen


Code - Tools - Science - Help - Social

View the Project on GitHub HackyHour/Giessen

Topic: AI models (LLMs) on your desktop computer

This is the third time we talked about an AI-topic. If you want to read the notes of a previous HackyHour about AI, look here and here.

Speaker: Nils Seipel
Notes: Christian Krippes

Nils provided his slides. Make sure to look into it, because not all graphics and code examples are included in the notes. Slides are in German.

What are LLMs - Large Language Models

Hardware requirements for ChatGPT


Game-changer: LLaMA (Large Language Model Meta AI)

This Arstechnica article describes the developement and background story in more detail.

Diversity explosion

Use cases

How to use it in practice

For code example, please take a look at the slides in the PDF document. Using a LLM locally is as easy:

  1. pip install llama-cpp-python
  2. Download a model from https://huggingface.co/TheBloke.
  3. Run it in Python (see the slides for the code)

It is important to look up what “talking” behaviour the model was trained on. The model expects input in the same way; otherwise, you’ll likely get weird results.

E.g a model was trained with the following input setup.

PROMPT: Blablabla

Security, Info, and Tools