Today I spent some time working with hf-llm.rs, a Rust library for interacting with Hugging Face models. Even though it’s still in early development, I found it surprisingly easy to get started, and things worked really well. The setup was smooth, and running the models didn’t require much effort.

One thing to note is that some models require a Hugging Face Pro subscription. When I encountered this, I decided to switch to using the model google/gemma-1.1-7b-it, which worked just as nicely for my case without needing a subscription. It’s definitely worth keeping an eye on this project as it evolves.

Some example outputs:

➜  hf-llm.rs git:(main) cargo run --release -- -m "google/gemma-1.1-7b-it" -c
    Finished `release` profile [optimized] target(s) in 0.43s
     Running `target/release/hf-llm -m google/gemma-1.1-7b-it -c`
Starting chat mode. Type 'exit' to end the conversation.
You: Write a Python function that prints the numbers from 1 to 100, but for multiples of 3, print "Fizz" instead of the number, and for multiples of 5, print "Buzz". For numbers that are multiples of both 3 and 5, print "FizzBuzz".

def fizzbuzz(limit):
    for i in range(1, limit + 1):
        if i % 3 == 0 and i % 5 == 0:
            print("FizzBuzz")
        elif i % 3 == 0:
            print("Fizz")
        elif i % 5 == 0:
            print("Buzz")
        else:
            print(i)

Example Usage:

fizzbuzz(100)

Output:

1
2
Fizz
4
Buzz
Fizz
7
8
Fizz
Buzz
11
Fizz
...

Explanation:

• The function takes one argument, limit, which defines the upper limit of the range.
• The for loop iterates over the numbers from 1 to limit.
• Inside the loop, an if-elif statement checks if the number is a multiple of both 3 and 5, 3, or 5.
• If the number is a multiple of both 3 and 5, it prints “FizzBuzz”.
• If the number is a multiple of 3, it prints “Fizz”.
• If the number is a multiple of 5, it prints “Buzz”.
• Otherwise, it prints the number.

While working with this, I also learned some Rust fundamentals that were new to me:

• Cargo and rustc:
  • Cargo: This is Rust’s tool for managing projects, dependencies, building, running, and testing code. It’s like combining pip and make.
  • rustc: The Rust compiler that actually compiles your Rust code into an executable. Cargo interacts with rustc behind the scenes during the build process, so you mostly use Cargo day-to-day.
• Cargo build with/without --release:
  • cargo build: Compiles the code in debug mode, including debugging symbols, which makes the binary larger and slower but easier to debug.
  • cargo build --release: Compiles the code with optimizations for better performance, resulting in a smaller, faster binary.
• Cargo run with/without --release:
  • cargo run: Compiles and runs your project in debug mode. It’s useful for development when you’re frequently changing code.
  • cargo run --release: Compiles your project in release mode and then runs the optimized version, useful when you’re testing performance or final code.
• Cargo.toml:
  • This file defines the project details (name, version, authors) and the dependencies your project requires. It’s similar to requirements.txt but more feature-rich.

  • Example:

    [package]
    name = "my_project"
    version = "0.1.0"
    edition = "2021"
    
    [dependencies]
    serde = "1.0"
    

• Separating Commands Using --:
  • The -- is used to separate arguments intended for Cargo from arguments that are meant for your Rust program.

  • Example:

    cargo run --release -- -m "google/gemma-1.1-7b-it" -c
    

• How to Install and Manage PATH:

  • To install Rust and Cargo, and ensure Cargo’s binaries are in your PATH, follow these steps:

    # Install Rust and Cargo using rustup
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
    
    # Add Cargo to your shell's PATH so it can be accessed globally
    export PATH="$HOME/.cargo/bin:$PATH"
    
    # Reload your shell configuration to apply the new PATH
    source ~/.zshrc  # or source ~/.bashrc
    
    # Verify that Cargo and Rust are properly installed
    cargo --version
    rustc --version
    

These are the basics I picked up while working with hf-llm.rs and learning Rust. If you’ve used Python extensively, you’ll find that Rust has some similarities, but it’s powerful and efficient in its own way.