timmy opened issue timmy/harness#4

timmy
Thu, 02 Apr 2026 18:57:59 +0000

Test Gemma 4 26B MoE - Multimodal

This issue tracks testing of vision capabilities using the Gemma 4 26B MoE model via the Hermes Profile.

Profile Location

~/.hermes/profiles/gemma4/

Test Script

#!/bin/bash
# test_gemma4_26b_multimodal.sh
# Multimodal/vision test for Gemma 4 26B MoE model

set -e

PROFILE_PATH="$HOME/.hermes/profiles/gemma4"
MODEL_NAME="gemma-4-26b-moe-it"

echo "Testing Gemma 4 26B MoE Multimodal Vision..."

# Load profile
source "$PROFILE_PATH/env"

# Test image description
hermes run --model "$MODEL_NAME" --image test_data/sample_image.jpg --prompt "Describe what you see in this image."

# Test OCR capability
hermes run --model "$MODEL_NAME" --image test_data/text_image.png --prompt "Extract all text from this image."

# Test visual reasoning
hermes run --model "$MODEL_NAME" --image test_data/chart.png --prompt "What trends do you observe in this chart?"

echo "Test completed successfully!"

Test Checklist

Profile loads correctly

Image description is accurate

OCR extracts text correctly

Visual reasoning works

Multiple image formats supported (jpg, png, webp)

MoE routing is efficient

Assignee

@timmy