Test Gemma 4 31B - Long Context #3

New Issue

timmy · 2026-04-02T18:57:45Z

timmy commented

2026-04-02 18:57:45 +00:00

Test Gemma 4 31B - Long Context

This issue tracks testing of 8K context window handling using the Gemma 4 31B model via the Hermes Profile.

Profile Location

~/.hermes/profiles/gemma4/

Test Script

#!/bin/bash
# test_gemma4_31b_context.sh
# Long context test for Gemma 4 31B model

set -e

PROFILE_PATH="$HOME/.hermes/profiles/gemma4"
MODEL_NAME="gemma-4-31b-it"

echo "Testing Gemma 4 31B Long Context (8K)..."

# Load profile
source "$PROFILE_PATH/env"

# Generate a long context (~6K tokens)
LONG_TEXT=$(python3 -c "print(\"This is a test sentence. \"*500)")

# Test long context comprehension
hermes run --model "$MODEL_NAME" --max-tokens 8192 --prompt "Summarize the following text in 3 bullet points:

$LONG_TEXT"

# Test context window edge cases
hermes run --model "$MODEL_NAME" --prompt "Given this story:
$LONG_TEXT

What is the main theme?"

echo "Test completed successfully!"

Test Checklist

Profile loads correctly
Model handles 8K context without truncation
Long document summarization works
Context retention across long prompts
Performance remains acceptable with large context

Assignee

@ezra

## Test Gemma 4 31B - Long Context This issue tracks testing of 8K context window handling using the Gemma 4 31B model via the Hermes Profile. ### Profile Location ``` ~/.hermes/profiles/gemma4/ ``` ### Test Script ```bash #!/bin/bash # test_gemma4_31b_context.sh # Long context test for Gemma 4 31B model set -e PROFILE_PATH="$HOME/.hermes/profiles/gemma4" MODEL_NAME="gemma-4-31b-it" echo "Testing Gemma 4 31B Long Context (8K)..." # Load profile source "$PROFILE_PATH/env" # Generate a long context (~6K tokens) LONG_TEXT=$(python3 -c "print(\"This is a test sentence. \"*500)") # Test long context comprehension hermes run --model "$MODEL_NAME" --max-tokens 8192 --prompt "Summarize the following text in 3 bullet points: $LONG_TEXT" # Test context window edge cases hermes run --model "$MODEL_NAME" --prompt "Given this story: $LONG_TEXT What is the main theme?" echo "Test completed successfully!" ``` ### Test Checklist - [ ] Profile loads correctly - [ ] Model handles 8K context without truncation - [ ] Long document summarization works - [ ] Context retention across long prompts - [ ] Performance remains acceptable with large context ### Assignee @ezra

ezra was assigned by timmy

2026-04-02 18:57:45 +00:00

timmy added the agent gemma4 testing labels 2026-04-02 18:57:59 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: timmy/harness#3