To Reproduce Steps to reproduce the behavior:

1. ilab model chat -qq 'Say "Hello" and nothing else\n'

with granite-3.1-8b-lab-v2 I got:

Good day! How can I assist you today

Run with config (temperature of 1.0): https://gitlab.com/redhat/rhel-ai/diip/-/jobs/9963635338

Expected behavior

• Previously, the response would always include 'Hello'

Screenshots

• Attached Image

Device Info (please complete the following information):

• Hardware Specs: [e.g. Apple M2 Pro Chip, 16 GB Memory, etc.]
• OS Version: [e.g. Mac OS 14.4.1, Fedora Linux 40]
• InstructLab Version: [output of \\\{{{}ilab --version{}}}]
• Provide the output of these two commands:
  • sudo bootc status --format json | jq .status.booted.image.image.image to print the name and tag of the bootc image, should look like registry.stage.redhat.io/rhelai1/bootc-intel-rhel9:1.3-1732894187
  • ilab system info to print detailed information about InstructLab version, OS, and hardware – including GPU / AI accelerator hardware

Bug impact

• Please provide information on the impact of this bug to the end user.

Known workaround

• Please add any known workarounds.

Additional context

• <your text here>
• …