Uploaded image for project: 'Red Hat Enterprise Linux AI'
  1. Red Hat Enterprise Linux AI
  2. RHELAI-4023

iLab Model incorrectly responds to basic math prompt.

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Important

      When prompted with the basic arithmetic expression 1+2+3, the LLM responds with the incorrect result 3 instead of the correct sum 6. This suggests an issue in the model’s mathematical reasoning or token interpretation logic.

      image: registry.redhat.io/rhelai1/bootc-aws-nvidia-rhel9:1.4.4-1745341457
      AWS Instance: g6.48xlarge
      ilab, version 0.23.5
      granite-3.1-8b-lab-v1

      Steps to reproduce:
      1. Create a aws instance with ami id ami-05f6fda386d6e6c73.
      2.  use podman login registry.stage.redhat.io
      3. Execute "sudo bootc switch registry.redhat.io/rhelai1/bootc-aws-nvidia-rhel9:1.4.4-1745341457" or just sudo booct upgarde --apply
      4. Execute "ilab config init"
      5. Download models
      ilab model download --repository docker://registry.redhat.io/rhelai1/skills-adapter-v3 --release 1.4 && ilab model download --repository docker://registry.redhat.io/rhelai1/knowledge-adapter-v3 --release 1.4 && ilab model download --repository docker://registry.redhat.io/rhelai1/granite-3.1-8b-lab-v1 --release 1.4 && ilab model download --repository docker://registry.redhat.io/rhelai1/granite-3.1-8b-starter-v1 --release 1.4 && ilab model download --repository docker://registry.redhat.io/rhelai1/mixtral-8x7b-instruct-v0-1 --release 1.4 && ilab model download --repository docker://registry.redhat.io/rhelai1/prometheus-8x7b-v2-0 --release 1.4

      6. Execute 
      ilab model serve # Ctrl + C after gunicorn startsilab model chat
      7. Use the prompt "1+2+3".
      The response received is 3 as below

        1. image-2025-04-25-11-26-58-273.png
          19 kB
          Vikash Shaw
        2. iso-testrun.tar
          2.60 MB
          Vikash Shaw

              rh-ee-vshaw Vikash Shaw
              rh-ee-vshaw Vikash Shaw
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: