Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-8024

[QA][PyTorch UT][CPU] inductor/test_gpu_cpp_wrapper tests are failing because of NotImplementedError

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • PyTorch
    • False
    • Hide

      None

      Show
      None
    • True
    • PyTorch Sprint 21, PyTorch Sprint 22, PyTorch Sprint 23

      inductor.test_select_algorithm tests are failing because of InductorError (AssertionError): Incorrect result from choice TritonTemplateCaller on main branch

      Tests Failing:
      test_convolution1

      Env details:
      PyTorch version: 2.10.0
      Branch: main
      OS: RHEL 9.6
      CPU: Intel
      python version: 3.12
      commit id : 6de6685797cabc6256df76803f3a5f772d5275a7 (tag: trunk/6de6685797cabc6256df76803f3a5f772d5275a7, origin/main, origin/HEAD)

      Steps to repro:
      Log in to H200.
      Login to quay.io: podman login quay.io
      Pull base image: podman pull quay.io/aipcc/pytorch:rhel_cuda_build_without_pins
      Run the image and specify the GPU to be used: podman run -it <IMAGE_NAME>
      Run the PyTorch UT: TEST_CONFIG=cpu python3 test/run_test.py -i inductor/test_select_algorithm

      Expected result: UTs should run fine.

      Actual result:
      InductorError (AssertionError): Incorrect result from choice TritonTemplateCaller. The selected TritonTemplateCaller algorithm produces incorrect results with numerical mismatch (84.7% of elements differ, greatest absolute difference: 132.32, greatest relative difference: inf), indicating the algorithm selection is choosing an incorrect implementation for the convolution operation.

      Logs are attached below

        1. inductor.test_select_algorithm.log
          300 kB
          Nayan Bhushan Kanganahalli Nagabhushana

              rh-ee-nkangana Nayan Bhushan Kanganahalli Nagabhushana
              rh-ee-nkangana Nayan Bhushan Kanganahalli Nagabhushana
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: