Uploaded image for project: 'Red Hat Enterprise Linux AI'
  1. Red Hat Enterprise Linux AI
  2. RHELAI-4522

Clarify the support status of the model serving endpoint

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • rhelai-1.5
    • Documentation
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      The introduction starts with:

      Red Hat Enterprise Linux AI is a platform that allows you to develop enterprise applications on open source Large Language Models (LLMs).

      But the only documented way to interact with models is the ilab model chat command, which is hardly a desirable programming interface.

      There is an endpoint with an industry-standard API at http://localhost:8000/v1/completions, but it's not clear whether this is a supported interface, or an internal implementation detail of ilab model chat.

      If models need to be transferred to RHEL AIIS for serving, this should be mentioned in the documentation (preferably also in the introduction).

              kelbrown@redhat.com Kelly Brown
              fweimer@redhat.com Florian Weimer
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: