Skip to content

[Feature] Remote Inference Endpoints Support #1972

Open
Feature
3 of 4 issues completed
Open
[Feature] Remote Inference Endpoints Support#1972
Feature
3 of 4 issues completed
Assignees
louie-tsaidevpramodsrinarayan-srikanthanalexsin368
Labels
BacklogfeatureNew feature or request
Milestone
v1.4

Description

@joshuayao
joshuayao
opened on May 19, 2025

Priority

P1-Stopper

OS type

Ubuntu

Hardware type

Xeon-GNR

Running nodes

Single Node

Description

The feature is from @louie-tsai:

  • GenAIExamples: enabled remote inference endpoints for AgentQnA, Productivity Suite, ChatQnA, DocSum, CodeSum, FinanceAgent, workflowExecAgent, AudioQnA, CodeTrans, MulitmodalQnA, VideoQnA, VisualQnA
  • GenAIComps : Enable remote endpoint for dynamic model switching/ endpoints without model name in it.
  • GenAIInfra: enabled remote inference endpoints in helm chats for ChatQnA, CodeGen and DocSum - Owner : Pramod

Activity

  1. joshuayao
    added this to the v1.4 milestone on May 19, 2025
  2. joshuayao
    assigned
    louie-tsai
    on May 19, 2025
  3. joshuayao
    added
    featureNew feature or request
    Backlog
    on May 19, 2025
  4. joshuayao
    added this to OPEAon May 19, 2025
  5. joshuayao
    added the
    Feature
    issue type on May 19, 2025
  6. joshuayao
    added sub-issues on May 19, 2025
  7. joshuayao
    removed this from OPEAon May 19, 2025
  8. louie-tsai
    assigned
    devpramod
    on May 27, 2025
  9. joshuayao
    added a sub-issue on May 28, 2025
  10. srinarayan-srikanthan
    self-assigned this
    on Jun 4, 2025
  11. louie-tsai
    assigned
    alexsin368
    on Jun 6, 2025

  12. srinarayan-srikanthan commented on Jun 27, 2025

    @srinarayan-srikanthan
    srinarayan-srikanthan
    on Jun 27, 2025
    Collaborator

    PRs for prodsuite and its sub examples #2085 and opea-project/GenAIComps#1806

  13. louie-tsai commented on Aug 19, 2025

    @louie-tsai
    louie-tsai
    on Aug 19, 2025
    Collaborator

    CI/CD enablement might need to move to v1.5 since PR is still under review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

BacklogfeatureNew feature or request

Type

Feature

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions