Let’s Discover More API Relations: A Large Language Model-based AI Chain for Unsupervised API Relation Inference

WorkDifferentWithAI.com Academic Paper Alert!

Written by Qing Huang, Yanbang Sun, Zhenchang Xing, Yuanlong Cao, Jieshan Chen, Xiwei Xu, Huan Jin, Jiaxing Lu

Category: “AI for IT”

Article Section: AI Development and Operations; AI-Assisted Programming

Publication Date: 2023-11-02

SEO Description: “AI Chain enhances unsupervised API relation inference, leveraging large language model, with increased reliability and robustness.”

AI-Generated Paper Summary

The paper “Let’s Discover More API Relations: A Large Language Model-based AI Chain for Unsupervised API Relation Inference,” written by Qing Huang and seven other authors, presents an innovative approach to inferring API relationships. The authors propose the use of large language models (LLMs) like GPT-3.5 as a neural knowledge base to understand complex API relations. The approach takes advantage of the information present on the web used to pre-train LLMs and is not affected by the context or complexity of input texts. The authors developed an AI Chain with three modules (API FQN Parser, API Knowledge Extractor, and API Relation Decider) to ensure accurate inference. The proposed AI Chain has improved inference reliability by 67%, and their AI-crowd-intelligence strategy has increased the approach’s robustness by 26%. The method outperformed the state-of-the-art, with an average F1 score of 0.76, which is significantly higher than the average F1 score of 0.40 from the current best method.

Keywords

API Relations, Large Language Model, Unsupervised Inference, Knowledge Graphs, AI Chain

Author’s Abstract

APIs have intricate relations that can be described in text and represented as knowledge graphs to aid software engineering tasks. Existing relation extraction methods have limitations, such as limited API text corpus and affected by the characteristics of the input text.To address these limitations, we propose utilizing large language models (LLMs) (e.g., GPT-3.5) as a neural knowledge base for API relation inference. This approach leverages the entire Web used to pre-train LLMs as a knowledge base and is insensitive to the context and complexity of input texts. To ensure accurate inference, we design our analytic flow as an AI Chain with three AI modules: API FQN Parser, API Knowledge Extractor, and API Relation Decider. The accuracy of the API FQN parser and API Relation Decider module are 0.81 and 0.83, respectively. Using the generative capacity of the LLM and our approach’s inference capability, we achieve an average F1 value of 0.76 under the three datasets, significantly higher than the state-of-the-art method’s average F1 value of 0.40. Compared to CoT-based method, our AI Chain design improves the inference reliability by 67%, and the AI-crowd-intelligence strategy enhances the robustness of our approach by 26%.

Read the full paper here