Lipeng Ma

Code retrieval aims to precisely find relevant code snippets that match natural language queries within massive codebases, playing a vital role in software development. Recent advances leverage pre-trained language models (PLMs) to bridge the semantic gap between unstructured natural language (NL) and structured programming languages (PL), yielding significant improvements over traditional information retrieval and early deep learning approaches. However, existing PLM-based methods still encounter key challenges, including a fundamental semantic gap between human intent and machine execution logic, as well as limited robustness to diverse code styles. To address these issues, we propose PseudoBridge , a novel code retrieval framework that introduces pseudo-code as an intermediate, semi-structured modality to better align NL semantics with PL logic. Specifically, PseudoBridge consists of two stages: First, we employ an advanced large language model (LLM) to synthesize pseudo-code, enabling explicit alignment between NL queries and pseudo-code. Second, we introduce a logic-invariant code style augmentation strategy and employ the LLM to generate stylistically diverse yet logically equivalent code implementations with pseudo-code, then align the code snippets of different styles with pseudo-code, enhancing model robustness to code style variation. We build PseudoBridge across 10 different PLMs and evaluate it on 6 mainstream programming languages. Extensive experiments demonstrate that PseudoBridge consistently outperforms baselines, achieving significant improvements in generalization, particularly in zero-shot scenarios like Solidity and XLCoST. Extended evaluations using open-source LLMs and advanced embeddings confirm that these gains stem from PseudoBridge's intrinsic design, independent of specific closed-source models. PseudoBridge achieves performance comparable to state-of-the-art embedding methods, highlighting the effectiveness of explicit logical and semantic alignment via pseudo-code as a robust solution for code retrieval.

Automated log analysis is crucial to ensure the high availability and reliability of complex systems. The advent of Large Language Models (LLMs) in Natural Language Processing (NLP) has ushered in a new era of language model-driven automated log analysis, garnering significant interest. Within this field, two primary paradigms based on language models for log analysis have become prominent. Small Language Models (SLMs) (such as BERT) follow the pre-train and fine-tune paradigm, focusing on the specific log analysis task through fine-tuning on supervised datasets. On the other hand, LLMs (such as ChatGPT) following the in-context learning paradigm, analyze logs by providing a few examples in prompt contexts without updating parameters. Despite their respective strengths, both models exhibit inherent limitations. By comparing SLMs and LLMs, we notice that SLMs are more cost-effective but less powerful, whereas LLMs with large parameters are highly powerful but expensive and inefficient. To tradeoff between the performance and inference costs of both models in automated log analysis, this article introduces an adaptive log analysis framework known as AdaptiveLog, which effectively reduces the costs associated with LLM while ensuring superior results. This framework collaborates an LLM and an SLM, strategically allocating the LLM to tackle complex logs while delegating simpler logs to the SLM. Specifically, to efficiently query the LLM, we propose an adaptive selection strategy based on the uncertainty estimation of the SLM, where the LLM is invoked only when the SLM is uncertain. In addition, to enhance the reasoning ability of the LLM in log analysis tasks, we propose a novel prompt strategy by retrieving similar error-prone cases as the reference, enabling the model to leverage past error experiences and learn solutions from these cases. We evaluate AdaptiveLog on different log analysis tasks, Extensive experiments demonstrate that AdaptiveLog achieves state-of-the-art results across different tasks, elevating the overall accuracy of log analysis while maintaining cost efficiency. Our source code and detailed experimental data are available at https://github.com/LeaperOvO/AdaptiveLog-review .

Traffic prediction serves as a cornerstone for systems and network services such as the Web of Vehicles (WoV), online navigation, and smart city applications. Despite the proliferation of model architectures in recent years, existing approaches often suffer from highly customized structures and weak transferability, making it difficult to cope with increasing task heterogeneity and modeling complexity. To address these challenges, we propose ST-LEGO, a modular assembly framework driven by large language models (LLMs) that supports flexible structural composition and automated code generation. ST-LEGO employs a multi-agent collaborative system comprising a Prompt Agent, Assemble Agent, and Code Agent, which are responsible for understanding task requirements, dynamically assembling structural modules, and automatically generating executable PyTorch code. By introducing a standardized module library and an intermediate structural description language (DSL), the framework enables controllable generation, reusable composition, and cross-task generalization of model architectures. Empirical results on multiple real-world traffic datasets demonstrate that models generated by ST-LEGO achieve superior accuracy, structural diversity, and convergence compared to a wide range of manually designed baselines. These results highlight the unique potential and scalability of LLMs as structural architects for traffic prediction, offering a new paradigm for integrating language models into web-interactive intelligent transportation systems.

Log analysis is crucial for monitoring system health and diagnosing failures in complex systems. Recent advances in large language models (LLMs) offer new opportunities for automated log analysis, leveraging their reasoning capabilities to perform tasks such as anomaly detection and failure prediction. However, general-purpose LLMs struggle to formulate structured reasoning workflows that align with expert cognition and deliver precise details of reasoning steps. To address these challenges, we propose LogReasoner, a coarse-to-fine reasoning enhancement framework designed to enable LLMs to reason log analysis tasks like experts. LogReasoner consists of two stages: (1) coarse-grained enhancement of expert thinking, where high-level expert thoughts are constructed from collected troubleshooting flowcharts and existing tasks to enable LLMs to formulate structured reasoning workflows and (2) fine-grained enhancement of specific steps, where we first fine-tune the LLM with task-specific stepwise solutions to enhance the LLM for instantiated reasoning, then employ the preference learning to calibrate the LLM's reasoning details from its mistakes, further strengthen the LLM's analytical granularity and correctness. We evaluate LogReasoner on four distinct log analysis tasks using open-source LLMs such as Qwen-2.5 and Llama-3. Experimental results show that LogReasoner significantly outperforms existing LLMs, achieving state-of-the-art performance and demonstrating its effectiveness in enhancing the reasoning capabilities of LLMs for log analysis. Our source code and detailed experimental data are available at https://github.com/LeaperOvO/LogReasoner .

Logs are vital for monitoring service health and troubleshooting in large-scale online service systems. Recent advancements in large language models (LLMs) have demonstrated potential for automated log analysis. However, existing general LLMs face two key challenges in log analysis: a lack of domain-specific knowledge and limited adaptability to diverse log analysis tasks. To address these limitations, we propose LogInstruct, a novel LLM-based knowledge-driven instruction synthesis framework designed to enhance the log analysis capabilities of LLMs. LogInstruct bridges the domain knowledge gap by organizing log-specific knowledge into a structured knowledge tree to guide the synthesis of instruction. To ensure the diversity of instructions and hierarchical log analysis skills, we leverage Bloom's Taxonomy to synthesize log instructions of varying difficulty levels, not only covering a wide range of log analysis scenarios but also progressively challenging the model's reasoning abilities. LogInstruct synthesizes 88 K high-quality instruction datasets without manual intervention. Experiments demonstrate that open-source LLMs fine-tuned on our dataset achieve state-of-the-art performance across five distinct log analysis tasks, outperforming proprietary LLMs such as GPT-4o. Our source code and synthesized data are available at https://github.com/LeaperOvO/LogInstruct.

Point cloud completion seeks to reconstruct a complete and high-fidelity point cloud from an incomplete and low-quality input. Current methods predominantly rely on Transformer architectures for feature extraction. However, these approaches face two major limitations, including the computational complexity associated with the attention mechanism and the potential loss of fine-grained details during pooling operations. These issues hinder their performance on large-scale and highly fragmented point clouds. To overcome these challenges, we propose 3DMambaComplete, a novel point cloud completion method based on the selective State Space Model (SSM), particularly leveraging the Mamba architecture. Unlike traditional Transformer-based methods, 3DMambaComplete utilizes Mamba’s linear-time complexity to efficiently extract global features with significantly reduced computational overhead. Furthermore, we introduce the concepts of discriminative nodes, referred to as hyperpoints, along with dynamic offsets, to improve reconstruction quality. Specifically, the HyperPoint Generation Module encodes the downsampled features of the point cloud using the Mamba Encoder, producing a set of hyperpoints that capture critical information. Subsequently, the HyperPoint Spread Module disperses these hyperpoints across various spatial locations employing dynamic offsets to mitigate aggregation. Finally, the Point Deformation Module implements a deformation technique to transform the 2D mesh into a detailed 3D structure, resulting in high-quality point cloud completions. Experiments on widely used benchmark datasets show that 3DMambaComplete outperforms existing point cloud completion techniques in both quantitative and qualitative evaluations.

With the growing demand in real-world applications, learning to understand 3D data has become increasingly critical for a variety of computer graphics tasks, including shape classification, model retrieval, scene reconstruction, and point cloud completion. While previous methods have explored self-supervised learning within single modalities, such as point clouds or images, the potential of multi-modal supervision remains underexplored due to the lack of aligned and scalable training signals. In this work, we present DR-Point, a tri-modal pre-training framework that jointly learns from RGB images, depth maps, and 3D point clouds to construct a unified embedding space across modalities. By leveraging cross-modal consistency among modality triplets, DR-Point enables effective alignment of 2D-3D features without requiring manual annotations. To further enhance geometric fidelity, we incorporate a differentiable rendering module that synthesizes depth information and refines structural details in reconstructed point clouds. This promotes improved representation of fine-grained surface geometry and spatial correspondence. Extensive experiments across multiple downstream benchmarks demonstrate that DR-Point consistently outperforms existing self-supervised baselines on tasks such as 3D object classification, part segmentation, semantic segmentation, and shape completion. Our results highlight the effectiveness of multi-modal pre-training for comprehensive 3D processing and its potential to benefit a wider range of graphics-related tasks.

Log analysis is crucial for monitoring system health and diagnosing failures in complex systems. Recent advances in large language models (LLMs) offer new opportunities for automated log analysis, leveraging their reasoning capabilities to perform tasks such as anomaly detection and failure prediction. However, general-purpose LLMs struggle to formulate structured reasoning workflows that align with expert cognition and deliver precise details of reasoning steps. To address these challenges, we propose LogReasoner, a coarse-to-fine reasoning enhancement framework designed to enable LLMs to reason log analysis tasks like experts. LogReasoner consists of two stages: (1) coarse-grained enhancement of expert thinking, where high-level expert thoughts are constructed from collected troubleshooting flowcharts and existing tasks to enable LLMs to formulate structured reasoning workflows and (2) fine-grained enhancement of specific steps, where we first fine-tune the LLM with task-specific stepwise solutions to enhance the LLM for instantiated reasoning, then employ the preference learning to calibrate the LLM's reasoning details from its mistakes, further strengthen the LLM's analytical granularity and correctness. We evaluate LogReasoner on four distinct log analysis tasks using open-source LLMs such as Qwen-2.5 and Llama-3. Experimental results show that LogReasoner significantly outperforms existing LLMs, achieving state-of-the-art performance and demonstrating its effectiveness in enhancing the reasoning capabilities of LLMs for log analysis.