Juncheol Ahn

As large language models (LLMs) pursue higher accuracy, their model sizes have surged, substantially increasing GPU memory consumption. Prior work mitigates this issue by distributing the memory burden across multiple GPUs. However, on clusters interconnected via Ethernet, the resulting computational intensity is insufficient to hide the significant network latency. Achieving a favorable compute-to-communication ratio is further constrained by the memory required to cache the massive activations generated during the forward pass. PyAO, proposed in this paper, effectively offloads activations, selects offloading strategies based on their offloading efficiency, and minimizes data-movement bottlenecks, thereby enabling larger micro-batch sizes. In Ethernet-interconnected cluster environments, experiments on popular models—including OPT-1.3B, GPT-0.8B, and Llama-1.2B—demonstrate that PyAO reduces peak GPU memory by up to 1.94× at the same micro-batch size, enables up to 2.5× larger batch sizes, and accelerates training by up to 3.63× relative to the baseline.

Large language models in IoT–edge–cloud settings face bursty, heterogeneous requests that make pipeline-parallel inference prone to micro-batch imbalance and communication stalls, causing GPU idle time and SLO violations. We propose a runtime-adaptive scheduler that jointly tunes token budgets and micro-batch counts to balance prefill/decode workloads and minimize pipeline bubbles under changing compute and network conditions. On a four-node pipeline-parallel cluster across Llama-2-13b and Qwen2.5-14b at 100/1000 Mbps, our method outperforms vLLM and SGLang, reducing GPU idle time by up to 55% and improving throughput by up to 1.61 × while improving TTFT/ITL SLO satisfaction. These results show that dynamic scheduling is essential for scalable, latency-stable LLM inference in IoT–edge–cloud environments.

Large language models (LLMs) are increasingly integrated into IoT–edge–cloud systems, where real-time analytics and natural-language interaction demand both high throughput and stable latency. However, IoT workloads are inherently bursty and heterogeneous: prompt and generation lengths vary widely, and prefill- and decode-heavy requests coexist. When served via pipeline-parallel LLM inference, these characteristics amplify micro-batch imbalance and communication stalls, leading to substantial GPU idle time and degraded TTFT/ITL service-level objectives (SLOs). We propose a runtime-adaptive scheduling framework that combines Dynamic Token-Budget Estimation with Dynamic Micro-Batch Scheduling. Unlike static token-budget settings—which act primarily as latency knobs—our approach dynamically adjusts token budgets to balance prefill/decode workloads across micro-batches, while selecting the optimal number of micro-batches to minimize pipeline bubbles under varying compute and network conditions. Implementing the framework on a four-node RTX 4070 cluster running pipeline-parallel Llama-2-13b-chat with vLLM, we evaluate both synthetic offline workloads and online Poisson-arrival workloads. The combined scheme reduces GPU idle time by up to 55% and improves throughput (completion time) by up to 1.61× compared with the baseline, while significantly increasing TTFT/ITL SLO satisfaction under bursty conditions. These results demonstrate that dynamic, workload-aware scheduling is essential for scalable and latency-stable LLM inference in IoT–edge–cloud environments.

The blockchain market has been experiencing rapid growth recently. Alongside this, Web3 services based on blockchain technology are expanding and gaining attention. These services can support not only encompass gaming and financial services but also leverage the numerous nodes existing in the network to distribute tasks, thereby supporting parallel computing. However, there is no way to directly access web3 services in the current network topology, which limits the expansion of services. Therefore, the current Web3 relies on centralized web servers as access points for services, resulting in the inevitable loss of benefits associated with decentralization, such as the shift in data sovereignty. In addition, the current Domain Name System(DNS) is managed by a single entity, ICANN. Such centralized management by a single entity includes a structural risk of potentially leading to monopolistic control by the institution in the future. To eliminate potential threats and enable direct access to web3, decentralized DNS can be considered. This approach can effectively mitigate the risk of monopolistic control by distributing authority through its structure. Additionally, It also ensures that the benefits of decentralization such as information ownership transfer are fully realized without the loss associated with the use of centralized access points. Therefore, this paper examines the existing implementations of decentralized DNS and proposes various interface designs to enable users to use these systems in a universal manner across different scenarios. After implementing these interfaces, we discuss their characteristics and the situations in which they are most effective. Additionally, we verified that accessing services through the implemented interfaces shows no significant time difference compared to accessing web2 via traditional DNS. Through these proposals and discussions, we aim to propose general criteria for interfaces that can be used with decentralized DNS.