TodayNewsResearchFieldsStacksMarketAnalytics
⌘K

Research

New papers ranked by attention, with their code, models and discussion linked. Pick a field on the left.

Research · AI field

Federated learning

Found by the topic model: federated learning, differential privacy.

Field page: people, tools, topics →
40papers this week+14.3% vs last week178 this month

Papers

Open problems

Rising research topics

Ask the research

Sample
ByteTrail · AI and developer-tools intelligence
NewsResearchStacksAPI
Fields · this week
All fields3276Multimodal496Agents485Frontier LLMs420Inference & serving244Learning verifiable rewards164Image segmentation161Graph learning1563D scene140Evals108Voice AI87Flow matching75Forecasting time75Classifier & causal discovery74Training & fine-tuning64Teacher policy distillation55Federated learning40All 32 fields
WeekMonthQuarterAll
NewestMost relevant

Papers

Open problems in Federated learning

  • Heterogeneous preferences and gradient conflicts in federated personalization1 paper raises it · 1 addresses it

    Diverse user objectives and scarce feedback make it hard to learn shared initializations that adapt well to individual preferences.

    • Collaborative Personalized Preference Alignment for LLMs under Data Deficiencyraises · addresses
  • Decentralized inference-time collaboration without global coordination1 paper raises it · 1 addresses it

    Exploiting distributed expertise in decentralized federated learning requires efficient, local-only routing and aggregation.

    • Learning to Refer: Inference-Time Collaboration in Decentralized Federated Learningraises · addresses
  • Communication-efficient federated learning under non-IID data and client drift1 paper raises it · 1 addresses it

    Bandwidth constraints and non-IID data cause high latency and client drift, degrading model performance.

    • Communication-Efficient Federated Domain Generalization in Edge Networks via Dynamic Prototype Regulationraises · addresses
  • Secure aggregation with adaptive user dropouts1 paper raises it · 1 addresses it

    Existing secure aggregation schemes cannot dynamically adjust to actual dropout scenarios, limiting practicality.

    • Fundamental Limits of Information-Theoretic Secure Aggregation with User Dropouts in Federated Learningraises · addresses
  • Federated continual learning with heterogeneous device capacities1 paper raises it · 1 addresses it

    Training models of varying sizes on devices with drifting data streams is underexplored and challenging.

    • DROD: Dual-Ratio Ordered Dropout for Federated Continual Learning on Heterogeneous Devicesraises · addresses

Named by an AI from these papers' abstracts.

Rising research topics

Coming with more sources: more tagged papers
  • Sensing communication16+23.1%
  • Federated learning40+14.3%
  • Agents485−3.6%
  • Frontier LLMs420−10.6%
  • Forecasting time75−16.7%
  • Learning verifiable rewards164−17.6%
  • Inference & serving244−19.2%
  • Quantum machine learning23−20.7%

Papers

  • arXiv cs.LGOct 19

    Learning to Refer: Inference-Time Collaboration in Decentralized Federated Learning

    Yuheng Ma, Chendi Wang, Qiong Zhang

    In decentralized federated learning, heterogeneous clients may develop complementary expertise, yet each query is typically evaluated only by the client where it arrives. To exploit this distributed expertise without global coordination, we formulate inference-time collaboration as budgeted local expert referral, a graph-constrained routing process in which a query can move only between neighboring clients and only visited clients contribute to its final prediction. EdgeDRM implements this framework by learning a density-based routing potential for each client: differences between neighboring potentials provide the comparisons used to select successive referrals, while local task models generate predictions that are aggregated along the resulting route. Both training and inference are decentralized, using only neighbor-local information without a central router or graph-wide client scoring. We establish estimation guarantees for the learned comparisons and characterize how local improvement and graph structure govern prior-weighted density gains from referral. Experiments on benchmark and real-world medical data show that EdgeDRM outperforms graph-matched decentralized baselines…

    PDF
  • arXiv cs.LGOct 19

    Communication-Efficient Federated Domain Generalization in Edge Networks via Dynamic Prototype Regulation

    Yuankun Xia, Hui Wang, Ling Yuan, Ziyu Hong, Di Yin +1

    Federated learning (FL) is a distributed paradigm for Internet of Things (IoT) edge networks. However, deploying Vision-Language Models (VLMs) over wireless edge networks presents distinct challenges: large model parameters increase uplink transmission latency over bandwidth-constrained fading channels, and non-independent and identically distributed (Non-IID) data exacerbates client drift. Existing feature alignment solutions often introduce auxiliary computations that consume time slots allocated for wireless parameter transmission, increasing the likelihood of straggler dropouts under system delay constraints. To address these issues, we propose DPGR-GCAM, a communication-efficient federated domain generalization framework. By employing a dual-parameter-efficient fine-tuning (PEFT) architecture, we design a channel-aware tiered transmission mechanism that adaptively truncates payloads to maximize node survival rates under fading conditions. To mitigate client drift, we introduce the Gradient-Conflict Aware Modulation (GCAM) mechanism. GCAM operates within the low-dimensional feature manifold to dynamically resolve multi-objective optimization conflicts with minimal edge…

    PDF
  • arXiv cs.ITOct 19

    Fundamental Limits of Information-Theoretic Secure Aggregation with User Dropouts in Federated Learning

    Jinbao Zhu, Kai Wan, Giuseppe Caire

    Secure aggregation enables a server to compute the sum of users' gradients or model updates without revealing individual values. With user dropouts, correlated random keys protect local gradients while allowing recovery of the desired gradient sum. We consider three phases: key distribution, masked-gradient upload, and mask elimination. Users establish correlated keys, upload masked gradients, and provide additional messages to eliminate residual aggregate masks. Existing information-theoretic schemes typically design mask-elimination communication for a prescribed dropout threshold and evaluate its cost under worst-case dropouts across the masked-gradient upload and mask-elimination phases. Such designs cannot adjust communication to actual dropout scenarios. Moreover, the randomness and communication costs of establishing correlated keys remain largely unexplored. To address these limitations, we formulate an adaptive secure aggregation framework under the T-colluding security constraint and construct a single scheme that supports all feasible dropout scenarios by adapting its communication to the actual number of dropouts. We establish converse bounds on four resources: key…

    PDF
  • arXiv cs.LGOct 18

    DROD: Dual-Ratio Ordered Dropout for Federated Continual Learning on Heterogeneous Devices

    Berk Sezer, Özgür Soysal

    Federated learning for capacity-heterogeneous clients and federated continual learning are each well studied on their own, but their intersection -- clients holding differently sized models while every client's data stream also drifts over time -- remains underexplored. We study this setting directly, training a ResNet-18 fleet from scratch on Split-CIFAR-100 (10 tasks, class-incremental), and introduce DROD (Dual-Ratio Ordered Dropout), which builds on FjORD's Ordered Dropout by using two independent reduction axes to extract sub-models: a shortcut ratio and a branch ratio. On a shared parameter/FLOP budget ladder, DROD achieves the highest mean accuracy among the heterogeneous-model federated learning strategies we compare. Its advantage over the two strongest baseline geometries persists across every replay-buffer size we test and under ER, DER++, and BiC; under EWC, all three geometries collapse alike.

    PDF
  • arXiv cs.LGOct 18

    Shallow Probes Are Sufficient for Near-Oracle Personalization Depth Allocation in Federated Learning

    Beyazit Bestami Yuksel, Emrah Dikbiyik, Ayse Yilmazer Metin

    Rather than proposing a new federated optimizer, we study whether a small client-specific measurement budget can support cost-aware personalization-depth decisions under a utility that trades Macro-F1 against deployment cost. Exhaustive evaluation of six depths defines an offline oracle that is not deployable. We audit the Probe Sufficiency Hypothesis: three shallow probes (P0--P2) may contain most decision-relevant information for approaching that oracle. FedPBA is a secondary learned instantiation for deep-action extrapolation. On CIFAR-10/100 and patient-disjoint PTB-XL (45 sessions; 900 clients; 2700 evaluations; GroupKFold), training-free greedy probing achieves regret 0.0071 and 98.3 percent oracle recovery, beating metadata-only policies and FedPBA (0.0110 / 97.3 percent). Aggregate shallow-best versus oracle correlation is 0.9818 but falls to 0.578 on deep-oracle rows; PTB-XL deep-oracle cases remain a high-regret boundary without a validated escalation gate.

    PDF
  • arXiv cs.LGOct 18

    Indistinguishable Agents: Training Agent Privacy as Twin-World Indistinguishability

    Jiaming He, Guanyu Hou, Yi Yu, Xudong Jiang

    LLM agents search, shop and send messages for their users, and every action is seen by some party. Existing privacy training rewards agents for not mentioning protected items, yet an observer can infer a private fact from what the agent does. Since an observer learns a fact only if its view changes with the fact, we propose \methodfull (\method), which trains on twin worlds that differ only in the private fact. A frozen distinguisher scores each observer's view by its log-likelihood ratio for the true world, and paired GRPO trains the agent on task success minus this ratio. We prove that this objective controls the Bayes success of every observer on every channel. On shopping and messaging tasks \method removes the leakage that disclosure judges and query-level rewards miss, while the agent still completes its tasks.

    PDF
  • arXiv cs.DSOct 18

    No Escape from the Quadratic Barrier: Adaptive Attacks on Linear $\ell_2$ Sketches

    Edith Cohen, Uri Stemmer

    Linear sketches estimate properties of high-dimensional inputs from a fixed low-dimensional projection. At constant accuracy, randomized Euclidean-norm sketches, notably the Johnson-Lindenstrauss transform, support exponentially many nonadaptive queries in sketch dimension $k$. For adaptive black-box queries, known privacy-based defenses guarantee $\widetildeΩ(k^2)$ and it remained open whether stronger guarantees were possible. Prior attacks used more queries, restricted the responder, or provided weaker failure guarantees. We show that the quadratic guarantee is optimal up to logarithmic factors, even with a computationally unbounded responder that knows the matrix, attack algorithm, history, and query distributions and may change its randomized rule after every query. For every fixed hidden $A\in\mathbb{R}^{k\times n}$ with $n>40k$ and target accuracy $α\in (0,1]$, our black-box attack uses $O(k+α^2k^2\log(2+α^2k))$ queries in $\widetilde{O}(1+α^2k)$ adaptive rounds to force, with constant probability, a round in which a constant fraction of answers have $Ω(α)$ relative error. Our results clarify the limits of defending a fixed compressive linear representation by adapting the…

    PDF
  • arXiv cs.CVOct 17

    PCHMR: Empowering and Benchmarking Human Mesh Recovery in Privacy-Constrained Real-World Settings

    Zeyu Cao, Qingxuan Wu, Zhiyang Dou, Rui Xu, Yuan Liu +4

    Fine-tuning directly on user-side data is an effective way to improve the performance of widely adopted data-driven human mesh recovery (HMR) models. However, doing so for existing HMR methods often assumes aggregating user-side data from real-world HMR-deployed devices to a central training server, posing a significant privacy risk as sensitive human images are transmitted. How can human mesh recovery be evaluated and improved in privacy-constrained real-world settings? We study this question through a benchmark in which state-of-the-art HMR models are trained with federated learning and a secure-aggregation variant, neither of which centralizes raw images, and evaluated across a wide range of client-scale and data-heterogeneity settings. We document that common HMR training pipelines are built around centralized data access and quantify how representative HMR backbones behave when that assumption is removed. Furthermore, to study the data bottleneck of privacy-constrained HMR training, we propose DePoser, a local annotation and fine-tuning pipeline enhanced with depth foundation models, with which collaboratively trained HMR models can be locally tailored to the end user's…

    PDF
  • arXiv cs.LGOct 17

    How Does Muon Work in Private Deep Learning? Scalable Differentially Private Training via Orthogonalization

    Alexander Gaponov, Grigory Malinovsky, Egor Shulgin, Peter Richtarik

    Training deep neural networks requires optimizers that balance computational efficiency with formal data privacy. While orthogonalized update rules, such as the recently introduced Muon optimizer, have shown strong empirical performance and scalability, their behavior under Differential Privacy (DP) is not yet understood. In this paper, we propose DP-Muon, a differentially private variant of the Muon optimizer. By analyzing Muon's orthogonalized updates through the lens of a Linear Minimization Oracle (LMO), we show that the leading effect of per-example clipping is a positive rescaling of the gradient, to which the LMO is blind, so that only a small directional residual controlled by the centered gradient variance enters the optimization error. This yields an $\mathcal O(K^{-1/4})$ convergence rate, up to logarithmic factors, to a clipping-dependent neighborhood at a fixed noise scale and, with noise calibrated to a fixed $(\varepsilon,δ)$ budget, the standard $(\sqrt d/(n\varepsilon))^{1/2}$ privacy--utility bound of DP-SGD, without assuming bounded gradients. Furthermore, we investigate the hyperparameter landscape of DP-Muon and document how the optimal learning rate,…

    PDF
  • arXiv cs.LGOct 17

    A Multi-Step Inertial ADMM Algorithm for Federated Learning

    Hao Li, Qingsong Wang, Xihong Yan, Chengcheng Yan, Deren Han

    Federated learning has recently emerged as an important paradigm for large-scale distributed model training, which can be naturally formulated as a class of distributed learning problems. The alternating direction method of multipliers (ADMM) is a widely used approach for solving such problems. However, its practical performance is often hindered by significant communication overhead and system heterogeneity. In this paper, we propose a multi-step inertial ADMM algorithm, called MiFedADMM, for federated learning. The proposed method leverages multiple historical iterates on the server side to construct a higher-order extrapolation after each aggregation to accelerate the convergence behavior. Under mild assumptions, we establish convergence results for the single-step variant of the proposed method and derive an O(1/K) convergence rate. Numerical experiments on both synthetic and real-world datasets demonstrate the effectiveness and improved performance of the proposed method compared with existing approaches.

    PDF
  • arXiv stat.MLOct 17

    Private Evaluation of High-Dimensional Functions

    Mohammad Afzali, Hassan Ashtiani, Adam Smith

    We study differentially private black-box evaluation of arbitrary high-dimensional functions. Given a dataset $S$ and query access to a function $f$---whose sensitivity may be unknown or unbounded---the goal is to privately output an approximation to $f(S)$. The black-box evaluator can only query the value of $f$ over a down neighborhood of $S$ (i.e., those subsets of $S$ that are obtained by removing a bounded number of elements from $S$). We design generic black-box evaluators for both pure and approximate differential privacy through a reduction to high-dimensional private quasi-concave optimization (QCO). Our black-box evaluator enjoys a geometric notion of utility. Consider the vectors obtained by applying $f$ on sets in a down neighborhood of $S$. With high probability, the evaluator's output is guaranteed to be in the convex hull of these vectors. Our work generalizes the result of Linder et al. [LRSS25] for real-valued function evaluation to high dimensions. It also recovers the private center-point guarantees of Beimel et al. [BMNS19], with an explicit additive approximation to maximum Tukey depth. We also apply our evaluators to user-level quasi-concave and convex…

    PDF
  • arXiv cs.AIOct 17

    Train Locally, Learn Together: Distributed Multi-Agent Training via ADMM

    Yiwei Chen, Changsheng Wang, Yicheng Lang, Qing Qu, Alfred O. Hero +1

    Multi-agent systems (MAS) can outperform individual language-model agents by combining specialized roles, diverse reasoning processes, and collaborative decision-making. However, collaborative MAS training is typically centralized, requiring a common trainer to aggregate role-local trajectories and learning signals and maintain model and optimization states. This motivates a distributed multi-agent learning paradigm in which agents improve system-level capabilities through role-local learning and decentralized communication, paving the way toward scalable, resilient, and self-organizing MAS. To this end, we introduce, to the best of our knowledge, the first optimization-grounded framework for distributed MAS training based on consensus alternating direction method of multipliers (ADMM). We formulate collaborative MAS learning as a consensus-constrained finite-sum optimization problem, naturally lending itself to distributed optimization via ADMM. The proposed approach alternates role-local learning with inter-agent consensus using distributed communication protocols such as ring all-reduce, enabling joint optimization while keeping each agent's data and training state local. We…

    PDF
  • arXiv stat.MLOct 16

    Task-aware Kernel Stein certificates for Rényi differential privacy

    Leoni Carla Wirth, Tara Trauthwein, Gesine Reinert

    Rènyi differential privacy, based on Rènyi divergence, has become a notion of choice for privacy guarantees. However, outside the Gaussian setting, Rènyi divergence can be difficult to compute. In particular, Rènyi differential privacy guarantees that exploit task-specific information, as in Rènyi Pufferfish privacy, can be challenging. Here we use kernel Stein discrepancies based on escort distributions to provide certificates for Rènyi differential privacy. By designing the kernel to focus on the relevant privacy directions, we also derive task-aware privacy certificates. We provide exact identities for Gaussian and Laplace mechanisms and bounds for exponential mechanisms, with separate task-aware guarantees. A real data set example illustrates how the protected comparison set and the released subspace jointly determine the resulting certificates.

    PDF
  • arXiv cs.CVOct 16

    FedRAPP: Federated Test-Time Prompt Tuning for Vision-Language Models with Reliability-Aware Prompt Personalization

    Syed Md. Ahnaf Hasan, Md Akil Raihan Iftee, Sajib Mistry, Amin Ahsan Ali, AKM Mahbubur Rahman

    Federated prompt learning enables vision-language models like CLIP to collaboratively adapt lightweight soft prompts across distributed clients without sharing raw data. However, existing federated prompt learning methods are designed primarily for supervised settings with labeled client data and assumed relatively stable training distributions. At test time, source-free federated prompt tuning remains largely unexplored. Clients may receive unlabeled target samples with heterogeneous, often non-IID distributions that differ substantially across clients and from the source training distribution, causing locally adapted prompts to drift in different directions and reducing the effectiveness of federated aggregation. In this work, we explore Source-Free Federated Test-Time Prompt Tuning (SF-FTTPT), where distributed clients collaboratively adapt soft prompts on unlabeled test streams without access to source data. Our study reveals that conventional federated aggregation methods struggle under heterogeneous client distributions, as locally adapted prompts become increasingly specialized and difficult to combine effectively. To address this challenge, we propose FedRAPP, a…

    PDF
  • arXiv cs.CROct 16

    TUPLE-Bench: Evaluating and Mitigating Privacy-Utility Trade-offs in Typed Decision Models

    Yiqu Yang, Qiaoyuan Zheng, Yang Tian, Yu Fan, Elliott Ash

    Typed decision models often return more than a selected category. They also expose probabilities for each candidate and, sometimes, a separate confidence score. These fields support downstream decisions but may reveal information beyond a recipient's authorization. We introduce TUPLE-Bench (Typed decision models' Utility and Privacy Leakage Evaluation Benchmark), a benchmark of paired synthetic cases in which recipients may learn a coarse operational category but not its fine-grained cause. Across nine models, instructions to withhold the cause fail to protect it: 91.88%-99.93% of responses reveal the cause through direct selection or residual scores. Even when causes are excluded from the candidate set, full outputs enable 62.85%-75.35% recovery on held-out cases, compared with 51.04%-57.64% using only the selected category (50% chance). Full outputs also reduce mean downstream cost in all 27 model-task comparisons. However, confidence fields increase leakage without consistently reducing cost. We further introduce workflow-constrained privacy quantization (WCPQ), which reduces secret recovery while preserving the behavior and cost of specified downstream workflows. Our findings…

    PDF
  • arXiv cs.LGOct 16

    Bridging Compressed Communication and Local Steps via Spectral Method in Federated Learning

    Dmitriy Bystrov, George Chirkov, Daniil Medyakov, Dmitry Bylinkin, Aleksandr Beznosikov

    Federated Learning (FL) enables collaborative training of complex models across large-scale networks of edge devices while keeping data localized. However, data heterogeneity and high communication overhead remain fundamental bottlenecks. Recently, the spectral optimizer Muon has demonstrated remarkable advantages over Adam-like methods in centralized training. While recent efforts have attempted to transition Muon to FL, naive integrations into traditional architectures fail short. Crucially, existing approaches do not support the the communication compression and local steps --- the two foundational techniques of efficient FL. To address this limitation, we introduce a novel spectral FL method that utilizes both local updates and compression. We establish rigorous convergence guarantees under mild assumptions, notably eliminating the restrictive bounded heterogeneity condition. Empirical evaluation on standard image classification benchmarks validates the efficiency and robustness of our approach.

    PDF
  • arXiv cs.LGOct 16

    Client-Aware Security Adaptive Early Exit Vision Transformers for Heterogeneous Federated Learning

    Sahar Refaei Ghalibafan, Samira Nazari, Christian Herglotz, Maksim Jenihhin, Mahdi Taheri

    Federated learning (FL) trains a shared model across data-owning clients without centralizing raw data, but client data difficulty and compute budgets can differ. We study local early-exit Vision Transformer (ViT) inference, where an intermediate classifier may return a prediction before the final Transformer stage. The proposed method replaces one global acceptance threshold with a client-specific base threshold from label entropy and shallow-exit confidence, then adjusts each input using predictive entropy, confidence, top-two class margin, and exit depth. Aggregation uses only clients that updated a parameter. Security-aware routing reaches 88.70% CIFAR-10 accuracy with 32.0% depth saving and 73.7% ImageNet-1K top-1 accuracy with 30\% saving. On matched six-client non-IID CIFAR-100, full-depth accuracy is 72.28% versus 68.20--71.80% for four heterogeneous-FL baselines; robustness-aware routing reaches 70.01% with 32.55% saving.

    PDF
  • arXiv cs.LGOct 16

    TALLOP: Joint Optimization of Local Training and Communication Compression for Time-Constrained Federated Learning

    Honoka Anada, Yafei Ou, Shinya Takamaeda-Yamazaki

    Minimizing the wall-clock time required to reach a target accuracy is crucial in federated learning (FL), where many applications require frequent model updates as data distributions evolve over time. Nevertheless, in heterogeneous environments, slow clients with limited computational and communication capabilities can delay global synchronization, forcing faster clients to wait and increasing the overall training time. To mitigate this straggler effect, recent studies have introduced per-round time constraints and adapted client workloads within the available time. However, these approaches primarily focus on local computation, whereas communication can become a major bottleneck in communication-limited environments, motivating the joint optimization of training workload and communication compression. Although previous studies have explored such joint optimization, determining an effective training--communication balance under a fixed per-round time budget remains challenging, as the resulting model performance depends on both training progress and compression-induced degradation. We therefore propose TALLOP, which jointly optimizes the local training workload and communication…

    PDF
  • arXiv cs.LGOct 16

    Towards Hyperparameter Transfer for Differentially Private Optimization

    Tianze Wang, Zhiqi Bu, Linjun Zhang

    Scaling laws and hyperparameter tuning remain central challenges in training large-scale models, particularly under differential privacy (DP), where optimization is further complicated by noise injection and per-sample gradient clipping. While Maximal Update Parametrization ($μ$P) allows for zero-shot hyperparameter transfer in standard training, we show that it fails under DP constraints due to the distinct spectral properties of high-dimensional noise versus low-rank gradients. In this work, we introduce DP-$μ$P, a theoretically grounded framework that extends $μ$P to the privacy-preserving setting. By analyzing the interaction between gradient clipping, noise injection, and feature learning, we derive a Signal-to-Noise Ratio (SNR) condition that governs layer-wise learning rate scaling. Combining this with our proposed spectral scaling formulation, we provide specific scaling rules for DP-SGD, DP-Adam, and DP-Muon optimizers. Empirically, we demonstrate that DP-$μ$P enables effective zero-shot transfer of learning rates across model widths on diverse architectures (MLP, ViT, GPT-2), matching or exceeding the performance of individually tuned baselines while significantly…

    PDF
  • arXiv cs.CROct 16

    Veil-Then-Reveal: Protecting PII Privacy via Reversible Pseudo-Anonymization in Zero-Trust User--LLM Interaction

    Mohamed Shaaban, Mohamed Elmahallawy, Thomas E. Carroll

    Large language models (LLMs) increasingly process sensitive documents and personal information, yet sending prompts containing personally identifiable information (PII) to remote providers exposes data subjects and organizations to risks from logging, retention, model training, and unintended disclosure. Existing anonymization methods often mask or remove PII, while conventional encryption eliminates the lexical and structural cues needed for effective LLM processing. We present Veil-then-Reveal (VtR), a user-side PII pseudo-anonymization framework for privacy-preserving LLM interaction under a zero-trust threat model. VtR treats remote LLM providers as untrusted and sends only pseudo-anonymized prompts, keeping plaintext PII on the user. VtR first detects PII and replaces it with realistic, same-type, in-vocabulary surrogates before prompts leave the user's device, preserving their linguistic and structural properties. It combines format-preserving encryption (FPE) for structured PII, such as phone and credit-card numbers, with a keyed-hash dictionary and secure vault for semantic PII, such as names, locations, and organizations. Session-specific mappings provide cross-session…

    PDF
Load more