Tackling the Unlimited Staleness in Federated Learning with Intertwined Data and Device Heterogeneities

Kavli Affiliate: Wei Gao | First 5 Authors: Haoming Wang, Wei Gao, , , | Summary: The efficiency of Federated Learning (FL) is often affected by both data and device heterogeneities. Data heterogeneity is defined as the heterogeneity of data distributions on different clients. Device heterogeneity is defined as the clients’ variant latencies in uploading […]


Continue.. Tackling the Unlimited Staleness in Federated Learning with Intertwined Data and Device Heterogeneities

Tackling the Unlimited Staleness in Federated Learning with Intertwined Data and Device Heterogeneities

Kavli Affiliate: Wei Gao | First 5 Authors: Haoming Wang, Wei Gao, , , | Summary: The efficiency of Federated Learning (FL) is often affected by both data and device heterogeneities. Data heterogeneity is defined as the heterogeneity of data distributions on different clients. Device heterogeneity is defined as the clients’ variant latencies in uploading […]


Continue.. Tackling the Unlimited Staleness in Federated Learning with Intertwined Data and Device Heterogeneities

Tackling the Unlimited Staleness in Federated Learning with Intertwined Data and Device Heterogeneities

Kavli Affiliate: Wei Gao | First 5 Authors: Haoming Wang, Wei Gao, , , | Summary: Federated Learning (FL) can be affected by data and device heterogeneities, caused by clients’ different local data distributions and latencies in uploading model updates (i.e., staleness). Traditional schemes consider these heterogeneities as two separate and independent aspects, but this […]


Continue.. Tackling the Unlimited Staleness in Federated Learning with Intertwined Data and Device Heterogeneities

Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Staleness

Kavli Affiliate: Wei Gao | First 5 Authors: Haoming Wang, Wei Gao, , , | Summary: Federated Learning (FL) can be affected by data and device heterogeneities, caused by clients’ different local data distributions and latencies in uploading model updates (i.e., staleness). Traditional schemes consider these heterogeneities as two separate and independent aspects, but this […]


Continue.. Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Staleness

Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation

Kavli Affiliate: Wei Gao | First 5 Authors: Kai Huang, Hanyun Yin, Heng Huang, Wei Gao, | Summary: Fine-tuning is the most effective way of adapting pre-trained large language models (LLMs) to downstream applications. With the fast growth of LLM-enabled AI applications and democratization of open-souced LLMs, fine-tuning has become possible for non-expert individuals, but […]


Continue.. Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation

Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation

Kavli Affiliate: Wei Gao | First 5 Authors: Kai Huang, Hanyun Yin, Heng Huang, Wei Gao, | Summary: Fine-tuning is the most effective way of adapting pre-trained large language models (LLMs) to downstream applications. With the fast growth of LLM-enabled AI applications and democratization of open-souced LLMs, fine-tuning has become possible for non-expert individuals, but […]


Continue.. Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation

Deep Learning with Photonic Neural Cellular Automata

Kavli Affiliate: Alireza Marandi | First 5 Authors: Gordon H. Y. Li, Christian R. Leefmans, James Williams, Robert M. Gray, Midya Parto | Summary: Rapid advancements in deep learning over the past decade have fueled an insatiable demand for efficient and scalable hardware. Photonics offers a promising solution by leveraging the unique properties of light. […]


Continue.. Deep Learning with Photonic Neural Cellular Automata

Exact coherent structures in two-dimensional turbulence identified with convolutional autoencoders

Kavli Affiliate: Michael P. Brenner | First 5 Authors: Jacob Page, Joe Holey, Michael P. Brenner, Rich R. Kerswell, | Summary: Convolutional autoencoders are used to deconstruct the changing dynamics of two-dimensional Kolmogorov flow as $Re$ is increased from weakly chaotic flow at $Re=40$ to a chaotic state dominated by a domain-filling vortex pair at […]


Continue.. Exact coherent structures in two-dimensional turbulence identified with convolutional autoencoders

mixed attention auto encoder for multi-class industrial anomaly detection

Kavli Affiliate: Feng Wang | First 5 Authors: Jiangqi Liu, Feng Wang, , , | Summary: Most existing methods for unsupervised industrial anomaly detection train a separate model for each object category. This kind of approach can easily capture the category-specific feature distributions, but results in high storage cost and low training efficiency. In this […]


Continue.. mixed attention auto encoder for multi-class industrial anomaly detection

ICM-SHOX. Paper I: Methodology overview and discovery of a baryon–dark matter velocity decoupling in the MACS J0018.5+1626 merger

Kavli Affiliate: Sunil Golwala | First 5 Authors: Emily M. Silich, Elena Bellomi, Jack Sayers, John ZuHone, Urmila Chadayammuri | Summary: Galaxy cluster mergers are rich sources of information to test cluster astrophysics and cosmology. However, cluster mergers produce complex projected signals that are difficult to interpret physically from individual observational probes. Multi-probe constraints on […]


Continue.. ICM-SHOX. Paper I: Methodology overview and discovery of a baryon–dark matter velocity decoupling in the MACS J0018.5+1626 merger