Kavli Affiliate: Salman Habib
| First 5 Authors: Nesar Ramachandra, Nesar Ramachandra, , ,
| Summary:
Pre-trained Large Language Models (LLMs) have revolutionized text processing,
yet adapting Transformer-based neural networks to non-textual scientific
modalities typically requires specialized architectures and extensive
computational resources. We demonstrate that LLaMA-3.1-8B can be efficiently
repurposed to predict galaxy redshifts from spectroscopic data through Low-Rank
Adaptation (LoRA), achieving competitive performance while preserving its
linguistic capabilities. Using only 16 GPU-hours and adapting 0.04% of model
parameters, our approach achieves a mean absolute error of 0.04 in redshift
prediction while retaining over 85% of performance on AstroBench and 89% on
general QA tasks from eval-harness. This minimal-effort adaptation–requiring
only simple standard fine-tuning APIs–lowers barriers to entry for domain
scientists and enables integrated agentic workflows where a single model
handles both spectroscopic data for quantitative analysis and natural language
for reasoning.
| Search Query: ArXiv Query: search_query=au:”Salman Habib”&id_list=&start=0&max_results=3