Smith42 (u/Smith42) - Readit News

Smith42 commented on AI and the Ship of Theseus lucumr.pocoo.org/2026/3/5... · Posted by u/pixelmonkey

Splinelinus · 10 days ago

I'm waiting for AGPL to become AIGPL: If you train a model with some or all of the licensed work, you agree that the weights of that model constitute a derivative work, and further for the weights, as well as any inference output produced as a result of those weights to be bound by the terms of the license. If you run a model with the licensed work in part or in full as input, you agree that any output from the model is bound by the terms of the license.

Smith42 · 10 days ago

So write it! Shouldn't be much extra to add to the AGPL licence?

Smith42 commented on Will we run out of data? Limits of LLM scaling based on human-generated data arxiv.org/abs/2211.04325... · Posted by u/Smith42

Smith42 · 2 years ago

We investigate the potential constraints on LLM scaling posed by the availability of public human-generated text data. We forecast the growing demand for training data based on current trends and estimate the total stock of public human text data. Our findings indicate that if current LLM development trends continue, models will be trained on datasets roughly equal in size to the available stock of public human text data between 2026 and 2032, or slightly earlier if models are overtrained. We explore how progress in language modeling can continue when human-generated text datasets cannot be scaled any further. We argue that synthetic data generation, transfer learning from data-rich domains, and data efficiency improvements might support further progress.

Smith42 commented on Astronomy Generates Mountains of Data. That's Perfect for AI – Universe Today universetoday.com/167153/... · Posted by u/Smith42

h2odragon · 2 years ago

Last I looked, most astronomy consisted of isolated data points with large amounts of extrapolation between them. This is not "data", no matter how often they insist that it is.

Smith42 · 2 years ago

That really isn't the case, and I am not sure how you could arrive at that unsubstantiated conclusion.

Smith42 commented on AstroPT: Scaling Large Observation Models for Astronomy arxiv.org/abs/2405.14930... · Posted by u/Smith42

Smith42 · 2 years ago

Abstract:

This work presents AstroPT, an autoregressive pretrained transformer developed with astronomical use-cases in mind. The AstroPT models presented here have been pretrained on 8.6 million 512 × 512 pixel grz-band galaxy postage stamp observations from the DESI Legacy Survey DR8. We train a selection of foundation models of increasing size from 1 million to 2.1 billion parameters, and find that AstroPT follows a similar saturating log-log scaling law to textual models. We also find that the models' performances on downstream tasks as measured by linear probing improves with model size up to the model parameter saturation point. We believe that collaborative community development paves the best route towards realising an open source `Large Observation Model' -- a model trained on data taken from the observational sciences at the scale seen in natural language processing. To this end, we release the source code, weights, and dataset for AstroPT under the MIT license, and invite potential collaborators to join us in collectively building and researching these models.