Skip to main content

Astronomy Generates Mountains of Data. That’s Perfect for AI

Consumer-grade AI is finding its way into people’s daily lives with its ability to generate text and images and automate tasks. But astronomers need much more powerful, specialized AI. The vast amounts of observational data generated by modern telescopes and observatories defies astronomers’ efforts to extract all of its meaning.

A team of scientists is developing a new AI for astronomical data called AstroPT. They’ve presented it in a new paper titled “AstroPT: Scaling Large Observation Models for Astronomy.” The paper is available at arxiv.org, and the lead author is Michael J. Smith, a data scientist and astronomer from Aspia Space.

Astronomers are facing a growing deluge of data, which will expand enormously when the Vera Rubin Observatory (VRO) comes online in 2025. The VRO has the world’s largest camera, and each of its images could fill 1500 large-screen TVs. During its ten-year mission, the VRO will generate about 0.5 exabytes of data, which is about 50,000 times more data than is contained in the USA’s Library of Congress.

The VRO's need for multiple sites to handle all of its data is a testament to the enormous volume of data it will generate. Without effective AI, that data will be stuck in a bottleneck. Image Credit: NOIRLab.
The VRO’s need for multiple sites to handle all of its data is a testament to the enormous volume of data it will generate. Without effective AI, that data will be stuck in a bottleneck. Image Credit: NOIRLab.

Other telescopes with enormous mirrors are also approaching first light. The Giant Magellan Telescope, the Thirty Meter Telescope, and the European Extremely Large Telescope combined will generate an overwhelming amount of data.

Having data that can’t be processed is the same as not having the data at all. It’s basically inert and has no meaning until it’s processed somehow. “When you have too much data, and you don’t have the technology to process it, it’s like having no data,” said Cecilia Garraffo, a computational astrophysicist at the Harvard-Smithsonian Center for Astrophysics.

This is where AstroPT comes in.

AstroPT stands for Astro Pretrained Transformer, where a transformer is a particular type of AI. Transformers can change or transform an input sequence into an output sequence. AI needs to be trained, and AstroPT has been trained on 8.6 million 512 x 512-pixel images from the DESI Legacy Survey Data Release 8. DESI is the Dark Energy Spectroscopic Instrument. DESI studies the effect of Dark Energy by capturing the optical spectra from tens of millions of galaxies and quasars.

AstroPT and similar AI deal with ‘tokens.’ Tokens are visual elements in a larger image that contain meaning. By breaking images down into tokens, an AI can understand the larger meaning of an image. AstroPT can transform individual tokens into coherent output.

AstroPT has been trained on visual tokens. The idea is to teach the AI to predict the next token. The more thoroughly it’s been trained to do that, the better it will perform.

“We demonstrated that simple generative autoregressive models can learn scientifically useful information when pre-trained on the surrogate task of predicting the next 16 × 16 pixel patch in a sequence of galaxy image patches,” the authors write. In this scheme, each image patch is a token.

This image illustrates how the authors trained AstroPT to predict the next token in a 'spiralised' sequence of galaxy image patches. It shows the token feed order. "As the galaxies are in the centre of each postage stamp, this set up allows us to seamlessly pretrain and run inference on differently sized galaxy postage stamps," the authors explain. Image Credit: Smith et al. 2024.
This image illustrates how the authors trained AstroPT to predict the next token in a ‘spiralised’ sequence of galaxy image patches. It shows the token feed order. “As the galaxies are in the centre of each postage stamp, this set up allows us to seamlessly pretrain and run inference on differently sized galaxy postage stamps,” the authors explain. Image Credit: Smith et al. 2024.

One of the obstacles to training AI like AstroPT concerns what AI scientists call the ‘token crisis.’ To be effective, AI needs to be trained on a large number of quality tokens. In a 2023 paper, a separate team of researchers explained that a lack of tokens can limit the effectiveness of some AI, such as LLMs or Large Language Models. “State-of-the-art LLMs require vast amounts of internet-scale text data for pre-training,” the wrote. “Unfortunately, … the growth rate of high-quality text data on the internet is much
slower than the growth rate of data required by LLMs.”

AstroPT faces the same problem: a dearth of quality tokens to train on. Like other AI, it uses LOMs or Large Observation Models. The team says their results so far suggest that AstroPT can solve the token crisis by using data from observations. “This is a promising result that suggests that data taken from the observational sciences would complement data from other domains when used to pre-train a single multimodal LOM, and so points towards the use of observational data as one solution to the ‘token crisis’.”

AI developers are eager to find solutions to the token crisis and other AI challenges.

Without better AI, a data processing bottleneck will prevent astronomers and astrophysicists from making discoveries from the vast quantities of data that will soon arrive. Can AstroPT help?

The authors are hoping that it can, but it needs much more development. They say they’re open to collaborating with others to strengthen AstroPT. To aid that, they followed “current leading community models” as closely as possible. They call it an “open to all project.”

“We took these decisions in the belief that collaborative community development paves the fastest route towards realising an open source web-scale large observation model,” they write.

“We warmly invite potential collaborators to join us,” they conclude.

It’ll be interesting to see how AI developers will keep up with the vast amount of astronomical data coming our way.

The post Astronomy Generates Mountains of Data. That’s Perfect for AI appeared first on Universe Today.



from Universe Today https://ift.tt/2qcJzoY
via IFTTT
Labels:

Comments

Post a Comment

Popular posts from this blog

More Data and Machine Learning has Kicked SETI Into High Gear

For over sixty years, astronomers and astrophysicists have been engaged in the Search for Extraterrestrial Intelligence (SETI). This consists of listening to other star systems for signs of technological activity (or “technosignatures), such as radio transmissions. This first attempt was in 1960, known as Project Ozma, where famed SETI researcher Dr. Frank Drake (father of the Drake Equation) and his colleagues used the Robert C. Byrd Green Bank Telescope in West Virginia to conduct a radio survey of Tau Ceti and Epsilon Eridani. Since then, the vast majority of SETI surveys have similarly looked for narrowband radio signals since they are very good at propagating through interstellar space. However, the biggest challenge has always been how to filter out radio transmissions on Earth – aka. radio frequency interference (RFI). In a recent study, an international team led by the Dunlap Institute for Astronomy and Astrophysics (DIAA) applied a new deep-learning algorithm to data collecte...
Post a Comment
Read more

The First Close-Up Picture of Star Outside the Milky Way

Like a performer preparing for their big finale, a distant star is shedding its outer layers and preparing to explode as a supernova. Astronomers have been observing the huge star, named WOH G64, since its discovery in the 1970s. It’s one of the largest known stars, and also one of the most luminous and massive red supergiants (RSGs). The star is surrounded by an envelope of expelled star-stuff, which could indicate it’s getting ready to explode. WOH G64 isn’t in the Milky Way; it’s in the Large Magellanic Cloud (LMC), the Milky Way’s largest satellite galaxy. Getting these detailed image is quite a feat for the ESO’s Very Large Telescope Interferometer. It’s also quite an accomplishment for the team of scientists behind the image. They’ve published their images and the results of their observations of the star in the journal Astronomy and Astrophysics. Their research is titled “ Imaging the innermost circumstellar environment of the red supergiant WOH G64 in the Large Magellanic ...
Post a Comment
Read more

Planetary Surfaces: Why study them? Can they help us find life elsewhere?

Universe Today recently explored the importance of studying impact craters and what they can teach us about finding life beyond Earth. Impact craters are considered one of the many surface processes—others include volcanism, weathering, erosion, and plate tectonics—that shape surfaces on numerous planetary bodies, with all of them simultaneously occurring on Earth. Here, we will explore how and why planetary scientists study planetary surfaces, the challenges faced when studying other planetary surfaces, what planetary surfaces can teach us about finding life, and how upcoming students can pursue studying planetary surfaces, as well. So, why is it so important to study planetary surfaces throughout the solar system? “Planetary surfaces record the history of the Solar System, a history that’s almost entirely lost to us here on Earth,” Dr. Paul Byrne, who is an Associate Professor of Earth, Environmental, and Planetary Sciences at Washington University in St. Louis, tells  Uni...
Post a Comment
Read more