As ubiquitous as artificial intelligence (AI) has become in modern life — from boosting our understanding of the cosmos to surfacing entertaining videos on your phone — AI hasn’t yet found its way into orbit.
That is until 2 September, when an experimental satellite about the size of a cereal box was ejected from a rocket’s dispenser along with 45 other similarly small satellites.
The satellite, named PhiSat-1, is now soaring at over 27 500kph in sun-synchronous orbit about 530km overhead.
PhiSat-1 contains a new hyperspectral-thermal camera and onboard AI processing thanks to an Intel Movidius Myriad 2 Vision Processing Unit (VPU) — the same chip inside many smart cameras and even a $99 selfie drone here on Earth. PhiSat-1 is actually one of a pair of satellites on a mission to monitor polar ice and soil moisture, while also testing intersatellite communication systems in order to create a future network of federated satellites.
The first problem the Myriad 2 is helping to solve? How to handle the large amount of data generated by high-fidelity cameras like the one on PhiSat-1.
“The capability that sensors have to produce data increases by a factor of 100 every generation, while our capabilities to download data are increasing, but only by a factor of three, four, five per generation,” says Gianluca Furano, data systems and onboard computing lead at the European Space Agency, which led the collaborative effort behind PhiSat-1.
At the same time, about two-thirds of our planet’s surface is covered in clouds at any given time. That means a whole lot of useless images of clouds are typically captured, saved, sent over precious down-link bandwidth to Earth, saved again, reviewed by a scientist (or an algorithm) on a computer hours or days later — only to be deleted.
“And AI at the edge came to rescue us, the cavalry in the Western movie,” says Furano. The idea the team rallied around was to use onboard processing to identify and discard cloudy images — thus saving about 30% of bandwidth.
“Space is the ultimate edge,” says Aubrey Dunne, chief technology officer of Ubotica. The Irish startup built and tested PhiSat-1’s AI technology, working in close partnership with cosine, maker of the camera, in addition to the University of Pisa and Sinergise to develop the complete solution. “The Myriad was absolutely designed from the ground up to have an impressive compute capability but in a very low power envelope, and that really suits space applications.”
The Myriad 2, however, was not intended for orbit. Spacecraft computers typically use very specialized “radiation-hardened” chips that can be “up to two decades behind state-of-the-art commercial technology,” explains Dunne. And AI has not been on the menu.
Dunne and the Ubotica team performed “radiation characterization,” putting the Myriad chip through a series of tests to figure out how to handle any resulting errors or wear-and-tear.
ESA “had never tested a chip of this complexity for radiation,” says Furano. “We were doubtful we could test it properly … we had to write the handbook on how to perform a comprehensive test and characterization for this chip from scratch.”
The first test, 36 straight hours of radiation-beam blasting at CERN in late 2018, “was a very high pressure situation,” Dunne says. But that test and two follow-ups “luckily turned out well for us.” The Myriad 2 passed in off-the-shelf form, no modifications needed.
This low-power, high-performance computer vision chip was ready to venture beyond Earth’s atmosphere. But then came another challenge.
Typically, AI algorithms are built, or “trained,” using large quantities of data to “learn” — in this case, what’s a cloud and not a cloud. But given the camera was so new, “we didn’t have any data,” says Furano. “We had to train our application on synthetic data extracted from existing missions.”
All this system and software integration and testing, with involvement of a half-dozen different organisations across Europe, took four months to complete. “We were very proud to be able to be so quick and so efficiently flexible, to put everything on board in such a short time,” says Max Pastena, PhiSat officer at ESA. As far as spacecraft development goes, the timeline “is a miracle,” adds Furano.
“Intel has given us background support on the Myriad device when we’ve needed it, to enable PhiSat-1’s AI using our CVAI Technology,” says Dunne. “That’s very much appreciated.”
Unfortunately, a series of unrelated events — delays with the rocket, the coronavirus pandemic and unfriendly summer winds — meant the teams had to wait more than a year to find out if PhiSat-1 would function in orbit as planned.
The 2 September launch from French Guiana — a first-of-its-kind satellite ride-share run by Arianespace — went fast and flawlessly. For the initial verification, the satellite saved all images and recorded its AI cloud detection decision for each, so the team on the ground could verify that its implanted brain was behaving as expected.
After a three-week deep breath, Pastena was able to proclaim: “We have just entered the history of space.”
ESA announced the joint team was “happy to reveal the first-ever hardware-accelerated AI inference of Earth observation images on an in-orbit satellite.”
By only sending useful pixels, the satellite will now “improve bandwidth utilisation and significantly reduce aggregated downlink costs” — not to mention saving scientists’ time on the ground.
Looking forward, the usages for low-cost, AI-enhanced teensy satellites are innumerable — particularly when you add the ability to run multiple applications.
“Rather than having dedicated hardware in a satellite that does one thing, it’s possible to switch networks in and out,” says Jonathan Byrne, head of the Intel Movidius technology office. Dunne calls this “satellite-as-a-service.”
Consider: When flying over areas prone to wildfire, a satellite can spot fires and notify local responders in minutes rather than hours. Over oceans, which are typically ignored, a satellite can spot rogue ships or environmental accidents. Over forests and farms, a satellite can track soil moisture and the growth of crops. Over ice, it can track thickness and melting ponds to help monitor climate change.
Many of these possibilities will soon be tested: ESA and Ubotica are working together on PhiSat-2, which will carry another Myriad 2 into orbit. PhiSat-2 will be “capable of running AI apps that can be developed, easily installed, validated and operated on the spacecraft during their flight using a simple user interface.”
For Intel, the potential impact is unquestionable. As Pastena puts it, we can eventually understand “the pulse of our planet”.