The world gets to see the rusty, dusty martian terrain as if humans themselves were riding atop the rovers. Who's responsible for our front-row seats on Mars? The masterminds working tirelessly behind the scenes in the Multimission Image Processing Lab (MIPL) at the Jet Propulsion Laboratory.
Like professional basketball stars, MIPL (pronounced 'mipple') team members at Jet Propulsion Laboratory perform amazing technical feats and shoot images from Mars into the Internet in record, near real-time before their shot clocks expire many times a day.
Not only do they create the seamless landscape pictures from Mars, but the imaging team also keeps the science game moving by creating image products that enable the rovers to rove and the scientists to collect science.
"We're what's under the hood," explained Helen Mortensen, former MIPL team leader and development engineer.
"You won't see us at press conferences, but we're behind the creation of the images and the safe movement of the rovers."
The Invention of Digital Cameras and Image Processing
In the early days of space exploration, engineers had to figure out a way to take a basic television signal and convert it to zeros and ones in order to store image information onto a computer and beam the pictures back to Earth.
Thus, digital cameras and digital image processing were invented at JPL for the space program, and enabled the first views of the moon and Mars.
We've certainly come a long way in a few decades. Not only do we have better and better pictures of Mars and the other planets we're interested in exploring, but we also have seen a number of benefits here on Earth. MIPL is an unsung hero of sorts.
"The technology designed for the space program has now made it into the commercial world, and anyone who uses a digital camera and a home computer to download and process pictures is using MIPL technology," said Zamani.
Scissors, Paper, and Paste for Space-Age Images?
Now, utilizing software that JPL has been developing and improving for over 30 years, scientists and engineers can see images from Mars within 60 seconds from the time the data hits Earth.
MIPL processes 30 to 250 images a day, per rover. Through a MIPL software program named File Exchange Interface, museum partners around the nation can also get high-resolution images from Mars out to the public in nearly real time.
More than 100 museums were able to participate, bringing the excitement of Mars exploration to people in the communities where they live. MIPL team members were even able to visit some of the museums, talking with students about their work.
Three-Dimensional Terrain Maps and Reaching the Rocks
However, other important but less obvious products generated by MIPL are the behind-the-scenes, high-tech stereo images that are used by rover drivers to maneuver safely on Mars.
Without knowing the distance to each rock near the rover or slope in the terrain around the rover, rover drivers could not safely command the rover to drive toward a specific location or send the robotic arm out to reach a target with the scientific instruments.
MIPL image processing uses some basic geometry to find distances of objects from the rovers.
"Geometry was my favorite subject in high school, but back then I didn't know why those techniques were taught, and now it's good to see that stuff teachers tell you to do actually can be used on exciting things like space missions," said Zamani.
Working just like human eyes, the MIPL team uses two images taken from different angles - a left and a right - to determine depth in the images from Mars.
"If you hold your finger in front of you and close one eye, you'll see that your finger moves if you look at it using your left eye or your right eye. That's parallax.
Now move your finger out far from your eyes and look at it from your left eye versus your right. Things that are closer jump more than things that are far away," explained Zamani.
The MIPL team finds common features in every left and right image pair taken from the same rover position, so they can calculate a triangle and figure out the distance of the common object from the rover, making a three-dimensional map.
Astoundingly, MIPL software does that pixel by pixel for 1,000,000 pixels per picture pair.
"There are very heavy-duty math algorithms in the MIPL correlation software that find matching pixels in the right image for every pixel in the left image," said Bob Deen, chief software developer for the MIPL Mars team.
Know Your X,Y, Zs to Move on Mars
The IDD (Instrument Deployment Device) maps and range maps determine which points are reachable by the arm's various instruments.
The XYZ maps and terrain wedges represent the 3D terrain to ensure the rover won't hit anything, and the slope images show slope changes that could be hazardous to drive down or up, Deen explained.
The MIPL team creates these maps from telemetry that comes down from each rover multiple times a day.
They organize the "metadata" for every image, which is over three hundred lines of essential information per image, such as exposure time, temperature, and rover body orientation and articulation at the moment the image was taken.
The team must process all of this information and whip out 14 vital products per image at a breakneck speed.
That way, scientists and engineers have enough time to digest the information and utilize the maps to command the rover in time to meet the next communications deadline each day.
In order to expedite the map-making and mosaic-making process, MIPL created a data "pipeline."
Organizing Millions of Bits of Information
The pipeline processes all of the data from Mars into an automated assembly line that enables scientists, engineers, and the public to see and comprehend the martian world in all of its glory.
A pipeline pilot ensures that all the information flying into the system is heading to the right destination.
"The MIPL pipeline is like a gourmet catering service that preps, cooks, and serves packets of information from Mars into the right people's hands on Earth almost instantaneously," said Doug Alexander, MIPL team leader and development engineer.
The pipeline is the "smarts" and "big glue" that runs a lot of programs, Zamani explained. "It says, 'If I have a left pancam image, I need a right pancam image to make a 3D image.'"
Depending on rover power levels and the amount of time Spirit and Opportunity have to transmit data via the Odyssey orbiter relay or a direct-to-Earth antenna link, team members on Earth receive only some of the data during communications downlinks.
The pipeline looks for any missing pieces of information based on time codes and knowledge of when information should have been collected. It may take four or five days to get all the images down to create one panoramic mosaic.
In the meantime, beyond getting the images for a broad landscape, the MIPL team must get enough pieces of information every day to create the 3D maps, so "the pilot watches the pipeline like a hawk," Zamani said.
Straddling Science and Operations
The MIPL team developed methods to make sure that what the rover uplink team commanded the rover to send back to Earth did in fact come down the next day.
As the pipeline pilot does the checks and balances over tens of thousands of lines of data, other MIPL team members check the quality of the other 14 products.
"We straddle the fence between science and operations," Alexander said.
Thrown into a small area called the "bullpen" for 12-hour shifts, six MIPL team members are generating the fundamentals that make the rover function around the clock.
The skilled professionals are constantly creating 3D terrain meshes and "gluing" together multiple pieces of data from the hazard-avoidance cameras (up-close images), navigation cameras (middle distance images), and panoramic cameras (far-away images) to give a different view of the martian landscape for multiple tactical purposes.
MIPL software products and team members even remove the fisheye view from camera images and create a human-eye view.
"The MIPL-engineered software has been optimized for processing rover camera data automatically, but the beautification of the mosaic products is a result of human intervention," Alexander said.
Non-stop Scientific Art
The software developed at JPL can process camera data to quickly generate a mosaic, but detail-oriented, tireless team members must eliminate the doubling of the foreground rocks and match the brightness for one scene that may have been shot over multiple days.
Known as the Mosaic Czar, Bob Deen didn't take one day off for the first 90 days of the rover mission.
"I was the general trouble-shooter at the beginning of the mission, and we all wanted to improve the software to generate products quicker for the rover drivers and press conferences, so it was too exciting to take a day off," said Deen.
"The worst stress comes from the terrain meshes for the rover drivers because they can't plan the traverse until we combine the terrain information from various cameras."
Early in the mission, stress also flared when press conferences were happening every day. Scientists would often come to Deen and the MIPL team and request new mosaics or other special images at 5:00 a.m. for a press conference planning meeting at 6:00 a.m.
One night, after processing Mars images for 14 hours, Helen walked out of the office and noticed an odd iridescent glow in the distance.
"I thought, 'gosh, that's really strange - I wonder what that is,'" said Helen. She later realized it was a band of the Sun rising on Earth . . . a sun she hadn't seen for a few days working the martian day.
"I have always liked science fiction and computers," Deen said.
"My dad had a business doing tax returns in the early '70s on computers when computers were big mainframes in a room. I got an Apple II in high school and learned everything."
"I thought I wanted to work for Lucasfilm (of Star Wars fame), but when I found out in college that NASA hired people to run the computers for space science, I knew I wanted to do real science instead of science fiction.
Now, after looking at all the rocks on Mars, I want to take a geology class," Deen said.
"Our team has an age range of 21-year-old college students to people at the twilight of their retirement, and it's interesting to see how everyone gets along with a common goal," Alexander said.
The rover team is full of some of the brightest and most creative scientists and engineers who have to balance their various goals for their instruments.
"People work shoulder to shoulder with no envy, and they get over political work issues quickly in order to ensure the science gets accomplished," Alexander explained.
Everybody makes sacrifices depending on the most efficient tasks of the day and how the team can use the various instruments to help solve the mysteries of history of water on Mars.
"It's amazing how much human energy is behind a six-wheeled rover," he said.
Rewriting Science Books
"Everybody is curious about the unknown, and one way to unveil that curiosity is to actually seek and give facts."
With the image processing technology MIPL has developed for NASA, students can use the Internet as their textbook and see crystal-clear images from Mars within hours of the moment the picture was taken on Mars.
"For the missions I've worked on at NASA, including missions to Neptune and Jupiter, I have seen that technology actually goes into the public and ultimately benefits our entire society," she said.
"Our current techniques for looking at data and analyzing the data are becoming faster and more sophisticated, and the commercial applications are infinite."
Working with every planetary mission at JPL, MIPL is creating new views of our universe and changing the way we see the solar system and beyond.
"My advice to any student who is interested in space is to learn as much math and science as you can. You are always going to see something really cool if you just keep your eyes open."
To learn about other star players on the MIPL team, please visit Zipcode Mars and type in the names of individuals in the team photo.
Mars Rovers at JPL
Mars Rovers at Cornell
Subscribe To SpaceDaily Express
Analysis End Not In Sight For Mars Rovers
by Phil Berardelli
Washington DC (UPI) Jul 6, 2004
At a NASA-sponsored briefing for journalists last December, scientists with the Mars Exploration Rover missions described the various hazards that could cut short the planned, twin 90 (Earth) day operations on the red planet and limit the amount of data shipped home by the robotic vehicles from their respective landing sites.
|The content herein, unless otherwise known to be public domain, are Copyright 1995-2016 - Space Media Network. All websites are published in Australia and are solely subject to Australian law and governed by Fair Use principals for news reporting and research purposes. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA news reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. Advertising does not imply endorsement, agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. Privacy Statement All images and articles appearing on Space Media Network have been edited or digitally altered in some way. Any requests to remove copyright material will be acted upon in a timely and appropriate manner. Any attempt to extort money from Space Media Network will be ignored and reported to Australian Law Enforcement Agencies as a potential case of financial fraud involving the use of a telephonic carriage device or postal service.|