This self-portrait of NASA's Curiosity Mars rover shows the vehicle at the "Big Sky" site, where its drill collected the mission's fifth taste of Mount Sharp, at lower left corner. The scene combines images taken by the Mars Hand Lens Imager (MAHLI) camera on Sol 1126 (Oct. 6, 2015). Credit: NASA/JPL-Caltech/MSSS

Why Don’t We See the Curiosity Rover’s Arm When it Takes a Selfie?

Article Updated: 23 Dec , 2015

by

Every time the Curiosity rover takes a ‘selfie’ on Mars, we get the same questions: “How was this picture taken?” “Why isn’t the rover’s arm or the camera visible in this picture?” “In all of Curiosity’s selfies, the camera mast is never visible… why?” And (sigh) “What is NASA hiding???”

The answer is simple and quite logical. Look any selfie image you’ve taken. Does your hand show up in the picture?

No, because it is behind the camera.

The same is true with the rover’s arm. For the most part, it is behind the camera, so it isn’t part of the picture. In your own selfies, if you’ve done a good job of positioning things, your arm doesn’t appear in the photo either. For example, take a look at this selfie taken last night by Astronomy Cast co-host Pamela Gay of her co-host (and Universe Today publisher) Fraser Cain, along with their fellow speakers at the Next Frontiers Symposium at Ohio State University.

You’ll notice Pamela’s arm isn’t showing, even though she took the picture of herself, just like the rover takes pictures of itself.

Just think of the rover’s arm as the ultimate interplanetary selfie stick. The best selfie-stick pictures are where the stick doesn’t show up in the image and it appears you had your own photographer. That’s what happens with the Curiosity rover self-portraits.

It’s important to note that while the rover selfies look like they are just one single image taken by the wide-angle camera on the rover, it is actually a series of individual images stitched together. The picture above is made from dozens of images the rover took of itself with the Mars Hand Lens Imager (MAHLI) camera at the end of the rover’s robotic arm. Curiosity moves its robotic arm around and over itself and the ground, taking pictures from every angle. These pictures are then stitched, just like panoramic images are put together to form a complete image of your total view. The rover’s arm isn’t long enough to make the camera’s field of view big enough to get the entire rover in one shot (similar to how it works if you hold your camera/phone close to your face you only get one feature, like your nose or eyes, not your entire body.)

Update: As for the questions of why the rover’s arm doesn’t show up in these rover selfie images, I conferred with Guy Webster from JPL and he said that portions of the arm do show up in some of the images used to create the selfie shots, but the portion of the arm pictured is very limited, and the team feels it would be even more confusing to include the small parts of the arm that are in some of the images and so have decided to leave it out entirely.

“Some of the component images do indeed show glimpses of the arm,” Webster said via email. “There’s selectivity in choosing which parts of which component frames to use in assembling the mosaic, to avoid having discontinuous bits of arm in the scene. That would cause confusion even quicker than making choices that leave out the arm.”

For example, here is one image from the series of pictures taken by the MAHLI to create the selfie, and it shows just a small piece of the arm, near the “shoulder”:

A small portion of the Curiosity rover's robotic arm (the white 'tube' on the top left of the image) shows up in one of the original raw images used to create the montage 'selfie.' Credit: NASA/JPL-Caltech/MSSS.

A small portion of the Curiosity rover’s robotic arm (the white ‘tube’ on the top left of the image) shows up in one of the original raw images used to create the montage ‘selfie.’ Credit: NASA/JPL-Caltech/MSSS.

You can see the entire collection of MAHLI images from Sol 1126 (Oct. 6, 2015) here. You can see how few images show parts of the arm, and how little of the arm shows up in the ones that do.

For the most part, though, because of the flexibility of the robotic arm and the way it is able to move, the arm ends up behind the camera. As Curiosity’s Engineering Camera Team Leader Justin Maki explains in the video below, “The rover is imaging the deck while the arm is behind the camera, and then to image the ground … again the arm is behind the camera when taking these pictures. When we stitch them all together, you don’t see the arm in any of the pictures.”

Click on the image to start the video (which shows very well why the arm isn’t in most of the shots):

It’s interesting to note, that while the lead image above — the latest rover selfie — does not include the rover’s robotic arm, the shadow of the arm is visible on the ground. You’ll notice there seems to be an extra “joint” in the arm, which is just part of the image editing, particularly the stacking of the images where the ground is, where the image editors used more than one image for that area. For the selfie image below, taken in 2012, the imaging team chose not to include any shadow of the arm.

A color self-portrait photo of Curiosity standing on Mars, on sol 84 (October 31, 2012). The photo is a mosaic of images shot with MAHLI, the camera on the end of the robotic arm. Credit: NASA/JPL/MSSS.

A color self-portrait photo of Curiosity standing on Mars, on sol 84 (October 31, 2012). The photo is a mosaic of images shot with MAHLI, the camera on the end of the robotic arm. Credit: NASA/JPL/MSSS.

Why does the rover imaging team take these rover selfies? Are they just joining in on the selfie craze here on Earth?

These images are actually a great way for the rover team to look at all the components on Curiosity and make sure everything looks like it’s in good shape. The wheels are of particular interest because there has been some damage to them from driving over sharp rocks. These images also document various areas where the rover has worked, and often include things like the holes the rover has drilled into the Martian rocks and soil.

Emily Lakdawalla at The Planetary Society has written an extremely detailed post on how the rover takes self-portraits. She created this composite image of the 72 individual frames the Mars Hand Lens Imager (MAHLI) had to take in order to cover the 360-degree view showing the rover’s underside, a “belly selfie“:

Curiosity's arm-mounted MAHLI camera took 72 individual photos over a period of about an hour in order to cover the entire rover and a lower hemisphere including 360 degrees around the rover and more than 90 degrees of elevation. It took 2 tiers of 20 images to cover the entire horizon, and fewer images at lower elevations to cover the bottom of the image sphere. The arm was kept out of most of the images but it was impossible to keep the arm's shadow from falling on the ground in positions immediately in front of the rover. Credit: NASA/JPL/MSSS/Emily Lakdawalla.

Curiosity’s arm-mounted MAHLI camera took 72 individual photos over a period of about an hour in order to cover the entire rover and a lower hemisphere including 360 degrees around the rover and more than 90 degrees of elevation. It took 2 tiers of 20 images to cover the entire horizon, and fewer images at lower elevations to cover the bottom of the image sphere. The arm was kept out of most of the images but it was impossible to keep the arm’s shadow from falling on the ground in positions immediately in front of the rover. Credit: NASA/JPL/MSSS/Emily Lakdawalla.

Here’s another longer video from JPL that explains all the rover’s cameras.

A mosaic of images from NASA's Curiosity rover shows what appears to be a "selfie" with a Martian mountain (Aeolis Mons)in the background. Credit: NASA/JPL-Caltech /MSS/ Image editing by Jason Major.

A mosaic of images from NASA’s Curiosity rover shows what appears to be a “selfie” with a Martian mountain (Aeolis Mons)in the background. Credit: NASA/JPL-Caltech /MSS/ Image editing by Jason Major.

, , , , ,



14 Responses

  1. paul w says:

    I may be setting myself up for a huge fail here, but…

    The correct answer is the photo is stitched together from lots of others, and so the picture of the arm holding the camera is not included.

    Using a selfie as an example is not the right answer – we cannot see our whole body. If we did, we would see our arm disappearing into the side of the frame.

    • Bo Zo says:

      … And the author eventually says just that. But only after the true but false statement about the hand being behind the camera, which unfortunately confuses the matter; that’s true, obviously, but ‘where’s the arm?’ is the actual question. This rover selfie would be “impossible” without stitching together 2 or more images.

      (Unless it was taken in a Hollywood backlot by the not-really-deceased gubberment stooge, Stanley Kubrick. Obvously.)

      • paul w says:

        …which I actually meant, but didn’t explain myself properly.

        (whisper) don’t talk about kubrick! We’ve gotten away with it so far! Hehe.

      • Bo Zo says:

        Great! Now that that’s settled, there’s still one unbelievable thing, which is how cool it is that humans have put such awesome rovers on Mars. We can just tap on a mobile phone and look at hi-res Mars photos. Unbelievably cool.

    • Yes, many images are stitched together, but beyond that, the arm *never* usually doesn’t show up in any images because it is alwaysusually behind the camera. Look at the Instagram video in the article to see why.

      • Dashdragon says:

        Right, but the piece that puzzles those that don’t think about how the resulting image is created, is that – on a selfie – your Hand or the end of the ‘selfie stick’ is what is never visible, but part of your body (usually your arm) or the stick must exit the edge of the frame to reach the point of the part that is holding the camera. This is because a selfie is generally a single image.

        In the example photo, for instance, most of Pamela’s body is not in frame which makes it easy to see that her left arm is probably holding the camera. Really many of the people shown could have taken the pic. Even Frazier could have taken it since his body and arms are obscured by the man in front of him whose head reaches the edge of the frame. That’s the key part – some part of the subject(s) must exit the edge of the frame in a selfie.

        With the Curiosity selfies the rover is shown fully in frame with barren Martian landscape around all sides. No part exits the frame to be holding the camera. This is because of the panoramic nature of the shot and the fact that the armature the camera is mounted on is purposely left out in the stitching process. In a single shot of Curiosity in full frame, the arm most-definitely would be visible…the camera mount obviously would not, but that is a bit different.

        The real answer then becomes – every single selfie used to create the composite image has either the arm or part of Curiosity leaving the edge of the frame to reach the point where the camera is mounted at. However, the panorama was purposely stitched together to remove that. In the end, this is not a ‘selfie’ but a panoramic image of selfies.

      • Dashdragon says:

        Which, I do understand that’s what you were getting at…just the wording makes it seem as if you should never see what is holding the camera on a selfie, which isn’t really accurate. You will always see ‘something’ leaving frame from whomever or whatever is holding the camera in a single shot selfie.

    • tgbrendaj says:

      Your answer is right on the money. And the author is at fault by not mentioning that the shot was photostitched thus removing the arm from each picture.

      • OK, I conferred with the folks at JPL, and they did say that small portions of the arm do show up in some of the series of images taken to create the selfies, but very little of the arm shows up in the ones that do. The team decided it would be more confusing to add the small bits of arm than to leave them in. I updated the article to reflect that and added one image that shows just how little of the arm appears in the images. There’s also a link to see the entire series of images used.

  2. Justin LG says:

    No you are right. Like you said if you took a selfie of your entire body you would see your arm going outside of the frame. In the Curiosity rover photos the arm is there, they either stitch it out or photoshop it.

    With arm: https://metrouk2.files.wordpress.com/2015/08/selfie1-e1439310110751.jpg?w=748&h=418&crop=1

    Without arm: http://www.daily-sun.com/assets/news_images/Curiosity.jpg

  3. Aubri says:

    That still doesn’t answer the question. For any given image of the rover’s “shoulder”, the arm must necessarily exit the frame at some point. When the images are stitched together, therefore, the arm must terminate at the edge of another photo. So once again: where is it?

    • Dashdragon says:

      On the cutting room floor. The arm exits the frame at different positions of each shot of the panorama used to create the image. They stitched the images together covering the arm in each shot with the non-obscured area from another shot. The end result gives you a picture with no arm exiting the frame. I clarified the problem with the wording to explain it on my reply to Nancy above as the current article makes it almost seem as if you should never see the arm on a selfie, which as you note is not the case – you won’t see your hand, but you will see the arm or shoulder, etc, exit the frame on one side or another.

  4. Justin LG says:

    Why is my comment from October 15th still awaiting moderation…?

    • Putting in more than one link sometimes puts comments on hold. Not much of the arm shows up, as you can see from the image I added, and you can see why in the Instagram video.

Comments are closed.