Midnight Planets goes to some lengths to present rover images in an appealing way.
The app’s approach is to present as much color as possible, and to adjust images so that they work well in the panoramic view.
Image fidelity is a high priority, but ultimately Midnight Planets is intended for a general audience, so bandwidth and storage requirements must be carefully managed.
The app’s approach to color is to get close to how things would actually look on Mars, while conceding that exact color is a highly technical subject which is beyond the scope of this project.
There are limitations on the source images the app has access to: some calibrated images are available, but in many cases images are “raw” and difficult to work with.
Opportunity images within the latest 6 to 9 months are always “raw”, and are presented in false-color with no attempt at correction.
The discussion below gets technical. Feel free to skip it; or, if you want those details, read on.
Midnight Planets applies corrections to rover images for several reasons:
Raw images from the rovers are often “vignetted”, meaning that the center of the image is brighter than the edges due to lens effects; this needs to be corrected to produce good panoramas.
Raw images posted to JPL’s web sites are often “brightness-stretched” to produce an nice image for casual users; this process has to be undone somehow to produce consistent images for panoramas. No information is provided as to what brightness stretch was applied to each image, so reversing the process is largely guesswork.
Curiosity (MSL) color images are similar to images that might be taken by a good consumer digital camera on Earth, but they require a slight correction to produce a “natural” color.
For the Mars Exploration Rovers (MER) Spirit and Opportunity, the Pancam instrument acquires grayscale images through different color filters, which can be combined to produce false-color or approximately “natural” color. The Midnight Planets app does this combination dynamically (that is, “on the fly”) according to some complicated rules.
There are basically two different sources of rover images: “raw” JPEG images posted for casual viewers, and “calibrated” images posted for reference on the Planetary Data System. (That is actually an oversimplification, but I’m trying to keep this clear.) Calibrated images may already incorporate corrections for lens vignetting and exposure time, which is good. However, calibrated images are only available to the public six to nine months after they’ve been received on Earth, which is a long wait. So, often the “raw” images are what Midnight Planets has to work with.
Some cameras on the rovers produce grayscale images, and some produce color images. What “true color” looks like on Mars is a surprisingly complex subject, which Midnight Planets is actually going to steer well clear of. Midnight Planet’s goal is to produce an approximately natural color view. The details of how this is achieved are somewhat technical, but may be of interest to some users, so they are described below.
Since the details of what is done are different between MSL (Curiosity) and MER (Spirit and Opportunity), we’ll need to cover those separately.
Due to unexpected complications in working with the PDS image data for MSL, Midnight Planets currently uses only the raw JPG images from the JPL image site. Images in the main view are brightness-corrected using parameters which are set by the Midnight Planets author when he has time. These brightness corrections are only approximate. Color Mastcam images in the main view are adjusted by multiplying the value in the red channel by 1.06, the green channel by 1.0, and the blue channel by 1.13. The MSL images on the Midnight Planets website are identical to those found on the JPL MSL raw image site.
Opportunity images older than about 6–9 months, and all Spirit images, are available in calibrated versions on the Planetary Data System. Midnight Planets’ author downloads these images and converts them to JPG format, applying a reduction to go from 16-bit radiance values to 8-bit image values. When the reduction is applied, “radianceOffset” and “radianceScaling” parameters are stored in the JPG EXIF data, so that the Midnight Planets app can reconstitute the image values back to 16-bit radiance values when the images are rendered in the main view. This works surprising well, as you can see for yourself in the app; the visual results in most cases are almost undistinguishable from using the original calibrated images, while the storage and bandwidth requirements are made much more manageable. However, the results should not be relied upon for any exacting scientific purposes.
The Pancam instrument on the Mars Exploration Rovers Spirit and Opportunity takes pictures through different filters, which can be combined to create false-color or approximately true-color images. When you see a description of an image like “L456”, for example, that means it is a combination of images taken with the Left 4, 5 and 6 filters. Midnight Planets combines these images dynamically, currently choosing the combination that yields the result closest to “natural” color. A good quick summary of the filters can be found at www.ominous-valve.com/pancam.html; more information is available on the Cornell Pancam site.
In the case of calibrated images, the Midnight Planets app performs simple combinations of two or three different filters to produce color in the RGB color space. The general goal is to produce a “natural” color view, but as most of the Pancam filters do not correspond to red, green or blue, the result is often very approximate. Specifically, L456 and R21 combinations produce something approaching “natural” color, but every other combination of filters is scaled to produce a false color that approximates L456 as nearly as possible.
MER-B Opportunity images from recent sols are not calibrated. The images posted to the JPL raw image site are individually brightness-stretched, which is nice for casual users of the website, but useless for presenting color. Therefore, no attempt is made to correct the color for these raw images, and you will see varying results depending on what is in the field of view. Occasionally, an L456 combination containing a bright part of the rover may produce something approaching natural color, more or less by accident, but other than that it’s a festival of wildly-varying false-color. For grayscale images, especially Navcam, the author may set parameters for brightness-correcting the images in the main view. The raw images on the Midnight Planets web are left untouched; they are the same as those from the JPL raw image site.
The color filter combinations that Midnight Planets can apply are listed below, in order of preference, along with the scaling applied to each channel for calibrated images. (Raw images are combined without scaling.) Midnight Planets’ goal is to provide as much Pancam color as possible, so a number of combinations are supported. To arrive at the scaling parameters, the author visually adjusted (or “eyeballed”) L456 images to best match “true color” images from the Cornell Pancam website and Curiosity Mastcam images. (The first image for comparison was Sol810A_P2536_1_True_RAD.) Then, the other filter combinations were adjusted visually to best match the L456 appearance. Although these are very subjective choices, the same filter parameters are applied to both Opportunity and Spirit, for some sort of internal consistency.
L456: R*1.0 G*1.25 B*1.18
L257: R*1.05 G*1.2 B*1.8
L256: R*1.05 G*1.2 B*1.18
L247: R*1.1 G*0.7 B*1.7
L27: R*1.1 B*1.7
g = 0.4r + 0.6b
L26: R*1.1 B*1.18
g = 0.3r + 0.7b
R21: R*1.05 B*1.8
g = 0.35r + 0.65b
In addition, some single-filter calibrated MER images are scaled in brightness to better match the typical brightness of other images:
L7: 3.0x
R1: 3.0x
L6: 1.8x
L8: 0.000025x
R8: 0.000004x
M1: 5.0x
M2: 5.0x
These values apply to Midnight Planets app version 1.3, in which the brightness of MER calibrated color combinations was increased by 20% to better match the brightness of background Navcam images. These values are always subject to change; the author will try to keep this page up to date for the latest version of the app.