SECCHI_PREP Users Guide

Using SECCHI_PREP with HI1 and HI2 data

$Id: hi_prep.html,v 1.9 2014/01/29 17:22:25 crothers Exp $ii

Overview

SECCHI_PREP,filenames, headers, images [,SAVEPATH=path] [,OUTSIZE=value] [,/WRITE_FTS] [,/WRITE_PNG][,/WRITE_JPG][,/TRIM_OFF]

[,COSMICS=variable] [,/SEBIP_OFF] [,/CALIMG_OFF][,/NEW_CALIMG] [,CALIMG_FILENAME=filename] [,/COLOR_ON] [,/DATE_ON] [,/LOGO_ON] [,/SHUTTERLESS_OFF][,/DESMEAR_OFF][/UPDATE_HDR_OFF] [,/NOCALFAC_BUTCORRFORIPSUM]

Any keywords for sub-procedures may be passed via SECCHI_PREP.

Default Procedures

  • image trim
  • correcting for SEB IP
  • removing CCD bias value
  • extract cosmic ray counts [replaced by adjacent pixels for subsequent processing]
  • masking columns suffering from significant saturation
  • removing the smearing effects of the shutter-less readout
  • applying discrete source calibration image
  • applying diffuse source correction [adjusting for the size of pixels on the sky]
  • applying calibration factor to MSB
  • setting pointing information according to star-fitting
image trim
IDL> img = scc_img_trim(img,hdr,[,/silent])

Full resolution images include an over and under scan region in the image. These regions are used to calculate the image bias and other calibrations. The over/under scan regions do not contain solar data. The default procedure for SECCHI_PREP is to trim off the over/under scan regions of the array. The images will not be trimmed if the keyword TRIM_OFF is set in SECCHI_PREP. If the keyword TRIM_OFF is set then the keyword CALIBRATION_OFF is also set. Also the output size of the images array becomes 2176x2048.

correcting for SEB IP
IDL> img = scc_sebip(img,hdr,ip_flag)

see the Science Operations Manual (MSWord), table 5.3, for descriptions of the HI instrument IP functions. Can be disabled by setting the flag SEBIP_OFF.

removing ccd bias value
IDL> img = img - get_biasmean(hdr)

Can be disabled by setting /bias_off

Extract cosmic ray counts
IDL> cosmics=hi_cosmics(hdr,im)

For images not summed on board this function returns hdr.cosmics. For summed images however it attempts to extract the individual cosmic ray pixelcounts that are stored in the image, these values are replaced in the image with the adjacent pixel values to minimise the effect on the subsequent desmear process. This may not be possible if for instance there is a missing block where the cosmic ray data should be stored. If the data is extracted an array will be returned and the data patched by replacing the cosmic ray field with a duplicate of the adjacent pixels. If the cosmic ray data can not be extracted the routine returns a scalar "-1". For the summed images the first sample is taken over a longer time period than the othes and so it almost invariably significantly higher.

masking columns suffering from significant saturation
IDL> img=hi_remove_saturation(img,hdr,saturation_limit=saturation_limit,nsaturated=nsaturated)

Masks columns adversely affected by saturation by setting all values in a column to "NaN". If more that a given number of pixels in a column record over a given thresold the entire column is masked out. The default is to set SATURATION_LIMIT=14000 (DN) and NSATURATED=5. This masking can be disabled by setting SATURATION_LIMIT=-1. See the discussion below on bleeding for more information.

removing the smearing effects of the shutterless readout
IDL> img=hi_desmear(img,hdr)

Due to the lack of a shutter the image is smeared, both when the CCD is cleared and when it is read out. This routine calculates the inverse operation to recover the original image. Missing blocks, saturated pixels, and cosmic rays can cause this operation to create artifacts. Using the switch /DESMEAR_OFF will select instead the routine "hi_exposure_wt.pro" which simple multipies each row of the final image by the recipricol of the time it was exposed as a combination of clearing the ccd, exposure and readout. Both of these routines normalize the exposure time to 1 second. Using the switch /EXPTIME_OFF will do nothing. See further discussion in the shutterless readout section below.

applying discrete source image calibration
IDL> calimg=get_calimg(hdr,fn)

The flat-field corrections and photometric calibrations of the HI-1 images are derived by fitting to response to a large number of stars of different spectral types and then normalizing to a Solar-type spectrum, for detailed information see "Determination of the photometric calibration and large-scale flatfield of the STEREO Heliospheric Imagers: HI-1",

Bewsher, D., et al., 2010, Solar Phys. 264, 433, "Long-term evolution of the photometric calibration of the STEREO Heliospheric Imagers: I. HI-1", Bewsher, D., et al., 2012, Solar Phys. 276, 491 and "On-Orbit Degradation of Solar Instruments", BenMoussa, A., et al., 2013, Solar Phys. on-line first. Due to the larger pixel sizes and wider point-spread functions, this technique has not so far yielded a good result for HI2 and so they use a pre-launch modelled flat-field and predicted photometric response. This function can be disabled by setting /CALIMG_OFF

applying diffuse source correction [adjusting for the size of pixels on the sky]
IDL> diffuse_image=scc_hi_diffuse(hdr)

This image calibration specified previously is computed based on stellar photometry (point sources). For diffuse sources an additional correction is needed to allow for the different sizes of the pixels off axis. This factor is only applied if both calfac and calimg are applied.

applying calibration factor to MSB
IDL> calfac=get_calfac(hdr)

This routine returns the conversion factor from DN/sec to mean solar brightness. Again, for HI1 this is taken from Bewsher et al 2010, and for HI2 it is a preflight value.

correct pointing information according to star-fitting
IDL> hi_fix_pointing,hdr [,ravg=ravg [default=5]] ,[/tvary]

Replaces pointing information in hdr (CRVALi) by that taken from fitting the image to a star map. Normally the star fit keeps mu constant; /TVARY selects an alternative database where mu is an additional variable. On return, hdr.ravg gives a measure of the uncertainty in the pointing as the average distance between the expected position of a star and the nearest peak in the image data, negative values have a special meaning as:- -881=> no pointing file available, -882=> no pointing data for this file, -883=> poor fit, -894=> too few stars in fit (normally implies sub-field images). In all cases where hdr.ravg < 0, fixed offsets to the satellite pointing parameters from 15-Oct-2007 are used. For more information on the procedure followed see "Calibrating the Pointing and Optical Parameters of the STEREO Heliospheric Imagers", Brown, D.S., et al., 2009, Solar Phys. 254, 185. Typical ravg values for each camara are HI1A 0.15-0.5, HI1B 0.2-0.5, HI2A 0.5, HI2A 1-2.

Pointing calibration is of particular significance for HI1B where the pointing of the instrument jumps occasionally. (See e.g. 20070728_000700_n4h1B.fts where the pointing changes during CCD readout causing the star trail from Venus to move across the CCD.) If this occurs between exposures in a summed image, stars can appear in multiple locations on the CCD and it will therefore not be possible to provide a pointing solution. When using HI1B data it is therefore sensible to filter on the RAVG value to remove images so affected (at RAL we normally use ravg>=0 & ravg<=5).

Optional Procedures

Use these keywords when creating images for display:

  • /COLOR_ON: Loads IDL color table for each the instrument from the $SSW/stereo/secchi/data/color directory. The color files are IDL save files that load the variables r, g, b.
  • /DATE_ON: Add the date and time of the image in the bottom left corner of the image. The font size of the date and time will scaled to match the size of the image.
  • /LOGO_ON: Add the SECCHI logo to the bottom right corner of the image.

Further Explanations

Shutterless Correction

Since the cameras do not have shutters, images are smeared across the camera during the process of clearing and reading out of the CCD. This operation is constrained by three numbers: the time the image is static on the CCD, the time to clear a row, and the time to readout a row. The measured pixel value is : 

 observe_pixel[i,j] =  sum(actual[i,0:j-1]*lineclear) + actual[i,j]*exptime + sum(actual[i:j:n]*linereadout).

Over an entire column, this equation can be expressed as a matrix where all the cells on the main diagonal are the exposure time, all those above that are the readout time and all those below are the cleartime. Taking the inverse of the matrix and matrix multiplying by the observation removes the desmear and also converts from DN to DN/s.  This computation is applied to the image by the procedure :

IDL>result = HI_DESMEAR(im, hdr)

Cosmic Rays: The full resolution single-exposure images (YYYYMMDD_hhmmss_nxxxx.fts' typically 9Mb in size) are not cleaned of cosmic ray effects. Cosmic rays appear as bright objects but do not smear. Applying simple desmear to these will result in negative trails of the cosmic ray in the output data. More sophisticated image processing would be necessary to remove these effects. 

Cosmic Ray Report: The summed (YYYYMMDD_hhmmss__s4xxx.fts, typically 4Mb) use a cosmic ray scrubbing process and should not be affected in this way. In addition, the last pixels in the image contain a counter of the number of pixels scrubbed for each image in the sum. Before the image can be corrected these pixels are replaced by duplicating pixels from the row below. This will occasionally cause desmearing artifacts in the same way as for cosmic rays.

Bleeding : When a pixel overflows the charge is constrained by the design of the CCD to bleed vertically up and down the columns. If this occurs to a small level contained within the CCD then the effect on shutterless correction is small, although the corrected image will still show the effects of this bleeding. Detection and removal of saturated pixels (with hi_remove_saturation.pro) is best done before desmearing since the saturation level will be convolved with the desmear matrix in the prepped result. On rare occasions where there is strong bleeding into the readout register, charge can leak sideways in the readout register and back up adjacent columns; see 20070114_000100_s4h1A.fts where this happens with comet McNaught. Here the desmear process produces a false reduction in the output signal in the few columns so affected. Even less often, significant bleeding can occur as the image is being read out. In this case you can see excess counts in the first row of an image, e.g. see 20070220_010000_n4h1B.fts. The default in secchi_prep is to mask columns where significant saturation has occured. In the event that saturation has spread through the readout register there is a utility routine:

IDL>result = HI_EXTEND_SATURATION(im, ncols)

that will extend nominally masked columns with additional columns.

Missing Blocks: Where there are missing blocks the shutterless correction causes bright columns the entire width of the missing data since the f-coronal signal is not being correctly subtracted. It is possible to reduce this effect by masking the missing block with a background image before the shutterless correction, although star trails will still be seen.

Background Subtraction

Solarsoft provides daily median images and monthly minimums of daily median images that you can just use to attempt to separate CME signals from the f-corona. See scc_getbkgimg.pro.

Pre-processed Data Archive

RALSpace holds a ready processed archive of secchi_prep output with standard options available via http://www.stereo.rl.ac.uk/ under the link "Data". Files with names like 20040101_000501_14h1A.fts are in units of DN/Sec/Pixel and so do not include the diffuse source correction. Files with names like 20040101_000501_1bh1A.fts are in units of mean solar brightness and so do include the diffuse source correction. Files with names like 20040101_000501_1th1A.fts are in units of S10 and again include the diffuse source correction.

At the same location there is also background-subtracted data computed using a background computed as the pixel-by-pixel mean of the lowest quartile of all images in running windows of different lengths (a technique suggested by Andy Buffington as being more numerically stable than some other techniques). The filenames here look like 20040101_000501_24h1A_br01.fts for DN/Sec/Pixel with a one day running window, or 20040101_000501_2bh1A_br11.fts for mean solar brightness with an eleven day running window. One-day and eleven-day data is available for HI-1 and three-day and eleven-day data for HI-2. These level-2 data are mirrored through the stereo science center.