SOC keyhole documentation

$Revision: 1.13 $, $Date: 2007/10/16 22:34:18 $, $Author: tsiili $

Keyhole documentation is in your keyholetoolbox and is called SOHO-Keyhole-Toolbox-Users-Guide.doc. For Emily, this is on McSoc at /Users/emily/keyholes/keyholetoolbox/doc. Follow the directions there instead of on this page, but use this for reference. Start idl with sswidlstart.


This page is the central source of SOHO keyhole documentation for the use of the SOHO Science Operations Coordinators. The page contains both standalone sections as well as (when documentation already exists elsewhere) links to other web pages and references to other sources of information. The purpose is to provide a reference and how-to instructions to assist in planning for the keyhole operations; perhaps the largest part of that planning is designing and implementing the keyhole plan.

A short table-of-contents is:

A list of SOHO-related acronyms and abbreviations can be found here.

Keyhole background information

In the context of the SOHO mission, a keyhole is a recurring period of reduced telemetry. A popular-level description of what a SOHO keyhole means and why it occurs can be found in this SOHO Hot Shot page. The interconnectivity between the keyhole issues and several other sectors of the SOHO mission are outlined both as a graphical mindmap and as a textual outline (general information on mindmaps). These sources are useful in recalling the big picture and avoiding forgetting potentially essential interconnections during the planning process.

Back to table-of-contents

Submodes

SOHO has five submodes (or in other words, telemetry subformats), numbered from 2 to 6. For keyhole planning, practically only submodes 5 and 6 are relevant. Outside of keyhole periods the choice between 5 and 6 depends on whether the SUMER instrument is observing or not; during the keyholes SUMER is not observing, it is safed — and even in submode 5 it is not using its telemetry allocation. The submode choice between 5 and 6 depends on the requirements of EIT/LASCO and of the available downlink capacity, as subsets in submode 6 require more.

Submode changes have in the past caused CDS to crash, but this has not happened in awhile. In case CDS should crash, they need time to recover, which needs to be taken into account in submode change planning.

Back to table-of-contents

Instruments and their connections with the keyhole planning

SOHO's 12 instruments, their alphabetic identifiers and their connections with the keyholes (described with a few keywords) are shown in the table below:

Note 1: CEPAC is in fact a two-instrument collaboration between COSTEP and ERNE. Note 2: EIT and LASCO share an electronics box known as the LASCO Electronics Box (LEB)
Instrument ID Keywords
CDS C Submode changes — may crash and may need 1.5-2 h for recovery. Needs to safe for SK/MM manoeuvres.
CELIAS F Subset
CEPAC H
EIT E Bakeout, submode — influences image resolution
GOLF G In the highest priority subset (VGM)
LASCO L Safes for manoeuvres, data rates and submode choice.
MDI M ALT magnetograms, VC2; In the highest priority subset (VGM); has loop opened for manoeuvres
SUMER S Submode
SWAN N Manoeuvre safing (SK/MM) or cross-calibration (roll-only)
UVCS U Safe configuration
VIRGO V In the highest priority subset (VGM)

Back to table-of-contents

Virtual Channels (VCs) and their transitions

SOHOs telemetry/downlink comprises five Virtual Channels or VCs — VC0–VC4. The image below illustrates their respective meanings:
SOHO virtual channels

Transitions to or from VC2 must not occur too close to ALT magnetogram recording times, which leads to a MDI transition or exclusion rule. That has implications for the keyhole plan design — for the allocations of VC2 times.

Back to table-of-contents

MDI ALT magnetograms

#TODO Add here a description or reference to what the ALT magnetograms are... Operationally magnetograms and transitions to VC2 are mutually exclusive

Magnetograms are recorded at fixed UTC times: 00:00, 01:36, 03:12, 04:48, 06:24, 08:00, 09:36, 11:12, 12:48, 14:24, 16:00, 17:36, 19:12, 20:48, 22:24.

Back to table-of-contents

Subsets and data recorders

SOHO has two different data recorders: the Solid State Recorder (SSR) and the Tape Recorder (TR). Originally both of these always recorded all instruments' data without any subset selection ability. An on-board software modification has allowed SOHO to record and store instruments' observations selectively on the SSR. The TR, however, (still) records everything. The subset selection is useful during keyholes, when the available downlink capacity is constrained and the downlink allocations need to be prioritised. These instrument selections have been grouped into (sub)sets identified by the groups of instrument IDs and the submode:

The recorder subsets, the instruments in the set and the capacity of the SSR in h when this subset is selected. SM = submode. Bold denotes the subsets used most.
Subset Instruments and submode SSR capacity in h
VG VIRGO, GOLF 229.9
VGM VIRGO, GOLF, MDI 65.5
VGML5 VIRGO, GOLF, MDI, LASCO/EIT (SM5) 30.1
VGMFL5 VIRGO, GOLF, MDI, CELIAS, LASCO/EIT (SM5) 26.9
VGML6 VIRGO, GOLF, MDI, LASCO/EIT (SM6) 19.9
VGMFL6 VIRGO, GOLF, MDI, CELIAS, LASCO/EIT (SM6) 18.5
ALL All instruments 10.4

The keyhole planning tool uses the subset identifiers and their respective SSR capacity values in the planning process. For that purpose the values are stored as a lookup table under the IDL directory (typically ${HOME}/idl) in file keyholes/subsets.all. In addition to the subsets listed above, the set ALL includes all of SOHO's instruments. Tape recorder's capacity (for the purposes of the keyhole planning tool) is taken to be 5.2 h and as mentioned above, the TR always records all instrument, regardless of the subset selected.

Back to table-of-contents

Manoeuvres

As explained in the keyhole description (the SOHO hotshot), the orbital geometry together with the fixed position of the HGA requires turning of the spacecraft upside down or a roll manoeuvre. Orbital and attitude control require also station-keeping (SK) and momentum management (MM) manoeuvres. These are carried out during keyhole periods. SK and MM are always made back-to-back, roll is made back-to-back with SK and MM every other keyhole and separately every other. The reasons for this alternation of SK/MM/roll and SK/MM, roll have to do with SOHO's orbit and the spacecraft geometry: roll is separated and delayed to as late as possible when SOHO is at or close to its farthest point from the Earth during a given keyhole period, since the larger distance and shadowing due to the spacecraft's structures degrade the connection via the LGA and hence the data quality.

SK and MM entail use of thrusters, which eject gases to the vicinity of the spacecraft. Since some instruments are sensitive to gases condensing on optical surfaces, those instruments need to be in safe configuration during SK/MM. Thrusters are not used in the roll manoeuvre, hence the safing needs are different for a roll-only manoeuvre.

Instrument activities and safing requirements for different manoeuvres.
Instrument Manoeuvre type Actions
CDS SK/MM/roll, SK/MM Safed
CDS roll Not safed
CELIAS SK/MM/roll, SK/MM Rule-of-thumb: safes, if more than 0.1 kg of fuel is used. Safing needs to be confirmed with the CELIAS team in any case.
CELIAS roll Not safed
EIT/LASCO SK/MM/roll, SK/MM, roll Safed
MDI SK/MM/roll, SK/MM, roll Loop opened for the manoeuvre duration, closed thereafter
SUMER SK/MM/roll, SK/MM, roll Safed, if observing
SWAN SK/MM/roll, SK/MM Safed
SWAN roll Cross-calibrations
UVCS SK/MM/roll, SK/MM, roll Is normally in safe configuration for keyhole already

Preparation for the manoeuvres has checklists — template.maneuvers and template.roll — in directory ecs@soc:/u/ecs/soc/SVM (also accessible via ecs@soc:/u/ecs/soc/manoeuvres).

Back to table-of-contents

Ground stations

The NASA DSN has three diameters of antennae relevant to SOHO: 26 m, 34 m and 70 m. List of the stations is in the file Keyhole, which is inserted as the header of every keyhole plan file from keyholes directory. Same information is also available in file dsn.contacts. In addition to the diameter, an antenna's transmitter does also play a role. Stations are located at three different locations (Goldstone, CA, USA; Madrid, Spain; Canberra, Australia) and each antenna has an identifying number NN. The complete identifier is formatted as DNN or DSS-NN — for example D46 or DSS-46. SOHO needs both uplink (commanding) and downlink (telemetry) functions, but the functions need not be on a same antenna and sometimes having only the downlink function is useful. Hence, we can have double (or even triple) passes (uplink and downlink on different antennae) or downlink-only passes (single antenna not capable of uplink).

In addition to the NASA DSN stations, an ESA ground station in New Norcia (Australia) is sometimes available for downlink-only capability. New Norcia schedulers/planners can be reached via e-mail at Esoc.SCHEDOD@esa.int.

Back to table-of-contents

DSN schedules

Schedule information is retrieved from JPL/DSN. The Macintosh in the SOC office (McSoc) does the retrieval automatically once per day during the local night (for more accurate timing, use the command crontab -l in a shell window on dsn@mcsoc). The information is copied to and further processed at ecs@soc. The key schedule file for keyhole planning is dsn_merged.dat. Schedule information can also be inserted into the creation of the merged file via so called long-term or keyhole schedule files sent to us by the DSN scheduler. For information on these and how to handle them, see long-term-schedule-info.txt. The DSN scheduler also sends the SOCs the following week's weekly schedule on the preceding Thursday. This is a table in Microsoft Word format and not usually used in the keyhole planning.

Back to table-of-contents

Keyhole plan

Background

Keyhole plan is

  1. a schedule of instrument operations for a keyhole period
  2. a text file containing the formal representation of the schedule
The planned schedule is developed by the SOCs through an iterative process with collaboration of and interaction with the

Inputs and constraints

This can also expressed in other words as a question: What information is needed before starting to work on a keyhole plan (indicated in bold below)?

Which pieces of information are crucial and which changes would require restarting the planning process?

Hardware and software tools

The keyhole planning software comprises several tools. The core tool is a program written in IDL, which uses the SolarSoft library — also written in IDL. Since IDL is commercial software, a valid IDL license is a prerequisite. In principle the keyhole planning package can be set up and the planning done on any machine running a unix/linux operating system, has an IDL license and can run the SolarSoft library routines. Currently the keyhole planning package is set up on two machines, spot.nascom.nasa.gov (a Sun workstation) and on Tero Siili's laptop (Apple OS X). There are small differences in running the package in these two environments, primarily since the soc@spot account is shared and it has been deemed necessary to identify, who has been working on a keyhole plan file.

In addition to the core IDL program, there are several unix shell scripts used in different phases of the workflow. One needs also an editor (such as emacs, xedit, jed or similar) to edit the keyhole plan file. The IDL programs create a plot of the plan initially in PostScript format; this needs to be converted to PDF for web serving, hence a PostScript-to-PDF conversion program is needed.

The computer used to run the keyhole planning package must also be able to have files served onto the web either by itself or by access to a server. Currently this is handled by copying (with cp or scp) the keyhole plan text and graphics files to soc@spot.nascom.nasa.gov:public_html, which is regularly copied ("mirrored") to the SOC directory on the SOHO web server. If the web serving setup changes, the planning package needs to be modified accordingly.

A version control system (such as the CVS or Subversion) may be useful to keep track of changes, add automatically timestamps and authors of the most recent changes and perhaps backtrack to earlier revisions of the plan. Currently spot.nascom.nasa.gov does not have a suitable version control software installed, Tero uses CVS locally on his laptop.

Back to table-of-contents

Keyhole plan workflow

Timing of the keyhole plan

The keyhole plan is the basis for FOT's more detailed planning. Consequently the plan (or parts of it) need to be made available to the FOT early enough. A good rule-of-thumb is 2-3 weeks before the actual date of the keyhole (or part of it). Sometimes the plan indeed needs to be made available piecemeal. In case of a long keyhole the FOT planning will also proceed in stages; sometimes the DSN schedule is simply not available for the entire keyhole by the time the keyhole planning has to be started.

Practical details

One needs to know, how to print a text file. This will depend on the environment (for instance, spot vs. OS X vs. linux) and the editor used to work on the keyhole plan file.

Preparation and setup of the keyhole plan file

  1. Ensure, that you have the most up-to-date DSN schedule information. If necessary, request for the keyhole schedule file from the SOHO DSN scheduler and process it according to long-term-schedule-info.txt.
  2. Negotiate and agree with Bud Benefield (or his acting replacement) on:
    1. Which pass is the pass numbered 0 in the keyhole plan
    2. Transponder swap pass from 26 m to 34 m keyhole
    3. Transponder swap pass from 34 m to 26 m keyhole
  3. If you are working on one of the SOC office workstations, enter the name thereof (socop1/socop2). The command is not mandatory, but improves connection speed.
  4. Start IDL and the SolarSoft library with the command sswidl. If they have not already been initialised, you need to give the command a second time. You need to end up in the IDL prompt: IDL>
  5. Create the keyhole object with IDL> k=obj_new('keyhole'). The script asks for
    initials
    (three-character string, e.g., tts; could also be, e.g., SOC)
    keyhole timing information i.e. <keyhole ID>
    (month and year, currently in the form mmmYY, e.g., jun06.
    These two determine the location of the keyhole files under the soc home directory and the name of the keyhole plan file: for example idl/keyholes/tts/jun06/new
  6. If a new keyhole plan is being created, use IDL> k->create_schedule,'<starttime>','<endtime>'.
  7. Suggestion: compare manually the created dsn_merged.<keyhole ID> with the keyhole schedule from the DSN scheduler. You may also want to add the pass numbering to a hardcopy of the keyhole schedule. Comparison should include station identifiers as well as their BOT and EOT times. The identifiers are important, since the antenna sizes and capabilities may differ and wrong id may lead to erroneous decisions and choices in the planning software. Note, that in the dsn_merged.<keyhole ID> file the double and possible triple passes have already been merged, whereas in the keyhole schedule they are not.
  8. Copy the dsn_merged.<keyhole ID> file to a file called Keyhole.<keyhole ID>. You'll be editing the Keyhole.<keyhole ID> file.
  9. Review and deal with special types of passes:
    1. double passes
    2. triple passes (rare, but can occur)
    3. downlink-only ("D/L") passes
    More detailed instructions for dealing with the special passes are behind the links above. These special passes occur, because some of the DSN antennas are only able to downlink from SOHO (for instance, they do not have transmitters in the S-band used by SOHO). Some antennas in turn are not able to downlink due to the keyhole conditions. In such cases either a downlink-only and an uplink-only antenna need to be paired to achieve both commanding and telemetry/dumping capabilities, or a downlink-only connection (no commanding) has to be tolerated to improve the likelihood of successful dump of the recorder(s).
  10. Insert the header file Keyhole from the keyholes directory and edit the header information to have the correct descriptions (which keyhole, etc.). If you are using a version control system, this is the step to add the relevant keywords to the header.
  11. Copy the subsets.all and assumptions files from the keyholes directory to the keyhole plan directory. Add the following lines between the commented header section and the first pass (usually the pass 00):
    %subsets.all
    %assumptions
        
    Note: in previous keyhole plans one or both of these files have been inserted from subdirectories higher in the tree; since there may be changes in those two files over time, it is advisable to have them locally in the directory, if one would need to revisit and reprocess the keyhole plan at a much later date.
  12. Insert the following:
    26 m keyhole start time
    26M STARTS
    Transponder swap time 26 m - 34 m
    34M STARTS
    Transponder swap time 34 m - 26 m
    34M ENDS
    34 m keyhole start time
    26M STARTS
    Note: since DSN passes preceding the actual keyhole period may influence the early keyhole period (via the filling degree of the recorders), you may want to include into the plan some passes before the actual onset of the keyhole. Symmetrically, you may want to extend the plan a bit past the end of the actual keyhole period (if the schedule is known beyond the end of the period).
  13. As a first test try the mode VGM (inserting SELECT VGM right after the 26M STARTS line) for the whole keyhole to see if the DSN passes can satisfy the primary keyhole science criterion: observations by VIRGO, GOLF and MDI that is, VGM. Process the keyhole plan file by giving the command IDL> k->vkhole (which also creates a plot spanning the entire keyhole planning period — see below). Run also the verification process IDL> k->verify to test, whether the plan is robust against pass losses.
    Complete keyhole plan plot/screen
    If even this choice leads to data losses, try obtaining additional passes by contacting
    1. Tina Kelly; if she can not help, contact
    2. Bernhard Fleck
    Suggestion: Since the plot spanning the entire keyhole tends to be very crowded and therefore difficult to view and interpret, it is advisable to generate a hardcopy of the weekly plot, print it and view it as whole. This is a good way of getting the "big picture" of the DSN schedule for the keyhole being planned. The IDL command to create the plot file to-be-printed is k->vkholew,/ps.
  14. If the first test with subset VGM is successful, find out some further information affecting some of the details of the plan. Examples of such details are:

After this you are ready to begin the iterative process of planning for the keyhole.

Before moving forward, a few words about the keyhole plot (see the partial plot below) and what information it displays:

Partial keyhole plan plot/screen

Commands and keywords of a keyhole plan

Commands/keywords used in a keyhole plan
Command Options, arguments Example Explanation, notes
STARTS 26M, 34M 26M STARTS Start of the 26-m or 34-m keyhole period
ENDS 26M, 34M 34M ENDS End of the 26-m or 34-m keyhole period
SELECT VG, VGM, VGML5, VGMFL5, VGML6, VGMFL6, ALL SELECT VGML5 When a subset is selected for use
SWAP to SSR, TR SWAP to SSR When data recording is swapped between SSR and TR
6VC2 6VC2 Allocation of 6 min of VC2
FVC2 6VC2 Allocation of forced VC2 (note: both start and end times have to be specified)
STOREMAG STOREMAG Which magnetogram is to be stored. Time tag is the time of the magnetogram.
DUMPALL DUMPALL Instruct to download/empty/dump recorders. SSR is dumped first and TR after SSR is empty and there is time left on the pass.
PASS D<nn> <pass> PASS D24 77 A DSN pass. DSN station and pass number are required as arguments. Time tags for beginning and end of pass are required.
MAGINIDLE MAGINIDLE
MAGINVC2 MAGINVC2
# text # Pass changed on... Beginning of a comment

Iteration

Commands used in the iteration

For the iterative process the most important tool is the weekly (partial) display of the keyhole plan. One gets to this via the IDL command IDL> k->vkholew. The command is interactive — the user controls the placement of the time window and temporary (test) removal of passes with one-letter commands, possibly combined with a numeric attribute.
Cmd Function
‹enter› Moves the time window 7 days forward and replots the plan graph
‹number›‹enter› Moves the time window ‹number› days forward and replots the plan graph
b‹enter› Moves the time window 7 days backward and replots the plan graph
r‹enter› Replots the plan graph, does not move the 7-day window
k‹pass›‹enter› Deletes pass number ‹pass› in the memory to test the effect of pass loss.
k‹enter› Restores passes deleted in the memory. Opposite of k‹pass›

The workflow

The stepwise and iterative workflow for building the keyhole plan is as follows:

Issues to keep in mind

MDI exclusion rule
Magnetogram handling

The program allocates default VC2 periods to passes using the rule "dump recorder first, then give VC2". This shows in the plot — in the topmost panel the VC2 times shown in green occur after the recorder dumps, unless the default behaviour has been changed or supplemented by the planner. When magnetograms occur close to a VC2 transition, the program gives a warning, either CLOSE CALL (situations which may be technically feasible but not comfortable) or STEPPING ON (technical violations of transition rules versus the magnetogram in question).

CLOSE CALL situations are handled by inserting MAGINIDLE/MAGINVC2 commands from the warnings given by the program. Those commands just acknowledge the situation and make the warning go away. Note: if there are changes in the DSN schedule, these inserted commands may need to revisited. One way of doing it is commenting them out in the vicinities of the changed times and checking, which types of warnings the program produces.

STEPPING ON situations can be handled by

STOREMAG is the command that's most easily used to modify the VC2 timing surrounding a mag that has caused a warning. STOREMAG disallows VC2 for 9 min before & after the specified (magnetogram) time, preventing any of the warnings, and making sure the mag status (yellow/red/green) is correctly reflected. STOREMAG is in a way the "safe option" - and has no detrimental effect unless it causes a mag to become yellow.

  • Check the plot for the interval in question for lost magnetograms or possibly retrievable magnetograms (yellow plus signs). Try to retrieve more magnetograms by adding in 6 min of VC2 (6VC2) near the start and/or end of the following pass to retrieve as many as possible. Some issues to keep in mind:
  • Deal with the program warnings of stepping on mag and close call mag (for more details, see section Magnetogram handling.

    Finally, as a "cover your ass exercise" after inserting commands to make all those warnings go away, it does not hurt to go over each mag that has been "closecalled" or "storemagged" with the involved parties.

    Unfortunately, to be 100 % certain, that walkthrough should involve 4 people: Sarah, Bud, Gary, and whoever will act as OE during the keyhole. Bud, Gary, and each OE have slightly different "styles" of handling transitions and interpreting the keyhole OCD (sigh). You get a long way, though, with just Sarah and Bud.

    As a last tip - be especially wary of using MAGINVC2 at the end of a pass... I think there are some cases [might be dependent on OE style!] that can trick you.

    From MDI: It is imperative that SOC office notify MDI *immediately* of any added or deleted passes to the keyhole DSN schedule. Changes like these might affect the timing and type of MDI load needed to change the ALT cadence or it might change the need to change the ALT cadence altogether.

    Subsets
    Handling of downlink-only passes
    Downlink
    Pass numbers/identifiers

    When the keyhole plan is initiated, the IDL program assigns a running number to each pass, starting with 0. Numbers less than 10 have the preceding 0 included. Sometimes during the keyhole planning process (starting from the creation of the plan up to the last modifications before the end of the keyhole in question) passes are deleted and sometimes even added to the plan. The numbering of existing or remaining passes does not change. Passes inserted into the plan are identified with a number (for example the same as the preceding pass) and a (UPPERCASE) letter. As an example: a new pass is inserted between passes 13 and 14. It is given the identifier 13A. If another would be added between 13A and 14, its identifier would be 13B and so forth. Note: the letter must be in uppercase, since the IDL software package ignores all lowercase letters (another example is the command to swap the recording: one can use either SWAP TR or SWAP to TR, since in the latter form the lowercase "to" is ignored.

    Publishing and delivery

    This comprises the following tasks:

    The derived keyhole plan files (the basic plan and the MDI plan and the plot are created from a releasable revision of the baseline or full keyhole plan file. All four products are published on the web via the SOHO Keyholes page.

    The keyhole OCD comprises of two parts, the default OCD that changes rarely and keyhole-specific OCD, that outlines directions for a specific keyhole. Significant changes to a keyhole plan may also require submission of an additional OCD.

    Back to table-of-contents

    Keyhole Frequently Asked Questions (FAQ) and glossary

    Question/word/topic Explanation and notes
    keyhole plan directory Directory in which the keyhole plan resides
    keyholes directory Directory in which the keyhole tools and plan reside. This is currently ${HOME}/idl/keyholes on both Tero's laptop and on spot.
    SELECT Keyhole plan command used to select the instrument subset for recording
    SWAP Keyhole plan command used to swap between the Solid State Recorder (SSR) and the Tape Recorder (TR)
    transponder swap Change between the 26 m and 34 m keyhole periods
    Virtual Channel 2 (VC2) MDI high-rate data to EOF


    Back to table-of-contents


    Change log and version information:

    $Source: /Users/tsiili/cvsroot/keyholes/documentation/index.html,v $
    $Revision: 1.13 $
    $Date: 2007/10/16 22:34:18 $
    $Author: tsiili $
    $Log: index.html,v $
    Revision 1.13  2007/10/16 22:34:18  tsiili
    Finished the iteration commands, working on the magnetogram handling instructions
    
    Revision 1.12  2007/10/11 22:28:31  tsiili
    Inserted part of changes due to Emily's comments.
    
    Revision 1.11  2007/07/31 22:29:18  tsiili
    Completed table-of-contents (top level headings), Simonin->Olive, some NAME tags, misc editing.
    
    Revision 1.10  2007/07/31 21:02:43  tsiili
    Several modifications in and additions to Subsets and data recorders, Subsets, Keyhole planning processes.
    
    Revision 1.9  2007/05/08 20:55:13  tsiili
    Completed keyhole plan setup phase (with description of the keyhole plot) and started with the iteration section.
    
    Revision 1.8  2007/04/24 21:35:36  tsiili
    Added notes about the pass numbering/identifiers
    
    Revision 1.7  2007/04/18 21:14:12  tsiili
    Completed with Emily's notes
    
    Revision 1.6  2007/04/18 20:22:32  tsiili
    Added Emily's comments
    
    Revision 1.5  2007/04/12 22:03:39  tsiili
    Added some Emily's comments.
    
    Revision 1.4  2007/04/03 19:49:25  tsiili
    Status on 3 Apr 2007
    
    Revision 1.3  2007/03/30 20:34:08  tsiili
    Added CVS keywords also to the top
    
    Revision 1.2  2007/03/30 20:08:03  tsiili
    Imported to CVS and added CVS keywords