Difference between revisions of "Checkpoint and Restarting"

From Earlham CS Department
Jump to navigation Jump to search
Line 6: Line 6:
 
</li></ol>
 
</li></ol>
  
<h2>Signals to <tt>mdrun</tt> and their relevance to the checkpoint/restart process</h2>
+
<h2>Signal accepted by <tt>mdrun</tt> and their relevance to the checkpoint/restart process</h2>
  
 
The <tt>mdrun</tt> process accepts SIGTERM and SIGUSR1.  These signals can be received by a <tt>mdrun</tt> process of any rank.  The effects of the signals are as follows:
 
The <tt>mdrun</tt> process accepts SIGTERM and SIGUSR1.  These signals can be received by a <tt>mdrun</tt> process of any rank.  The effects of the signals are as follows:
Line 19: Line 19:
 
</ul></li>
 
</ul></li>
 
</ul>
 
</ul>
 +
 +
Neither of these signals provide useful checkpointing mechanism.  With some modicition (remove the code that modifies <tt>nsteps</tt>), the SIGUSR1 mechanism could be useful.  Note that we are already using SIGINT in nanny/child communication and SIGINT is our only free signal due to COSM.

Revision as of 16:18, 1 June 2005

Checkpoint Frequency

Checkpointing in Folding@Clusters has two distinct parts:

  1. mdrun generating checkpoints in an interval given by the nstxout parameter in grompp.mdp.
  2. The nanny checks the size of the checkpoint file. If the file has become larger, it is transferred to the mother. This check happens about every two seconds (as of 1 June 2005).

Signal accepted by mdrun and their relevance to the checkpoint/restart process

The mdrun process accepts SIGTERM and SIGUSR1. These signals can be received by a mdrun process of any rank. The effects of the signals are as follows:

  • SIGTERM
    • Sets nsteps to current steps plus one.
  • SIGUSR1
    • Sets nsteps the next multiple of nstxout past the current step.

Neither of these signals provide useful checkpointing mechanism. With some modicition (remove the code that modifies nsteps), the SIGUSR1 mechanism could be useful. Note that we are already using SIGINT in nanny/child communication and SIGINT is our only free signal due to COSM.