https://kdm.wcss.pl/w/index.php?title=Running_Jobs_on_Bem&feed=atom&action=history
Running Jobs on Bem - Historia wersji
2024-03-28T08:30:30Z
Historia wersji tej strony wiki
MediaWiki 1.35.2
https://kdm.wcss.pl/w/index.php?title=Running_Jobs_on_Bem&diff=5714&oldid=prev
Ask o 14:05, 15 maj 2017
2017-05-15T14:05:07Z
<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="pl">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← poprzednia wersja</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Wersja z 14:05, 15 maj 2017</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l5" >Linia 5:</td>
<td colspan="2" class="diff-lineno">Linia 5:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Queues ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Queues ==</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">You </del>do not need to provide the queue name, the system will submit your job to the proper queue based of the walltime parameter </div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">To submit a job you </ins>do not need to provide the queue name, the system will submit your job to the proper queue <ins class="diffchange diffchange-inline">'''</ins>based of the walltime<ins class="diffchange diffchange-inline">''' </ins>parameter </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> qsub -l walltime</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> qsub -l walltime</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l12" >Linia 12:</td>
<td colspan="2" class="diff-lineno">Linia 12:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>To check the parameters of the queue run:</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>To check the parameters of the queue run:</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">> </del>qstat -Qf <del class="diffchange diffchange-inline">queue-name</del></div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"> </ins>qstat -Qf <ins class="diffchange diffchange-inline">queue_name</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>You may submit your job to a dedicated queue '''prace''', but the jobs in <del class="diffchange diffchange-inline">the </del>queue still compete with other jobs for resources. </div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>You may submit your job to a dedicated queue '''prace''', but <ins class="diffchange diffchange-inline">in current configuration </ins>the jobs in <ins class="diffchange diffchange-inline">this </ins>queue still compete with other jobs for resources. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>If you need dedicated resources, in case the cluster is heavy loaded, please [[Helpdesk|contact us]]. We will configure the prace queue with different parameters or priority for your project.</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>If you need dedicated resources, in case the cluster is heavy loaded, please [[Helpdesk|contact us]]. We will configure the prace queue with different parameters or priority for your project.</div></td></tr>
</table>
Ask
https://kdm.wcss.pl/w/index.php?title=Running_Jobs_on_Bem&diff=5713&oldid=prev
Ask o 13:55, 15 maj 2017
2017-05-15T13:55:50Z
<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="pl">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← poprzednia wersja</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Wersja z 13:55, 15 maj 2017</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l3" >Linia 3:</td>
<td colspan="2" class="diff-lineno">Linia 3:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>The [[Bem overview|Bem Cluster]] uses PBS Pro to schedule jobs. Writing a submission script is typically the most convenient way to submit your job to the batch system. Example submission scripts (with explanations) are provided below for a few job types.</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>The [[Bem overview|Bem Cluster]] uses PBS Pro to schedule jobs. Writing a submission script is typically the most convenient way to submit your job to the batch system. Example submission scripts (with explanations) are provided below for a few job types.</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">== Queues ==</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">You do not need to provide the queue name, the system will submit your job to the proper queue based of the walltime parameter </ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"> qsub -l walltime</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">To see all the queues please run:</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"> qstat -q</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">To check the parameters of the queue run:</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">> qstat -Qf queue-name</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">You may submit your job to a dedicated queue '''prace''', but the jobs in the queue still compete with other jobs for resources. </ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">If you need dedicated resources, in case the cluster is heavy loaded, please [[Helpdesk|contact us]]. We will configure the prace queue with different parameters or priority for your project.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Qsub ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Qsub ==</div></td></tr>
</table>
Ask
https://kdm.wcss.pl/w/index.php?title=Running_Jobs_on_Bem&diff=5712&oldid=prev
Jaroslaw.zaklika o 13:52, 15 maj 2017
2017-05-15T13:52:51Z
<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="pl">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← poprzednia wersja</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Wersja z 13:52, 15 maj 2017</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l62" >Linia 62:</td>
<td colspan="2" class="diff-lineno">Linia 62:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><br> User have access to the directories of your tasks <code>/lustre/scratch/tmp/pbs.JobID</code> and your personal directory <code>/lustre/scratch/$USER</code> </div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><br> User have access to the directories of your tasks <code>/lustre/scratch/tmp/pbs.JobID</code> and your personal directory <code>/lustre/scratch/$USER</code> </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''/lustre/scratch/tmp/pbs.JobID''' <br>Directories are created at the start of a task and deleted at the end of the task. <br> Environment variable: $TMPDIR <br> User have the right to write to your directories only in temporary space.</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''/lustre/scratch/tmp/pbs.JobID''' <br>Directories are created at the start of a task and deleted at the end of the task. <br> Environment variable: $TMPDIR <br> User have the right to write to your directories only in temporary space.</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''/lustre/scratch/$USER''' <br> Personal directories on <code>/lustre/scratch</code> are created after the user has made a reasoned request, which requires the administrator to review. Requests for such a directory can be sent to '''kdm @ wcss.pl'''<del class="diffchange diffchange-inline">. </del><br> The contents of these directories are not deleted at the end of the computational task. <br> <del class="diffchange diffchange-inline">You </del>should control the occupancy of these directories so that redundant files do not take up unnecessary space. </div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''/lustre/scratch/$USER''' <br> Personal directories on <code>/lustre/scratch</code> are created after the user has made a reasoned request, which requires the administrator to review. <ins class="diffchange diffchange-inline"><br> </ins>Requests for such a directory can be sent to<ins class="diffchange diffchange-inline">: </ins>'''kdm @ wcss.pl''' <br> The contents of these directories are not deleted at the end of the computational task. <br> <ins class="diffchange diffchange-inline">User </ins>should control the occupancy of these directories so that redundant files do not take up unnecessary space. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><br> This directories would be available only on worker nodes, so if you want to access its contents, please do it in an interactive job.</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><br> This directories would be available only on worker nodes, so if you want to access its contents, please do it in an interactive job.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><small>< [[Bem User Guide]] < Running Jobs on Bem</small></div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><small>< [[Bem User Guide]] < Running Jobs on Bem</small></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>[[Kategoria:User Guide]]</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>[[Kategoria:User Guide]]</div></td></tr>
</table>
Jaroslaw.zaklika
https://kdm.wcss.pl/w/index.php?title=Running_Jobs_on_Bem&diff=5711&oldid=prev
Jaroslaw.zaklika o 13:36, 15 maj 2017
2017-05-15T13:36:00Z
<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="pl">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← poprzednia wersja</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Wersja z 13:36, 15 maj 2017</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l59" >Linia 59:</td>
<td colspan="2" class="diff-lineno">Linia 59:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Scratch directory ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Scratch directory ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline">If your application is </del><code><del class="diffchange diffchange-inline">I</del>/<del class="diffchange diffchange-inline">O</del></code> <del class="diffchange diffchange-inline">intensive you may ask (please write to </del>the <del class="diffchange diffchange-inline">prace-support@</del>wcss.pl<del class="diffchange diffchange-inline">) for setting up </del>your scratch directory <del class="diffchange diffchange-inline">for storing </del>and <del class="diffchange diffchange-inline">accessing </del>the <del class="diffchange diffchange-inline">data during computation</del>:</div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">Directories for temporary files generated by computational programs are in the file system </ins><code>/<ins class="diffchange diffchange-inline">lustre/scratch</ins></code> <ins class="diffchange diffchange-inline">on Bem. <br> The Lustre file system is available from all Bem cluster computing nodes and </ins>the <ins class="diffchange diffchange-inline">archive node archiwum.wcss.pl. <br> System plików Lustre jest dostępny ze wszystkich węzłów obliczeniowych klastra oraz na węźle do archiwizacji archiwum.</ins>wcss.pl<ins class="diffchange diffchange-inline">. <br>Directory backups in temporary space are not executed.</ins></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del class="diffchange diffchange-inline"> </del>/lustre/scratch/$USER</div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"><br> User have access to the directories of </ins>your <ins class="diffchange diffchange-inline">tasks <code>/lustre/</ins>scratch<ins class="diffchange diffchange-inline">/tmp/pbs.JobID</code> and your personal </ins>directory <ins class="diffchange diffchange-inline"><code>/lustre/scratch/$USER</code> </ins></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>This <del class="diffchange diffchange-inline">directory </del>would be available only on worker nodes, so if you want to access its contents, please do it in an interactive job.</div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* '''/lustre/scratch/tmp/pbs.JobID''' <br>Directories are created at the start of a task </ins>and <ins class="diffchange diffchange-inline">deleted at the end of </ins>the <ins class="diffchange diffchange-inline">task. <br> Environment variable</ins>: <ins class="diffchange diffchange-inline">$TMPDIR <br> User have the right to write to your directories only in temporary space.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">* '''</ins>/lustre/scratch/$USER<ins class="diffchange diffchange-inline">''' <br> Personal directories on <code>/lustre/scratch</code> are created after the user has made a reasoned request, which requires the administrator to review. Requests for such a directory can be sent to '''kdm @ wcss.pl'''. <br> The contents of these directories are not deleted at the end of the computational task. <br> You should control the occupancy of these directories so that redundant files do not take up unnecessary space. </ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline"><br> </ins>This <ins class="diffchange diffchange-inline">directories </ins>would be available only on worker nodes, so if you want to access its contents, please do it in an interactive job.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><small>< [[Bem User Guide]] < Running Jobs on Bem</small></div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><small>< [[Bem User Guide]] < Running Jobs on Bem</small></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>[[Kategoria:User Guide]]</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>[[Kategoria:User Guide]]</div></td></tr>
</table>
Jaroslaw.zaklika
https://kdm.wcss.pl/w/index.php?title=Running_Jobs_on_Bem&diff=5710&oldid=prev
Ask o 13:27, 15 maj 2017
2017-05-15T13:27:39Z
<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="pl">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← poprzednia wersja</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Wersja z 13:27, 15 maj 2017</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1" >Linia 1:</td>
<td colspan="2" class="diff-lineno">Linia 1:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>{{Prace-user-guide}}</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>{{Prace-user-guide}}</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><small>< [[Bem User Guide]] < Running Jobs on <del class="diffchange diffchange-inline">Supernova</del></small></div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><small>< [[Bem User Guide]] < Running Jobs on <ins class="diffchange diffchange-inline">Bem</ins></small></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>The [[Bem overview|Bem Cluster]] uses PBS Pro to schedule jobs. Writing a submission script is typically the most convenient way to submit your job to the batch system. Example submission scripts (with explanations) are provided below for a few job types.</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>The [[Bem overview|Bem Cluster]] uses PBS Pro to schedule jobs. Writing a submission script is typically the most convenient way to submit your job to the batch system. Example submission scripts (with explanations) are provided below for a few job types.</div></td></tr>
</table>
Ask
https://kdm.wcss.pl/w/index.php?title=Running_Jobs_on_Bem&diff=5707&oldid=prev
Ask: /* Scratch directory */
2017-05-15T12:52:26Z
<p><span dir="auto"><span class="autocomment">Scratch directory</span></span></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="pl">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← poprzednia wersja</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Wersja z 12:52, 15 maj 2017</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l59" >Linia 59:</td>
<td colspan="2" class="diff-lineno">Linia 59:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Scratch directory ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Scratch directory ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>If your application is <code>I/O</code> intensive <del class="diffchange diffchange-inline">use </del>your scratch directory for storing and accessing the data during computation:</div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>If your application is <code>I/O</code> intensive <ins class="diffchange diffchange-inline">you may ask (please write to the prace-support@wcss.pl) for setting up </ins>your scratch directory for storing and accessing the data during computation:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> /lustre/scratch/$USER</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> /lustre/scratch/$USER</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>This directory <del class="diffchange diffchange-inline">is </del>available only on worker nodes, so if you want to access its contents, please do it in an interactive job.</div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>This directory <ins class="diffchange diffchange-inline">would be </ins>available only on worker nodes, so if you want to access its contents, please do it in an interactive job.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><small>< [[Bem User Guide]] < Running Jobs on Bem</small></div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div><small>< [[Bem User Guide]] < Running Jobs on Bem</small></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>[[Kategoria:User Guide]]</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>[[Kategoria:User Guide]]</div></td></tr>
</table>
Ask
https://kdm.wcss.pl/w/index.php?title=Running_Jobs_on_Bem&diff=5549&oldid=prev
Ask: /* Job submission scripts */
2016-04-05T10:04:56Z
<p><span dir="auto"><span class="autocomment">Job submission scripts</span></span></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="pl">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← poprzednia wersja</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Wersja z 10:04, 5 kwi 2016</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l30" >Linia 30:</td>
<td colspan="2" class="diff-lineno">Linia 30:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>To submit a job you just type:</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>To submit a job you just type:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> qsub test.sh</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> qsub test.sh</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>It submits your <code>test.sh</code> script as a 24 h job (<code>-l walltime=24:00:00</code> with <code>job-name test-job</code> (<code>-N</code> parameter) requesting 1CPU and 100MB RAM in one chunk (<code>-l</code> option). PBS server will notice you on job beginning (<code>-m b</code>) and end (<code>-m e</code>).</div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>It submits your <code>test.sh</code> script as a 24 h job (<code>-l walltime=24:00:00</code><ins class="diffchange diffchange-inline">) </ins>with <code>job-name test-job</code> (<code>-N</code> parameter) requesting 1CPU and 100MB RAM in one chunk (<code>-l</code> option). PBS server will notice you on job beginning (<code>-m b</code>) and end (<code>-m e</code>).</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>After the end of the job it's output will be saved in a file "<code>test-job.o$JOBID</code>". <code>$JOBID</code> is the job number printed by qsub after successful job submission. File "<code>test-job.e$JOBID</code>" will contain <code>STDERR</code>. If your application's <code>STDOUT</code> will exceed 30MB please redirect it to a file in your home directory using standard bash redirection:</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>After the end of the job it's output will be saved in a file "<code>test-job.o$JOBID</code>". <code>$JOBID</code> is the job number printed by qsub after successful job submission. File "<code>test-job.e$JOBID</code>" will contain <code>STDERR</code>. If your application's <code>STDOUT</code> will exceed 30MB please redirect it to a file in your home directory using standard bash redirection:</div></td></tr>
</table>
Ask
https://kdm.wcss.pl/w/index.php?title=Running_Jobs_on_Bem&diff=5324&oldid=prev
F: Utworzono nową stronę "{{Prace-user-guide}} <small>< Bem User Guide < Running Jobs on Supernova</small> The Bem Cluster uses PBS Pro to schedule jobs. Writing a submissio..."
2016-02-22T09:09:21Z
<p>Utworzono nową stronę "{{Prace-user-guide}} <small>< <a href="/wiki/Bem_User_Guide" title="Bem User Guide">Bem User Guide</a> < Running Jobs on Supernova</small> The <a href="/wiki/Bem_overview" title="Bem overview">Bem Cluster</a> uses PBS Pro to schedule jobs. Writing a submissio..."</p>
<p><b>Nowa strona</b></p><div>{{Prace-user-guide}}<br />
<small>< [[Bem User Guide]] < Running Jobs on Supernova</small><br />
<br />
The [[Bem overview|Bem Cluster]] uses PBS Pro to schedule jobs. Writing a submission script is typically the most convenient way to submit your job to the batch system. Example submission scripts (with explanations) are provided below for a few job types.<br />
<br />
== Qsub ==<br />
To run a simple '''interactive job''' with default resources (1CPU, 2000MB RAM) use the <code>-I</code> option and the <code>-l walltime=6:00:00</code> parameter (to set a walltme limit):<br />
qsub -I -l walltime=6:00:00<br />
<br />
Resources are requested by using <code>-l</code> option. The resources are assigned to chunks, and you may request different resources to every chunk. To request interactive job with 2 chunks with 2 CPUs and 300MB of memory plus 3 chunks with 1 CPU and 100MB of memory execute:<br />
qsub -I -l walltime=6:00:00 -l select=2:ncpus=2:mem=300mb+3:ncpus=1:mem=100mb<br />
<br />
Qsub manual pages describes a lot of other interesting options (such as job dependencies, mail events etc). To see manual type:<br />
man qsub<br />
<br />
== Job submission scripts ==<br />
<br />
If you are using the same software many times you probably want to use some scripting.<br />
Qsub allows you to run your application from a script with all options defined inside. The example script "test.sh" defines PBS options, loads prace module and executes some commands:<br />
#!/bin/bash<br />
#PBS -l walltime=24:00:00<br />
#PBS -l select=1:ncpus=1:mem=100MB<br />
#PBS -m be<br />
#PBS -N test-job<br />
<br />
module load prace<br />
cd $dir<br />
pwd<br />
date<br />
To submit a job you just type:<br />
qsub test.sh<br />
It submits your <code>test.sh</code> script as a 24 h job (<code>-l walltime=24:00:00</code> with <code>job-name test-job</code> (<code>-N</code> parameter) requesting 1CPU and 100MB RAM in one chunk (<code>-l</code> option). PBS server will notice you on job beginning (<code>-m b</code>) and end (<code>-m e</code>).<br />
<br />
After the end of the job it's output will be saved in a file "<code>test-job.o$JOBID</code>". <code>$JOBID</code> is the job number printed by qsub after successful job submission. File "<code>test-job.e$JOBID</code>" will contain <code>STDERR</code>. If your application's <code>STDOUT</code> will exceed 30MB please redirect it to a file in your home directory using standard bash redirection:<br />
app_exec_command > output_filename<br />
<br />
We usually write scripts that prepare some environment and then run qsub. The simple sub-test script to change job execution directory to the one the job was submitted from:<br />
#!/bin/bash<br />
<br />
C=$PWD<br />
<br />
cat << EOF | qsub<br />
#!/bin/bash<br />
#PBS -l walltime=24:00:00<br />
#PBS -l select=1:ncpus=1:mem=100MB<br />
#PBS -m be<br />
#PBS -N test-job<br />
<br />
module load prace<br />
cd $C<br />
pwd<br />
EOF<br />
<br />
== Checking status of jobs ==<br />
<br />
To see all your waiting and running jobs type:<br />
qstat -u $USER<br />
<br />
== Scratch directory ==<br />
<br />
If your application is <code>I/O</code> intensive use your scratch directory for storing and accessing the data during computation:<br />
/lustre/scratch/$USER<br />
This directory is available only on worker nodes, so if you want to access its contents, please do it in an interactive job.<br />
<br />
<small>< [[Bem User Guide]] < Running Jobs on Bem</small><br />
[[Kategoria:User Guide]]</div>
F