Limits...
Evaluating the performance of parallel subsurface simulators: An illustrative example with PFLOTRAN.

Hammond GE, Lichtner PC, Mills RT - Water Resour Res (2014)

Bottom Line: [1] To better inform the subsurface scientist on the expected performance of parallel simulators, this work investigates performance of the reactive multiphase flow and multicomponent biogeochemical transport code PFLOTRAN as it is applied to several realistic modeling scenarios run on the Jaguar supercomputer.PFLOTRAN scales well (with regard to strong scaling) for three realistic problem scenarios: (1) in situ leaching of copper from a mineral ore deposit within a 5-spot flow regime, (2) transient flow and solute transport within a regional doublet, and (3) a real-world problem involving uranium surface complexation within a heterogeneous and extremely dynamic variably saturated flow field.Weak scalability is discussed in detail for the regional doublet problem, and several difficulties with its interpretation are noted.

View Article: PubMed Central - PubMed

Affiliation: Applied Systems Analysis and Research, Sandia National Laboratories Albuquerque, New Mexico, USA.

ABSTRACT

[1] To better inform the subsurface scientist on the expected performance of parallel simulators, this work investigates performance of the reactive multiphase flow and multicomponent biogeochemical transport code PFLOTRAN as it is applied to several realistic modeling scenarios run on the Jaguar supercomputer. After a brief introduction to the code's parallel layout and code design, PFLOTRAN's parallel performance (measured through strong and weak scalability analyses) is evaluated in the context of conceptual model layout, software and algorithmic design, and known hardware limitations. PFLOTRAN scales well (with regard to strong scaling) for three realistic problem scenarios: (1) in situ leaching of copper from a mineral ore deposit within a 5-spot flow regime, (2) transient flow and solute transport within a regional doublet, and (3) a real-world problem involving uranium surface complexation within a heterogeneous and extremely dynamic variably saturated flow field. Weak scalability is discussed in detail for the regional doublet problem, and several difficulties with its interpretation are noted.

No MeSH data available.


Related in: MedlinePlus

PFLOTRAN wall-clock time for the initialization stage (Init Stage) of the IFRC problem scenario where the initialization time is divided into time spent in Restart and reading HDF5 files and initial/boundary conditions (Reading Condition Data). Note that up to 1024 processes, within the realm of realistic problem size per process for good scalability from the flow and transport code, the initialization stage scales well (i.e., it remains small, at or below 13 s).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4257577&req=5

fig12: PFLOTRAN wall-clock time for the initialization stage (Init Stage) of the IFRC problem scenario where the initialization time is divided into time spent in Restart and reading HDF5 files and initial/boundary conditions (Reading Condition Data). Note that up to 1024 processes, within the realm of realistic problem size per process for good scalability from the flow and transport code, the initialization stage scales well (i.e., it remains small, at or below 13 s).

Mentions: [67]As discussed earlier, the time spent in initialization and I/O (i.e., Other) for the copper leaching and regional doublet scenarios was negligible. However, this was not the case with the IFRC scenario. The major contributors to Other were time spent in initialization and checkpointing. The initialization stage of the code can be broken down into time spent restarting the simulation (i.e., reading checkpointed solution), reading boundary and initial condition data, and reading HDF5 files containing material ids, permeabilities, initial pressures, and initial concentrations. Note that initial conditions (i.e., pressure, concentration, etc.) are still read when restarting a simulation, though they are overwritten. A breakdown of the time spent in initialization is shown in Figure 12 where Init Stage, the total time spent in initialization, is divided into the three major contributors: Restart, Reading Condition Data, and HDF5. Below 1024 processes, the performance of initialization stage scales well (i.e., the time spent mainly in I/O remains small at s). However, this figure clearly illustrates the presence of I/O contention at larger process counts. It should be noted that the initialization time varied greatly between jobs on Jaguar suggesting that other jobs running on the supercomputer may contribute to the I/O contention, a known issue on Jaguar. The performance of checkpointing at the end of the simulation (the other major contributor to Other in Figure 10) was nearly identical to Restart. The use of PFLOTRAN's two-stage I/O capability would reduce the time spent in HDF5, but the capability is not yet supported with checkpoint/restart.


Evaluating the performance of parallel subsurface simulators: An illustrative example with PFLOTRAN.

Hammond GE, Lichtner PC, Mills RT - Water Resour Res (2014)

PFLOTRAN wall-clock time for the initialization stage (Init Stage) of the IFRC problem scenario where the initialization time is divided into time spent in Restart and reading HDF5 files and initial/boundary conditions (Reading Condition Data). Note that up to 1024 processes, within the realm of realistic problem size per process for good scalability from the flow and transport code, the initialization stage scales well (i.e., it remains small, at or below 13 s).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4257577&req=5

fig12: PFLOTRAN wall-clock time for the initialization stage (Init Stage) of the IFRC problem scenario where the initialization time is divided into time spent in Restart and reading HDF5 files and initial/boundary conditions (Reading Condition Data). Note that up to 1024 processes, within the realm of realistic problem size per process for good scalability from the flow and transport code, the initialization stage scales well (i.e., it remains small, at or below 13 s).
Mentions: [67]As discussed earlier, the time spent in initialization and I/O (i.e., Other) for the copper leaching and regional doublet scenarios was negligible. However, this was not the case with the IFRC scenario. The major contributors to Other were time spent in initialization and checkpointing. The initialization stage of the code can be broken down into time spent restarting the simulation (i.e., reading checkpointed solution), reading boundary and initial condition data, and reading HDF5 files containing material ids, permeabilities, initial pressures, and initial concentrations. Note that initial conditions (i.e., pressure, concentration, etc.) are still read when restarting a simulation, though they are overwritten. A breakdown of the time spent in initialization is shown in Figure 12 where Init Stage, the total time spent in initialization, is divided into the three major contributors: Restart, Reading Condition Data, and HDF5. Below 1024 processes, the performance of initialization stage scales well (i.e., the time spent mainly in I/O remains small at s). However, this figure clearly illustrates the presence of I/O contention at larger process counts. It should be noted that the initialization time varied greatly between jobs on Jaguar suggesting that other jobs running on the supercomputer may contribute to the I/O contention, a known issue on Jaguar. The performance of checkpointing at the end of the simulation (the other major contributor to Other in Figure 10) was nearly identical to Restart. The use of PFLOTRAN's two-stage I/O capability would reduce the time spent in HDF5, but the capability is not yet supported with checkpoint/restart.

Bottom Line: [1] To better inform the subsurface scientist on the expected performance of parallel simulators, this work investigates performance of the reactive multiphase flow and multicomponent biogeochemical transport code PFLOTRAN as it is applied to several realistic modeling scenarios run on the Jaguar supercomputer.PFLOTRAN scales well (with regard to strong scaling) for three realistic problem scenarios: (1) in situ leaching of copper from a mineral ore deposit within a 5-spot flow regime, (2) transient flow and solute transport within a regional doublet, and (3) a real-world problem involving uranium surface complexation within a heterogeneous and extremely dynamic variably saturated flow field.Weak scalability is discussed in detail for the regional doublet problem, and several difficulties with its interpretation are noted.

View Article: PubMed Central - PubMed

Affiliation: Applied Systems Analysis and Research, Sandia National Laboratories Albuquerque, New Mexico, USA.

ABSTRACT

[1] To better inform the subsurface scientist on the expected performance of parallel simulators, this work investigates performance of the reactive multiphase flow and multicomponent biogeochemical transport code PFLOTRAN as it is applied to several realistic modeling scenarios run on the Jaguar supercomputer. After a brief introduction to the code's parallel layout and code design, PFLOTRAN's parallel performance (measured through strong and weak scalability analyses) is evaluated in the context of conceptual model layout, software and algorithmic design, and known hardware limitations. PFLOTRAN scales well (with regard to strong scaling) for three realistic problem scenarios: (1) in situ leaching of copper from a mineral ore deposit within a 5-spot flow regime, (2) transient flow and solute transport within a regional doublet, and (3) a real-world problem involving uranium surface complexation within a heterogeneous and extremely dynamic variably saturated flow field. Weak scalability is discussed in detail for the regional doublet problem, and several difficulties with its interpretation are noted.

No MeSH data available.


Related in: MedlinePlus