The Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia’s national science agency, has teamed up with Dell to deliver CSIRO’s newest high performance computing cluster (HPC), named Pearcey. This cluster is named after Australian ICT pioneer Dr. Trevor Pearcey who led the CSIRO project team which built one of the world’s first digital computers.
Pearcey is a Dell HPC designed by CSIRO and Dell that delivers 230 nodes supporting data-intensive research and computational modelling. Its features include:
- Based on Dell PowerEdge 13th generation M630 blade servers, each with 128GB RAM, and four PowerEdge R930 nodes each with 3 TB of memory for large memory applications.
- Servers are connected via 1:1 Mellanox FDR InfiniBand Networking.
- Built with Bright Cluster Manager, enabling a software defined approach to management.
- Sixteen of the PowerEdge M630 blade servers are configured with 512GB RAM and use ScaleMP software connected architecture to create a single, high memory, 8 TB cluster under a single operating system
The leading-edge hardware available at Pearcey cluster accelerates research process and delivers research impacts to customer within shorter time with the powerful computer simulations.
Interview with Dr. Dayalan Gunasegaram, CSIRO
- Could you briefly share with us the research activities that utilise HPC facilities of Pearcey?
On the broad level, the new HPC cluster called Pearcey, developed through the collaboration between Commonwealth Scientific and Industrial Research Organisation (CSIRO) and Dell, supports research activities in a broad range of areas, such as Bioinformatics, Fluid Dynamics and Materials Science. More specifically, I am leveraging this technology to develop an improved nylon mesh for use in pelvic organ prolapse surgery which could benefit the one-in-five Australian women who have surgery for the condition at some point in their lives.
- What are the benefits that you think Pearcey can deliver in the biomedical / healthcare industry?
Pearcey can be used to analyse complex mathematical models more quickly and accurately than used to be possible in the past. Simulations and models of real life scenarios, such as stressful forces on a patient and medical implants can lead to the design of better implants, for example.
One example of a use case is the work that I’m doing with researchers from Monash University on the development of an improved nylon mesh for supporting prolapsed organs in pelvic organ prolapse (POP) surgery. One in ten women worldwide will suffer a pelvic organ prolapse (POP), and in Australia, one in five will require surgery. Some treatments integrate synthetic meshes into the tissue, but this can result in infection and damage to the tissue wall. Up to one-third of POP surgeries can have complications.
Our improved mesh, made of custom polymers, must behave as human tissue. Designing a synthetic fabric that emulates the body at such a delicate scale requires immense numbers of calculations and analyses too great for typical computers.
With the high performance computing capability of the Pearcey cluster we can simulate the stressful forces a patient would experience, such as during coughing or running, and model a variety of different situations to assess the mesh under stress. Current mesh implants can lead to pain and discomfort; the results from Pearcey will provide more insights on how to improve the mesh so that the condition can be treated more effectively.
With thousands of processor cores available, Pearcey allows us to do more finite element analysis computations within a given time, which means we're able to advance to the next stage of testing much sooner than if we didn't have something so powerful.
The computer simulations allow us to better understand the cause-and-effect relationships between mesh parameters, such as pore size and their expected in-vivo performance after implantation, and really focus on the areas with the most influence.
The Pearcey cluster is used for many other simulations as well, including molecular dynamics (MD) simulations which also require a huge amount of computing power. As these simulate the physical movements of atoms and molecules, a large number of these entities need to be taken into account even to study a small volume of matter. The availability of the cluster enables CSIRO’s computational materials scientists to study materials in the virtual world and optimise the composition of materials even before any physical experiments are planned by the experimental scientists. This cuts time and cost from materials development. In one example, the CSIRO corrosion team is developing corrosion inhibitors based on MD simulations where they insert various atoms in different configurations into a potential recipe for an inhibitor to study how this would affect the corrosion performance.
- How accurate would the computer simulations be able to generate mimicking the patients’ experience under stressful conditions, compared to physical testing?
The simulations are generated based on data we input from patients records and feedback we receive from them during consultations or treatments. Thus they are realistic in mimicking patients’ experience, and as such enable us to reduce the amount of physical testing we need to carry out on animals and humans, and the associated ethical and financial costs.
- What other applications that CSIRO scientists use Dell supercomputers for?
High performance computing facilities such as Pearcey are an integral part of the advanced information and communications technologies that enable CSIRO science to solve real issues.
Pearcey is one of the most widely and heavily used pieces of research infrastructure in CSIRO and is helping our scientists tackle problems facing Australia and the planet in many diverse areas – everything from childbirth to oceans, energy, minerals, stars, flora, fauna, manufacturing, aerospace and agriculture.
Interview with Andrew Underwood, Dell
- CSIRO and Dell had teamed up to build a HPC cluster solution, Pearcey to support data driven research activities in CSIRO. When and how did the collaboration between CSIRO and Dell start?
Dell has been working closely with the Australian research community to assist them with accelerating their scientific research via the use of Dell technologies, over the past number of years. Through these activities we’ve built a strong partnership with CSIRO, as their leadership as the national science agency of Australia has provided Dell and our customers with a strong blueprint for technical innovation.
More recently we’ve engaged directly with the CSIRO, to undertake a collaborative approach to designing their next-generation HPC system, with the goal to build a platform that would reaffirm CSIRO’s position as Australia’s innovation catalyst.
The new HPC system, named Pearcey, is designed to power through data and compute intensive research at a pace far greater than other HPC systems and addresses the growing demands of data.
- Are there any challenges in the process of developing HPC for Pearcey?
As the CSIRO is a catalyst for Australian innovation, the workloads that researchers run on the Pearcey HPC system can be very demanding and are somewhat unknown by nature.
By partnering with CSIRO Information Management & Technology (IM&T) team, we were able to leverage their deep workload expertise to architect a solution that would exceed historical scientific data growth patterns over the life of the system.
This partnership was critical in addressing the technical, business, and scientific requirements and potential challenges that were driven by CSIRO researchers, and ensured that the tightly-coupled system was complementary to the vision of the Organisation.
- What are the steps that Dell carried out to make the collaboration a successful one?
Dell believes in building a deep relationship with every customer, not just for the challenges at hand, but for the long term. CSIRO is a National Computational Infrastructure (NCI) collaborator, and was already familiar with Dell’s HPC expertise in building a HPC cloud for the NCI.
For the Pearcey project, the Dell team took the time to understand CSIRO’s pain points thoroughly before proposing a HPC cluster solution that would be easy to implement, manage and maintain, with optimum scalability, reliability, and availability. Dell also provided proofs of concept and product trials to show what Dell’s proposed solution could achieve, and arranged for CSIRO to meet with key reference customers with similar implementations to answer queries.
During the implementation stage, Dell monitored project activities closely and ensured that project milestones were met despite tight deadlines. Dell still maintains contact with CSIRO today to ensure that Pearcey continues to run smoothly and handle all computational tasks efficiently.
Most importantly we ensured that once operational, the Pearcey HPC system would achieve all the goals that Dell and CSIRO set out to achieve.