1   Resources

2   Types of Access

   2.1   Preparatory Access
      2.1.1   Requirements
      2.1.2   Detailed information
   2.2   Project Access
      2.2.1   Requirements
      2.2.2   Detailed information
   2.3   Programmatic access

3   Assessment of projects

   3.1   Technical assessment
   3.2   Scientific assesment
   3.3   Useful Hints
      3.3.1   Technical Aspects
      3.3.2   Scientific Aspects

4   Allocation

This document is intended as a first overview and quick checklist for PRACE applications. For definitive information, please consult the PRACE web site.

1 Resources

As of 2014, six Tier-0 installations are being operated by 4 countries. Actual availability of these resources varies per call. System types are differentiated by

  • Processor type (X86, Power PC)
  • Number of cores (16-128) and memory (16-512GB) per node
  • Availability and type of accelerators (none, Nvidia, Xeon Phi)
  • Type (Infiniband, proprietary) and topology (Fat tree, fat tree with islands, 3D-5D Torus) of network

Overview (table headers link to installation web sites)

Curie Fermi Hornet MareNostrum SuperMUC
FN TN HN Fermi Hornet Standard Hybrid TN FN (*)
Country France France France Italy Germany Spain Spain Germany Germany
System Type Bullx Bullx Bullx Blue Gene/Q Cray XC30 IBM System x iDataPlex IBM System x iDataPlex IBM System x iDataPlex IBM BladeCenter HX5
Compute Processor type Intel Nehalem EX 2,27 Ghz Intel SandyBridg e EP 2,7 Ghz Intel Westmere EP 2,67 Ghz IBM PowerPC A2 (1,6 GHz) 16 cores/node Intel Xeon E5-2680v3 (Haswell) Intel Sandy Bridge EP Intel Sandy Bridge EP Intel Sandy Bridge EP Intel Westmere EX
Total number of nodes 90 5040 144 10240 3944 3028 42 9216 205
Number of cores per node 128 16 8 16 24 16 16 16 40
Total number of cores 11520 80640 1152 163840 94656 48448 672 147456 8200
Number of accelerators / node - - 2 - - - 2 - -
Type of accelerator - - Nvidia M2090 - - - Intel Xeon Phi 5110P - -
Memory Memory / Node 512 GB 64 GB 24 GB 16 GB 128GB 32 64 GB 32 GB 256 GB
Network Network Type Infiniband QDR 10 Infiniband QDR 10 Infiniband QDR 10 IBM Custom Cray Aries Infiniband FDR10 Infiniband FDR10 Infiniband FDR10 Infiniband QDR
Connectivity Fat tree Fat tree Fat tree 5D Torus Dragonfly Fat Tree Fat Tree Fat tree within island (8192 cores), pruned tree between islands Fat Tree
capacity 8 TB 8 TB 8 TB 100 TB 60 TB 59 TB 59 TB 1,5 PB 1,5 PB
Work file system type Lustre Lustre Lustre GPFS Lustre GPFS GPFS GPFS GPFS
capacity 600 TB 600 TB 600 TB 2PB 7 PB 612 TB 612 TB 7 PB 7 PB
Scratch file system type Lustre Lustre Lustre GPFS - GPFS GPFS GPFS GPFS
capacity 3,4 PB 3,4 PB 3,4 PB 1 PB - 1,1 PB 1,1 PB 3 PB 3 PB
Archive capacity Unlimited Unlimited Unlimited On demand On demand 2,4 PB 2,4 PB 30 PB On demand 30 PB On demand
Minimum required job size Number of cores 128 512 32 2048 2048 1024 16 512 512

(*) Only for pre-/post-processing purposes. Please verify with Gauss@LRZ for large computing resource requests on FN.

2 Types of Access

PRACE resources can be requested for

  • Preparatory Access (code scaling, optimization),
  • Project Access (large-scale, computationally intensive projects), and
  • Programmatic Access.

2.1 Preparatory Access

Purpose: prepare code for production runs (development, optimization, scaling)

Time window: Rolling call, no closing date, quarterly cut-off dates.

Maximum CPU allocation: typically 50000 - 250000 core hours depending on type of access and system.

  • Type A: code scalability tests - max 2 months
  • Type B: code development and optimization - max 6 months
  • Type C: code development and optimization with support by PRACE experts - max 6 months

2.1.1 Requirements

  • Detailed Work plan mandatory
  • No Production Runs
  • Aim is good scaling of code (min 1024c)
  • Avoid job farming; if jobs farming is necessary: provide job bundling
  • Statement about I/O (parallel I/O?)
  • Contact the ZID before submission

2.1.2 Detailed information

2.2 Project Access

Purpose: large scale scientific production runs using codes that have been tested and demonstrate high efficiency and scalability.

It is strongly recommended to start with preparatory access before applying for project access.

Time window: semi-annual calls for proposals (February, September) open for one month, access starts 6 months later

Time allocation: typically more than 3 million core-hours; total available capacity ranging from 200000 (hybrid systems) or 70 to 200 million (conventional systems) core-hours per installation and call.

  • Duration max 1 year; multi-year projects are possible if need for resource allocation is demonstrated.
  • Preparatory access recommended but not mandatory if scalability and efficiency has been previously demonstrated.

2.2.1 Requirements

In addition to filling in the request form, you must supply the following information:

  • Description of research project (including discussion of scientific scientific questions and goals, novelty, impact and timeliness of proposal.
  • Your experience using HPC systems, detailed Work plan (how resources and staff are managed)
  • Numerical methods and algorithms
  • Need for Tier-0 system
  • Suitability and fitness of desired machine
  • Justification for necessary number of CPU-hours.

2.2.2 Detailed information

2.3 Programmatic access

Purpose: stable and reliable access for particularly large-scale long-term projects of exceptional scientific quality and broad European importance.

3 Assessment of projects

Each scientific project application undergoes two phases of assessment (preparatory access will only be technically assessed).

3.1 Technical assessment

Applicants are expected to select suitable resources based on the characteristics of their code. For regular access, the following criteria must be met:

  • The need to use a PRACE resource
  • Software availability, demonstration of efficiency, scaling and suitability (pre-installed software, tests of software developed by applicant)
  • Feasibility, suitability of project for selected system.

3.2 Scientific assesment

Proposals must address the following criteria:

  • Scientific excellence, impact of proposed research
  • Novelty, transformative qualities (development of important and relevant scientific topic, expected advances)
  • Relevance to call (if scope is stated in the call)
  • Methodology (description and appropriateness of methods to achieve project goals)
  • Dissemination (planned channels and resources for dissemination of results, list of recent publications relevant to proposed project)
  • Management (demonstration of solid management structure, ensuring successful project completion)

3.3 Useful Hints

3.3.1 Technical Aspects

  • Target large scaling (at least 8000 cores)
  • I/O Strategy has to be convincing (not millions of files)
  • Provide restart or snapshot capabilities
  • Max. Runtime is 48 h
  • Max #jobs is 10
  • Be sure to show that you need max 1.6 GB/c
  • LRZ: min 3 million cpuh, max 170 million cpuh

3.3.2 Scientific Aspects

Scientific reviewers will asses your project using the Reviewers peer review form. Keep this in mind when preparing your application.

  • Include description of objects under study (e.g. materials, problem size, parameters...) and detailed motivation for planned investigations.
  • Include your previous publications on the same topic.
  • Describe dissemination of results, in particular if external funding might interfere with free publication.

4 Allocation

Resources will be allocated to successful projects and have to be used within the given time frame. Application for extensions or multi-year projects are possible.