Compiler optimisations with initial notes on working with maqao

core/compiler-optimisations.tex

@@ -1,74 +1,2 @@
 \section{Compiler optimisations}
 \label{section:core:compiler-optimisations}
-
-Before we continue, it is important to double-check whether we have used the correct
-compiler settings. It makes no sense to benchmark a code which is not producing
-tailored code for our particularly machinery (therefore it is important to
-benchmark on the right system), and it makes no sense to benchmark a code that
-is not using the latest compiler features.
-
-
-First of all, we work with the the most aggressive optimisation that our
-compiler can offer. The minimum choice is to compile with \texttt{-O3},
-although some other compilers might offer further granularity. Also consider
-using \texttt{-ffast-math} which enables aggressive floating-point optimisation.
-
-
-Second, we create code that is tailored towards our particular machinery.
-The correct setting for this depends once again on your compiler. 
-With LLVM and GNU, you can use flags starting with \texttt{-mXXXX} to specify 
-machinery-specific optimisations, i.e.~you can exactly say for which processor
-generation, instruction set, \ldots~you want for the compiler to create binary code.
-If your compile node is of the same type as your test node, then simply
-use \texttt{-xhost} and the compiler will pick the highest instruction set
-available.
-The flag is called \texttt{-mhost} with GNU.
-
-
-\begin{wip}
-A question which was raised at RSECon25: when applying aggressive optimisation, do we include validation checks on the code? I.e., to ensure that the validity of the code still passess when potentially using approximations in the higher level optimisation flags.
-\end{wip}
-
-\begin{instructions}{\intelcompiler}
- With the Intel compiler, you sometimes can pick between the generic
- LLVM (community) versions of a feature and one optimised by Intel itself (with
- the latter obviously performing better on bespoke Intel architecture). For
- example, \texttt{-fopenmp} enables the general OpenMP implementation whereas
- \texttt{-fiopenmp} gives you an in-house version of the same. 
-
- And while \texttt{-xhost} uses Intel-specific optimisations, \texttt{-mhost}
- will use LLVM's optimisation stack, i.e.~Intel has replaced some optimisation
- steps with bespoke variants, which are enabled if and only if you go down the
- \texttt{-xhost} route. Be aware that Intel's optimisation passes typically will
- be disabled if you use the Intel toolchain on other vendor's hardware,
- i.e.~you'll get unoptimised code.
-\end{instructions}
-
-\begin{checklist}
- \item Your benchmark is compiled with the highest optimisation level (\texttt{-O3} plus a hardware-specific instruction set such as picked by \texttt{-xhost}).
-\end{checklist}
-
-\begin{instructions}{\maqao}
- If you run \maqao~over your code, the tool will report back to you what
- optimisations you might have missed out throughout the compilation.
- The top-level rubric \emph{Global} provides you with information on the optimisation chosen, but it also makes recommendations which options might lead to better runtime.
-
- The nice thing about \maqao~is that the tool can handle situations where you translate different translation units with different options.
- Also, the recommendation aspect goes beyond pure reporting and is helpful.
-\end{instructions}
-
-
-
-\paragraph*{How it works}
-
-\begin{itemize}[noitemsep]
-  \item[$\boxplus$] Use the most aggressive compiler optimisation that preserve the code semantics (aka still return valid results) and nevertheless are tailored towards your assessment machinery.
-  \item[$\boxplus$] Check what optimisations the users' build system uses by default. Often, this is a low-hanging fruit.
-\end{itemize}
-
-
-\paragraph*{How it does not work}
-
-\begin{itemize}[noitemsep]
-  \item[$\boxminus$] Use a low optimisation level and machine-generic optimisations and then draw conclusions on code behaviour on a particular machine. 
-\end{itemize}

tailoring/core.tex

@@ -1,2 +1,93 @@
 \section{Core performance}
 \label{section:tailoring:core}
+
+Before we continue, it is important to double-check whether we have used the correct
+compiler settings. It makes no sense to benchmark a code which is not producing
+tailored code for our particularly machinery (therefore it is important to
+benchmark on the right system), and it makes no sense to benchmark a code that
+is not using the latest compiler features.
+
+
+First of all, we work with the the most aggressive optimisation that our
+compiler can offer. The minimum choice is to compile with \texttt{-O3},
+although some other compilers might offer further granularity. Also consider
+using \texttt{-ffast-math} which enables aggressive floating-point
+optimisation. Compiler documentation can be extensive, but there are often dedicated
+pages which list optimisation flags.
+
+
+Second, we create code that is tailored towards our particular machinery.
+The correct setting for this depends once again on your compiler. 
+With LLVM and GNU, you can use flags starting with \texttt{-mXXXX} to specify 
+machinery-specific optimisations, i.e.~you can exactly say for which processor
+generation, instruction set, \ldots~you want for the compiler to create binary code. Often we can tailor to the hardware with flags such as \texttt{-march=native}, with will automatically pickup the local architecture.
+If your compile node is of the same type as your test node, then simply
+use \texttt{-xhost} and the compiler will pick the highest instruction set
+available.
+The flag is called \texttt{-mhost} with GNU.
+
+\begin{instructions}{\intelcompiler}
+ With the Intel compiler, you sometimes can pick between the generic
+ LLVM (community) versions of a feature and one optimised by Intel itself (with
+ the latter obviously performing better on bespoke Intel architecture). For
+ example, \texttt{-fopenmp} enables the general OpenMP implementation whereas
+ \texttt{-fiopenmp} gives you an in-house version of the same. 
+
+ And while \texttt{-xhost} uses Intel-specific optimisations, \texttt{-mhost}
+ will use LLVM's optimisation stack, i.e.~Intel has replaced some optimisation
+ steps with bespoke variants, which are enabled if and only if you go down the
+ \texttt{-xhost} route. Be aware that Intel's optimisation passes typically will
+ be disabled if you use the Intel toolchain on other vendor's hardware,
+ i.e.~you'll get unoptimised code.
+\end{instructions}
+
+\begin{checklist}
+ \item Your benchmark is compiled with the highest optimisation level (\texttt{-O3} plus a hardware-specific instruction set such as picked by \texttt{-xhost}).
+\end{checklist}
+
+\begin{instructions}{\maqao}
+ If you run \maqao~over your code, the tool will report back to you what
+ optimisations you might have missed out throughout the compilation.
+  The top-level rubric \emph{Global} provides you with information on the optimisation chosen, but it also makes recommendations which options might lead to better runtime.
+
+ The nice thing about \maqao~is that the tool can handle situations where you translate different translation units with different options.
+ Also, the recommendation aspect goes beyond pure reporting and is helpful.
+\end{instructions}
+
+Not all compiler optimisations lead to decreases in runtime. With
+tools like MAQAO it can be beneficial to \emph{experiment} with compiler
+optimisations. For example, by default the GNU compilers do not implement loop
+unrolling, and so MAQAO may initially suggest to switch on the
+\texttt{-funroll-loops} flag. However, it is not a given that this will speed
+up the code, and so we can the benchmark through MAQAO, with and without this
+flag, to see if there is noticable differences in the total runtime, and the
+runtimes of individual loops.
+
+Some optimisations can implement aggressive optimisations. Many algorithms may
+be robust to, for example, floating point approximations but it is worth
+checking. If the code comes with test cases, or has known results which can be
+checked against then it is very much advised to run these as validation checks.
+Test the code with and without these aggressive optimisations to ensure that
+these compiler flags have not noticably changed the code outcome. 
+
+We recommend, if the analyst settles on using further compiler optimisations,
+then these should be clearly documented in the performance report.  This is to
+ensure that the code owner is aware of any aggressive optimisations that the
+analyst uses, and the code owner makes any final decisions, or undertakes any
+final checks, to ensure that the optimisations have not impacted the code
+results.
+
+\paragraph*{How it works}
+
+\begin{itemize}[noitemsep]
+  \item[$\boxplus$] Use the most aggressive compiler optimisation that preserve the code semantics (aka still return valid results) and nevertheless are tailored towards your assessment machinery.
+  \item[$\boxplus$] Check what optimisations the users' build system uses by default. Often, this is a low-hanging fruit.
+\end{itemize}
+
+
+\paragraph*{How it does not work}
+
+\begin{itemize}[noitemsep]
+  \item[$\boxminus$] Use a low optimisation level and machine-generic optimisations and then draw conclusions on code behaviour on a particular machine.
+  \item[$\boxminus$] Use aggresive mathematical approximations without validating correctness of the results.
+\end{itemize}

tools/maqao.tex

@@ -7,3 +7,67 @@ \section{MAQAO}
 Which loops consume most runtime, vectorise poorly or are not balanced reasonably?
 That's the type of prime application area of \maqao.
 
+The high-level report is called OneView and it composes the different MAQAO
+components such as the Code Quality Analyser (CQA) and the Lightweight Profiler
+(LProf). One way of outputting the results of MAQAO is into an HTML file,
+\texttt{index.html}, which can be viewed in a browsers and views a GUI that the
+analyst can navigate. We here run through some of the ways of working with
+MAQAO to aid compiler optimisation, performance analysis of loops, and load
+balancing across threads and ranks. As with all tools, we refer the reader to
+the MAQAO documentation for further information
+\footnote{\url{https://www.maqao.org}}.
+
+\subsection{Setup}
+
+The MAQAO website offer binaries for x86 and ARM architectures that a
+performance analyst can simply download locally to a cluster. The only other
+step needed in setup is to compile the code with debug flags, \texttt{-g},
+switched on. This is common to many profilers to allow the profiler to list the
+correspondences between the binary and source code. In some cases it can be
+necessary to also use the \texttt{-grecord-gcc-switches} flag.
+
+\subsection{Generating results}
+
+To run a serial code through MAQAO we can use
+\begin{verbatim}
+maqao OV -R1 -- ./code
+\end{verbatim}
+Where we are here listing to produce a report through OneView (OV). We can also
+specify OpenMP threads through an environment variable
+\begin{verbatim}
+maqao OV -R1 --envv_OMP_NUM_THREADS=4 -- ./code
+\end{verbatim}
+or even with MPI processes as 
+\begin{verbatim}
+maqao OV -R1 --number-processes \
+--mpi_command="mpirun -n <number_processes>" -- ./code
+\end{verbatim}
+and these can also be combined to run a hybrid MPI+OpenMP application.
+
+As discussed with Score-P in Section~\ref{section:localisation:scorep},
+experiment directories are useful way of keeping track of different runs for
+different analyses. MAQAO has a model which can often require frequent reruns
+to try out different compiler optimisations or loop configurations, and so
+experiment directory names are useful for this iterative workflow. An
+experiment directory can be named via the \texttt{-xp=...} flag.
+
+Once a set of experiment directories have been generated, it can be useful to
+view a comparison of the experiments by combining them into an overview report
+\begin{verbatim}
+maqao OV --compare-reports --inputs=exp_dir1,exp_dir2,...
+\end{verbatim}
+As with the single OneView reports, this combined report can be viewed in a
+browser.
+
+Finally, if multiple experiments are planned then it can be useful to create
+configuration scripts, e.g., to produce a scaling analysis. An initial
+configuration file can be generated via
+\begin{verbatim}
+maqao OV --create-config=initial_config.json
+\end{verbatim}
+and then tailoring this configuration to the specific analyses.
+
+\subsection{Navigating and interpreting results}
+
+% here we can populate with some information on navigating information in the
+% MAQAO GUI