\section{Parallelization, optimizations and graphics processing units}
\label{sec:parallelization}

Computational cost has been and still is a fundamental factor in the
development of electronic structure methods, as the small spatial
dimensions and the fast movement of electrons severely limit the
size of systems that can be simulated. In order to study systems of
interest as realistically and accurately as possible, electronic-structure codes must execute efficiently in modern computational
platforms. This implies support for massively parallel platforms and
modern parallel processors, including graphics processing units
(GPUs). 

Octopus has been shown to perform efficiently on parallel
supercomputers, scaling to hundreds of thousands of cores~\cite{Andrade_2012,Alberdi_2014}. The code also has an implementation of GPU acceleration~\cite{Andrade_2012_gpus,Andrade_2012} that has shown to be competitive in performance with Gaussian DFT running on GPUs~\cite{Andrade_2013}.

Performance is not only important for established methods, but also
for the implementation of new ideas. The simplicity of real-space grids allows us to provide Octopus developers with building blocks that they can use to produce
highly efficient code without needing to know the details of the
implementation, isolating them as much as possible from the optimization and parallelization requirements. In most cases, these building blocks allow developers to write
code that is automatically parallel, efficient, and that can transparently
run  on GPUs. The type of operations available run from simple ones, like
integration, linear algebra, and differential operators, to more
sophisticated ones, like the application of a Hamiltonian or solvers for differential equations.

However, it is critical to expose an interface with the adequate level
that hides the performance details, while still giving enough
flexibility to the developers. For example, we have found that the
traditional picture of a state as the basic object is not adequate for
optimal performance, as it does not expose enough data
parallelism~\cite{Andrade_2012_gpus}. In Octopus we use a higher-level
interface where the basic object is a group of several states.

In the case of functions represented on the grid, the developers work with a linear array that contains the values of
the field for each grid point. Additional data structures provide
information about the grid structure. This
level of abstraction makes it simple for developers to write code that
works for different problem dimensionality, and different kinds and
shapes of grids.

In terms of performance, by hiding the structure of the grid, we can
use sophisticated methods to control how the grid points are stored in
memory with the objective of using processor caches more efficiently
in finite-difference operators. We have found that by using space-filling curves~\cite{Peano_1890}, as shown in Fig.~\ref{fig:gridmaps}, and
in particular the Hilbert curve~\cite{Hilbert_1891,Skilling2004}, we can produce a significant improvement in the performance of semi-local operations. For example, in Fig.\ref{fig:gpu_laplacian} shows that a performance gain of around 50\% can be obtained for the finite-difference Laplacian operator running on a GPU by using a Hilbert curve to map the grid into memory.

