\documentclass{report}
% rubber: rules rules.ini

\usepackage{xspace}
\usepackage{graphicx}

\newcommand{\eztrace}{EZTrace\xspace}
\newcommand{\reporttitle}{\eztrace}
\newcommand{\reportsubtitle}{Developper Manual}

\newcommand{\pptrace}{PPTrace\xspace}
\newcommand{\ldpreload}{LD\_PRELOAD}
\newcommand{\eg}{\emph{eg.}}
\newcommand{\ie}{\emph{ie.}}

% Title Page
\title{
{\Huge\bf \reporttitle{}}\\[6mm]
{\LARGE\bf \reportsubtitle}\\[12mm]
}

\begin{document}
\maketitle

\tableofcontents

\chapter{Introduction}

The aim of this document is to describe \eztrace internals.

In addition to the content of this document, a few research papers
describe some aspects of \eztrace internals:
\begin{itemize}
\item \emph{\eztrace: a generic framework for performance
  analysis}~\cite{TraRue11Ccgrid} briefly describes the architecture
  of \eztrace.
\item \emph{An open-source tool-chain for performance
  analysis}~\cite{benedict2010periscope} describes the \eztrace + gtg
  + ViTE tool chain. This paper includes explanations on the way
  traces are generated
\item \emph{Runtime function instrumentation with
  EZTrace}~\cite{pptrace_2012} describe how \eztrace instruments
  applications
\item \emph{Selecting points of interest in traces using patterns of
  events}~\cite{PDP_2015} describes how \eztrace detects patterns of
  events (this work is under development in the
  \texttt{eztrace\_pattern\_matching} branch)
\end{itemize}

\chapter{Instrumentation}

In order to intercept the application calls to the \eztrace modules
function, \eztrace needs to \emph{instrument} the application. The
goal of the instrumentation of function \texttt{foo} is to be able to
record an event before and after each occurence of \texttt{foo}.

Depending on which type of function is \texttt{foo}, \eztrace uses two
mechanisms:
\begin{itemize}
\item if \texttt{foo} is in a shared library, \eztrace uses a
  mechanism based on \ldpreload.
\item otherwise, \eztrace uses \pptrace
\end{itemize}

\section{\ldpreload}

When instrumenting a program in order to intercept the calls to
function \texttt{f()} located in a shared library, \eztrace uses
\ldpreload.

As depicted in Figure~\ref{fig:ldpreload}, the basic idea is to ask
the program loader to pre-load a library that defines \texttt{f()}
(\eg \texttt{libeztrace-foo.so}) when loading the application. This
way, the application calls to function \texttt{f()} are directed to
\eztrace implementation of \texttt{f()} (let's call this
implementation \texttt{eztrace\_f()}).

Once \texttt{eztrace\_f()} has recorded an event, it needs to call the
actual \texttt{f()} function. To do this, \eztrace retrieves the
address of \texttt{f()} using the \texttt{dlsym()} function.  This is
done at when \texttt{libeztrace-foo.so} is loaded (
\texttt{DYNAMIC\_INTERCEPT\_ALL()} in the \texttt{libinit} function).

\begin{figure}
\centering
\includegraphics[width=0.95\linewidth]{Figures/instrumentation_ldpreload}
\caption{Instrumentation using \ldpreload}
\label{fig:ldpreload}
\end{figure}


\section{\pptrace}

When the function to instrument is not in a shared library, the
\ldpreload mechanism cannot work. In that case, \eztrace modifies the
application binary using \pptrace.

The main idea is that \pptrace load the application binary and uses
\texttt{ptrace} in order to modify the application code section. When
instrumenting a function, \pptrace injects a jump instruction at the
beginning of the function to instrument. This Section the detailed
procedure to do so.

Two techniques can be used depending on the where \eztrace managed to
allocate memory:
\begin{itemize}
\item If the memory allocated ``close'' to the code segment (for
  instance on 32-bits architectures like ARMv7), \pptrace uses a long
  jump
\item If the memory is allocated ``far'' from the code segment, (for
  instance, on 64-bits architectures like x86\_64),
  \pptrace uses a trampoline technique
\end{itemize}

Both strategies are implemented in the \texttt{src/pptrace/hijack.c}
file.

\subsection{Using long jump}

In some cases, \pptrace uses a long jump for changing the execution
flow of the application. The resulting execution flow is depicted in
Figure~\ref{fig:pptrace_long}.

\pptrace first allocates a buffer (pptrace\_allocated in the Figure)
where it stores the first opcodes of function \texttt{foo()}. These
opcodes are now located at address \texttt{reloc\_addr}. They are followed
by a jump (\texttt{back\_jump}) to the remaining opcodes of \texttt{foo}.

\pptrace then changes the first opcode of \texttt{foo()} to the
instruction that jumps (\texttt{first\_jump}) to \texttt{repl\_addr}
(which contains the address of \eztrace implementation of
\texttt{foo}).

The address of the relocated code (\texttt{reloc\_addr}) is then
assigned to the module callback for function foo (stored at address
\texttt{orig\_addr}).

\begin{figure}
\centering
\includegraphics[width=0.95\linewidth]{Figures/instrumentation_pptrace_arm}
\caption{Instrumentation using a long jump}
\label{fig:pptrace_long}
\end{figure}

The drawback of this technique is that the \texttt{first\_jump} may
overwrites many opcodes. Some of these opcodes may be problematic (for
instance, \texttt{ret} or \texttt{mov (\%rip+1042), \%eax}). When it
is possible, another technique that uses a \emph{trampoline} can be
used.

\subsection{Using a trampoline}
In order to reduce the number of opcodes that need to be relocated,
\pptrace can use the \emph{trampoline} technique depicted in
Figure~\ref{fig:pptrace_trampoline}. The idea is to first do a ``small
jump'' to a trampoline that jumps (\texttt{long\_jump}) to \eztrace
implementation of \texttt{foo}.

The small jump can be implemented as a relative jump (\ie ``jump to
\%rip + 1000'').

\begin{figure}
\centering
\includegraphics[width=0.95\linewidth]{Figures/instrumentation_pptrace_x86_64}
\caption{Instrumentation using a trampoline}
\label{fig:pptrace_trampoline}
\end{figure}



\bibliographystyle{plain}
\bibliography{bib}
\end{document}
