\section{Experimental Results}

\subsection{Tracking Performance}
\label{tracking_calibration}

Tracking accuracy was evaluated by way of the CLEAR MOT metric Measure Of Tracking Accuracy (MOTA) \cite{Bernardin_2008} using as ground truth manually annotated trajectories with an open source annotation tool from the \href{http://www.jpjodoin.com/urbantracker/index.html}{Urban Tracker project} \cite{Jodoin_2014}. Perfect tracking would yield MOTA of 100\%, while it can become negative if more false alarms are done than there are ground truth objects. Nearly 40,000 observations (instants) of 371 motor vehicles were annotated manually across two prototypical sample sites: one site where objects were tracked at a distance of between 20 and 50 meters away from the camera and one site where objects were tracked at a distance no greater than 20 meters away. An initial MOTA was calculated for default tracking parameters and offered modest performance (in the neighbourhood of $70\%$). A tracking optimization was then performed using a genetic algorithm similarly to \cite{ettehadieh15systematic} to search for parameters that maximized MOTA. After optimization, accuracy increased to $94\%$ (measured over the same ground truth used for optimization). These statistics are summarized in Table~\ref{tab:track_calib_data}. Tracking parameter optimization converged in less than 24 hours.

Of important note is the fact that this calibration was performed on the oldest and poorest quality video data (resolution of $800\times 600$, without software-assisted image stabilization, and before lens correction). Tracking results are expected to improve with increases in quality of video data and video pre-treatment. The optimized tracking parameters should be portable to other sites with similar view and camera characteristics, though this will be investigated further and be the subject of a future paper in depth.

\begin{table}
    \caption{Tracking Performance}
    \label{tab:track_calib_data}
    \begin{tabularx}{\textwidth}{XXXXX}
    \hline
    \textbf{Site}             & \textbf{Ground Truth Objects} & \textbf{Ground Truth Observations} & \textbf{Unoptimized MOTA (Default Parameters)} & \textbf{Optimized MOTA} \\
    \hline
    \#1-1-12 (far camera)  & 146                  & 12887                     & 0.685                                 & 0.944          \\
    \#22-3-16 (near camera) & 225                  & 25011                     & 0.744                                 & 0.853          \\
    \end{tabularx}
\end{table}



\subsection{Data Size}

In total, 473 hours of video data were collected (see Table~\ref{tab:data_details} for details). Data was typically collected for 12 hours from 7 a.m. to 7 p.m.. At 30 frames per second, a data collection at an intersection over a driving distance of 50 metres and at an average driving speed of 30~km/h, and with an average hourly flowrate of 500 veh/h yields approximately 90,000 instantaneous moving object measurements per hour. Additionally, each of these observations can have anywhere between 3 and 100 feature tracks associated with it. For an adequate balance between performance and tracking accuracy, the recommended number of features to aim for is roughly 15-20 per object over time, depending on the typical time spent by a road user in the field of view. This yields manageable data sizes (roughly 500~MB of storage per hour of video) while maintaining an adequate level of data richness and object representation. Video storage needs will vary greatly by camera choice, resolution, frame rate, and video encoding settings.

\begin{table}
    \caption{Data details}
    \label{tab:data_details}
    \begin{tabular}{ll}
    \hline
    \textbf{Roundabouts} & 20 \\
    \textbf{Analysis Areas} & 36 \\
    \textbf{Hours of Video Data} & 473 \\
    \textbf{Estimated Total Traffic Volume} & 79,432 \\
    \textbf{Disk Space (Video + Data + Overhead)} & 1.9 TB \\
    \textbf{Veh-km Traveled} & 9505 veh-km \\
    \hline
    \end{tabular}
\end{table}

Figure~\ref{fig:flow-userpairs} shows hourly number of user pairs observed versus traffic volume. Trends are evident, but explaining factors are not clear (probably a mix of several lane arrangement indicators). This will need further study. The number of user pairs per hour should be linearly correlated with the number of interaction instants as is demonstrated in Figure~\ref{fig:userpairs-interactions}. If they are not, it is possible that analysis areas across sites are not comparable, particularly for disaggregated TTCs.
% paul: "The number of user pairs per hour should be linearly correlated with the number of interaction instants as is demonstrated in Figure~\ref{fig:userpairs-interactions}. If they are not, it is possible that some site analysis areas are not comparable." not sure why...
% paul->nicolas: Added more details to explain this.

Most of the analysis is conducted on a pair of dedicated consumer-grade high-performance machines (INTEL Core i7 3770k processor with 16 to 32~GB of memory), with some work offloaded to a computing cluster when acceleration of work is necessary. Feature tracking performance depends on video resolution and special post-processing requirements such as stabilization or lens correction (for distortion). A typical one hour $800\times 600$~pixels video at 30~frames per second is processed with current consumer-grade hardware in about an hour. A typical one hour $1280\times 960$ video with correction for distortion can be processed in about two hours. Basic analysis on one of these trajectory sequences takes between 5 minutes and 30 minutes, depending on traffic in the scene, while interaction analysis, particularly using motion patterns, can typically take anywhere between 1 and 48 hours to complete. Interaction analysis processing times are very sensitive to the interaction complexity of the scene.