Projections are well-known techniques that help the visual exploration of high-dimensional data by creating depictions thereof in a low-dimensional space. While projections that target the 2D space have been studied in detail both quantitatively and qualitatively, 3D projections are far less understood. We fill this gap by first presenting a quantitative study that compares 2D and 3D projections along a rich selection of datasets, projection techniques, and quality metrics. Next, we refine these insights by a more detailed qualitative study that compares the preference of users in exploring high-dimensional data using 2D vs 3D projections augmented by visual explanations. Our findings indicate that, in general, 3D projections bring only limited added-value atop of the one provided by their 2D counterparts. All our datasets, source code, and Overview are made public for ease of replication and extension.