Name: Reefshape: A System for the Efficient Collection and Automated Processing of Time-Series Underwater Photogrammetry Data for Benthic Habitat Monitoring (Video) | JoVE Journal
Uploaded: 2025-06-13T13:25:00.000+00:00
Duration: 13 min 35 s
Description: Learn the ReefShape protocol for automated underwater photogrammetry to monitor coral reef habitats. This method uses permanent ground control markers and custom Python scripts to streamline data processing for ecological analysis.

ai_agent_IP

ai_agent

maintenance

instantTrialDurationMinutes

isBiopharma

enableNewMyJoveSectionDesign

educationFeatures

seoMetaApi

ai_copilot_institution_blacklist

ai_chat_institution_blacklist

freeAccessWindow

ssoRoleCapture

scheduledMaintenanceBanner

joveLabsFeatures

disable_autoplay_for_bots

extended_trial_duration

5 minutes

ReefShape is a non-copyrighted term that we have used as a name for this method. Project conceived by CD and WG, methods development by WG, coding by WG and SM, writing by WG, editing and reviewing by JL and CD. Special thanks to the entire team at the Perry Institute for Marine Science for their feedback and support throughout the development of this method. Funding was provided by Disney Conservation Fund. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. 2233001. Data collected under Bahamas Department of Environmental Planning and Protection permits no. SRBS-0013-2021-CD, BS-2021-930119, BS-2022-281752, BS-2022-315006, BS-2023-661916, BS-2023-610959, and BS-2023-211510.

<ol><li>Hughes, T. P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Coral+reefs+in+the+Anthropocene%5BTitle%5D)+AND+%22Nature%22%5BJournal%5D)">Coral reefs in the Anthropocene.</a> Nature. 546 (7656), 82-90 (2017).</li>
<li>Fisher, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Species+richness+on+coral+reefs+and+the+pursuit+of+convergent+global+estimates%5BTitle%5D)+AND+%22Current+Biology%22%5BJournal%5D)">Species richness on coral reefs and the pursuit of convergent global estimates.</a> Current Biology. 25 (4), 500-505 (2015).</li>
<li>Costanza, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Changes+in+the+global+value+of+ecosystem+services%5BTitle%5D)+AND+%22Glob+Environ+Change%22%5BJournal%5D)">Changes in the global value of ecosystem services.</a> Glob Environ Change. 26 (1), 152-158 (2014).</li>
<li>Jackson, J. B. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Reefs+since+Columbus%5BTitle%5D)+AND+%22Coral+Reefs%22%5BJournal%5D)">Reefs since Columbus.</a> Coral Reefs. 16 (1), S23-S32 (1997).</li>
<li>Bourne, D., Iida, Y., Uthicke, S., Smith-Keune, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Changes+in+coral-associated+microbial+communities+during+a+bleaching+event%5BTitle%5D)+AND+%22ISME+Journal%22%5BJournal%5D)">Changes in coral-associated microbial communities during a bleaching event.</a> ISME Journal. 2 (4), 350-363 (2008).</li>
<li>Asner, G. P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Mapped+coral+mortality+and+refugia+in+an+archipelago-scale+marine+heat+wave%5BTitle%5D)+AND+%22Proc+Natl+Acad+Sci+U+S+A%22%5BJournal%5D)">Mapped coral mortality and refugia in an archipelago-scale marine heat wave.</a> Proc Natl Acad Sci U S A. 119 (19), e2123331119(2022).</li>
<li>Li, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+global+coral+reef+probability+map+generated+using+convolutional+neural+networks%5BTitle%5D)+AND+%22Coral+Reefs%22%5BJournal%5D)">A global coral reef probability map generated using convolutional neural networks.</a> Coral Reefs. 39 (6), 1805-1815 (2020).</li>
<li>Hedley, J. D., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Remote+sensing+of+coral+reefs+for+monitoring+and+management%3A+A+review%5BTitle%5D)+AND+%22Remote+Sens%22%5BJournal%5D)">Remote sensing of coral reefs for monitoring and management: A review.</a> Remote Sens. 8 (2), (2016).</li>
<li>Lirman, D., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Development+and+application+of+a+video-mosaic+survey+technology+to+document+the+status+of+coral+reef+communities%5BTitle%5D)+AND+%22Environ+Monit+Assess%22%5BJournal%5D)">Development and application of a video-mosaic survey technology to document the status of coral reef communities.</a> Environ Monit Assess. 125 (1), 59-73 (2007).</li>
<li>Edwards, C. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Large-area+imaging+reveals+biologically+driven+non-random+spatial+patterns+of+corals+at+a+remote+reef%5BTitle%5D)+AND+%22Coral+Reefs%22%5BJournal%5D)">Large-area imaging reveals biologically driven non-random spatial patterns of corals at a remote reef.</a> Coral Reefs. 36 (4), 1291-1305 (2017).</li>
<li>Burns, J. H. R., Delparte, D., Gates, R. D., Takabayashi, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Utilizing+underwater+three-dimensional+modeling+to+enhance+ecological+and+biological+studies+of+coral+reefs%5BTitle%5D)+AND+%22Int+Arch+Photogramm+Remote+Sens+Spat+Inf+Sci%22%5BJournal%5D)">Utilizing underwater three-dimensional modeling to enhance ecological and biological studies of coral reefs.</a> Int Arch Photogramm Remote Sens Spat Inf Sci. 40 (5W5), 61-66 (2015).</li>
<li>Burns, J. H. R., Delparte, D., Gates, R. D., Takabayashi, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Integrating+structure-from-motion+photogrammetry+with+geospatial+software+as+a+novel+technique+for+quantifying+3D+ecological+characteristics+of+coral+reefs%5BTitle%5D)+AND+%22PeerJ%22%5BJournal%5D)">Integrating structure-from-motion photogrammetry with geospatial software as a novel technique for quantifying 3D ecological characteristics of coral reefs.</a> PeerJ. 7, (2015).</li>
<li>Nocerino, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Coral+reef+monitoring+by+scuba+divers+using+underwater+photogrammetry+and+geodetic+surveying%5BTitle%5D)+AND+%22Remote+Sens%22%5BJournal%5D)">Coral reef monitoring by scuba divers using underwater photogrammetry and geodetic surveying.</a> Remote Sens. 12 (18), (2020).</li>
<li>Westoby, M. J., Brasington, J., Glasser, N. F., Hambrey, M. J., Reynolds, J. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((34%3BStructure-from-Motion%26%2334%3B+photogrammetry%3A+A+low-cost%2C+effective+tool+for+geoscience+applications%5BTitle%5D)+AND+%22Geomorphology%22%5BJournal%5D)">34;Structure-from-Motion&#34; photogrammetry: A low-cost, effective tool for geoscience applications.</a> Geomorphology. 179, 300-314 (2012).</li>
<li>Burns, J. H. R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((3D+habitat+complexity+of+coral+reefs+in+the+Northwestern+Hawaiian+Islands+is+driven+by+coral+assemblage+structure%5BTitle%5D)+AND+%22Int+Arch+Photogramm+Remote+Sens+Spat+Inf+Sci%22%5BJournal%5D)">3D habitat complexity of coral reefs in the Northwestern Hawaiian Islands is driven by coral assemblage structure.</a> Int Arch Photogramm Remote Sens Spat Inf Sci. 42 (2/W10), 61-67 (2019).</li>
<li>Storlazzi, C. D., Dartnell, P., Hatcher, G. A., Gibbs, A. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((End+of+the+chain%3F+Rugosity+and+fine-scale+bathymetry+from+existing+underwater+digital+imagery+using+structure-from-motion+%28SfM%29+technology%5BTitle%5D)+AND+%22Coral+Reefs%22%5BJournal%5D)">End of the chain? Rugosity and fine-scale bathymetry from existing underwater digital imagery using structure-from-motion (SfM) technology.</a> Coral Reefs. 35 (3), 889-894 (2016).</li>
<li>Fox, M. D., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Limited+coral+mortality+following+acute+thermal+stress+and+widespread+bleaching+on+Palmyra+Atoll%2C+central+Pacific%5BTitle%5D)+AND+%22Coral+Reefs%22%5BJournal%5D)">Limited coral mortality following acute thermal stress and widespread bleaching on Palmyra Atoll, central Pacific.</a> Coral Reefs. 38 (4), 701-712 (2019).</li>
<li>Kopecky, K. L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Quantifying+the+loss+of+coral+from+a+bleaching+event+using+underwater+photogrammetry+and+AI-assisted+image+segmentation%5BTitle%5D)+AND+%22Remote+Sens%22%5BJournal%5D)">Quantifying the loss of coral from a bleaching event using underwater photogrammetry and AI-assisted image segmentation.</a> Remote Sens. 15 (16), (2023).</li>
<li>Pascoe, K. H., Fukunaga, A., Kosaki, R. K., Burns, J. H. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((3D+assessment+of+a+coral+reef+at+Lalo+Atoll+reveals+varying+responses+of+habitat+metrics+following+a+catastrophic+hurricane%5BTitle%5D)+AND+%22Sci+Rep%22%5BJournal%5D)">3D assessment of a coral reef at Lalo Atoll reveals varying responses of habitat metrics following a catastrophic hurricane.</a> Sci Rep. 11 (12050), (2021).</li>
<li>Peterson, E. A., Carne, L., Balderamos, J., Faux, V., Gleason, A., Schill, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+use+of+unoccupied+aerial+systems+%28UASS%29+for+quantifying+shallow+coral+reef+restoration+success+in+Belize%5BTitle%5D)+AND+%22Drones%22%5BJournal%5D)">The use of unoccupied aerial systems (UASS) for quantifying shallow coral reef restoration success in Belize.</a> Drones. 7 (4), 221(2023).</li>
<li>Neufeld, A. M., Fundakowski, G. <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=author%3aNeufeld%2C+AM+%22Coral+Restoration+Foundation+TM+Photomosaic+Manual+Second+Edition-November+2020%22">Coral Restoration Foundation TM Photomosaic Manual Second Edition-November 2020</a>. , www.coralrestoration.org (2020).</li>
<li>Roach, T. N. F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+field+primer+for+monitoring+benthic+ecosystems+using+structure-from-motion+photogrammetry%5BTitle%5D)+AND+%22J+Vis+Exp%22%5BJournal%5D)">A field primer for monitoring benthic ecosystems using structure-from-motion photogrammetry.</a> J Vis Exp. , e61815(2021).</li>
<li>Suka, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Processing+photomosaic+imagery+of+coral+reefs+using+structure-from-motion+standard+operating+procedures%5BTitle%5D)+AND+%22NOAA+Technical+Memorandum.+%28NOAA-TM-NMFS-PIFSC-93%22%5BJournal%5D)">Processing photomosaic imagery of coral reefs using structure-from-motion standard operating procedures.</a> NOAA Technical Memorandum. (NOAA-TM-NMFS-PIFSC-93. , (2019).</li>
<li>Sandin, S. A., et al. Large area imagery collection & processing standard operating procedures - Version 3.0 (2021). <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=author%3aSandin%2C+SA+%22Standard+Operating+Procedure+Documents+for+Coral+Reef+Ecological+Monitoring+Collection.%C2%A0UC+San+Diego+Library+Digital+Collections%22">Standard Operating Procedure Documents for Coral Reef Ecological Monitoring Collection. UC San Diego Library Digital Collections</a>. , (2023).</li>
<li><a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=%22Agisoft+Metashape+Professional+Edition%22">Agisoft Metashape Professional Edition</a>. , (2024).</li>
<li><a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=%22ESRI+Survey123%22">ESRI Survey123</a>. , (2023).</li>
<li>Pavoni, G., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((TagLab%3A+AI-assisted+annotation+for+the+fast+and+accurate+semantic+segmentation+of+coral+reef+orthoimages%5BTitle%5D)+AND+%22J+Field+Robot%22%5BJournal%5D)">TagLab: AI-assisted annotation for the fast and accurate semantic segmentation of coral reef orthoimages.</a> J Field Robot. 39 (3), 246-262 (2022).</li>
<li>Hernández-Landa, R. C., Barrera-Falcon, E., Rioja-Nieto, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Size-frequency+distribution+of+coral+assemblages+in+insular+shallow+reefs+of+the+Mexican+Caribbean+using+underwater+photogrammetry%5BTitle%5D)+AND+%22PeerJ%22%5BJournal%5D)">Size-frequency distribution of coral assemblages in insular shallow reefs of the Mexican Caribbean using underwater photogrammetry.</a> PeerJ. 8, e8957(2020).</li>
<li>Fukunaga, A., Burns, J. H. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Metrics+of+coral+reef+structural+complexity+extracted+from+3D+mesh+models+and+digital+elevation+models%5BTitle%5D)+AND+%22Remote+Sens%22%5BJournal%5D)">Metrics of coral reef structural complexity extracted from 3D mesh models and digital elevation models.</a> Remote Sens. 12 (17), RS12172676(2020).</li>
<li>Akkaynak, D., Treibitz, T. Sea-Thru: a method for removing water from underwater images. <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=author%3aAkkaynak%2C+D+%22Proc+IEEE+Comput+Soc+Conf+Comput+Vis+Pattern+Recognit%22">Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit</a>. , 1682-1691 (2019).</li>
<li>DeepSeeColor: realtime adaptive color correction for autonomous underwater vehicles via deep learning methods. In. Jamieson, S., How, J. P., Girdhar, Y. 2023 IEEE Int Conf Robot Autom, , 3095-3101 (2023).</li>
<li>Menna, F., Nocerino, E., Chemisky, B., Remondino, F., Drap, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Accurate+scaling+and+levelling+in+underwater+photogrammetry+with+a+pressure+sensor%5BTitle%5D)+AND+%22Int+Arch+Photogramm+Remote+Sens+Spat+Inf+Sci%22%5BJournal%5D)">Accurate scaling and levelling in underwater photogrammetry with a pressure sensor.</a> Int Arch Photogramm Remote Sens Spat Inf Sci. 43 (B2-2021), 667-672 (2021).</li>
<li>Edwards, C. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Large-area+imaging+in+tropical+shallow+water+coral+reef+monitoring%2C+research+and+restoration%3A+a+practical+guide+to+survey+planning%2C+execution%2C+and+data+extraction%5BTitle%5D)+AND+%22NOAA+Technical+Memorandum+NOS+NCCOS+313%22%5BJournal%5D)">Large-area imaging in tropical shallow water coral reef monitoring, research and restoration: a practical guide to survey planning, execution, and data extraction.</a> NOAA Technical Memorandum NOS NCCOS 313. , (2023).</li>
<li><a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=%22Agisoft+Agisoft+Metashape+User+Manual+Professional+Edition%2C+Version+2.1%22">Agisoft Agisoft Metashape User Manual Professional Edition, Version 2.1</a>. , https://www.agisoft.com/pdf/metashape-pro_2_1_en.pdf (2024).</li>
</ol>

The authors have no competing financial interests or other conflicts of interest.

This protocol was designed to address some of the key challenges in underwater photogrammetry for coral reef habitats, including image quality and overlap during acquisition, metadata collection for the scaling, leveling, geolocation, and time-series alignment of models and maps, human decision-making during the photogrammetry processing pipeline, and preparation, and export of data for analysis. We address the first challenge with careful selection, testing, and recommendation of a camera system and important settings. While many systems can work for image acquisition, we find that setups involving action cameras (e.g., GoPro or similar) are insufficient due to rolling shutter distortion, lack of manual white balance, and inability to collect RAW photos at 1 Hz, all of which limit the quality of the final imagery and the ability to precisely align time-series data. Multi-camera DLSR setups can facilitate improved image coverage of the study plot24, but these setups are expensive and more cumbersome underwater. We instead choose a single, relatively inexpensive (~$2,800 USD total cost) mirrorless camera system with a mechanical shutter, custom white balancing, the ability to collect RAW and JPEG photos, a modern APS-C image sensor with low noise levels, and a wide-angle lens (~100&#176; field of view) and dome port combination that yields sharp images. A wide lens is chosen to increase image coverage and overlap, which improves 3D rendering of vertical and overhung surfaces, and reduces potential model holes. Using a 1 s interval and a single camera reduces the overall number of images over other methods without losing detail or model quality, speeding up processing. Finally, while RAW imagery is not immediately critical to the function of the current image processing pipeline that relies on JPEG images, we consider RAW imagery essential for archival purposes because it contains higher-quality color information. The white balance can be adjusted after collection in image processing software, and the higher quality color information could, in the future, be fed into a&#160;color correction algorithm such as SeaThru30 or DeepSeeColor31 and integrated into the photogrammetry pipeline for more consistent colors and detailed study of phenomena such as coral pigmentation and bleaching.
The second key challenge is 3D geolocation, scaling, and time-series alignment. While real-world coordinates are not necessary for many analyses, models must be scaled and leveled accurately for the orthorectification process and accurate measurements32,33. This process is difficult to automate within photogrammetry software without the use of detectable coded targets or without depth measurements and an XY coordinate system. Most protocols require the manual incorporation of reference information either in Metashape or later in other software, adding complexity and inefficiency to the workflow. By incorporating coded targets into scale bars of precisely known length and permanently fixed corner markers, coupled with a user-friendly geolocation and depth collection system, we provide the software with the necessary information to automatically define a coordinate system, locate, scale, and level the model. By specifying the accuracy of each measurement, the software correctly weights the final location solution such that it places priority on the scale, then depth, and finally XY coordinates, allowing highly accurate scaling and leveling even with relatively low-accuracy GNSS data. Co-registration of time-series photomosaics typically requires human intervention to locate, mark, and match consistent points on the reef; this process is very time-consuming and challenging if there are no clearly static features between time points. The use of durable ground control point markers alleviates this problem by providing a set of 4 automatically detectable static targets. With our scripted process, subsequent time points inherit the precise georeferencing from a previous time point, simplifying the alignment process significantly and reducing human input while facilitating highly accurate co-registration of time points that allow tracking fine-scale ecological and structural changes on the reef.
The third challenge that is addressed by this protocol is inefficiencies within the photogrammetry workflow, both in terms of human intervention and demands on computer hardware. We have designed the ReefShape scripts to allow the user to input all required information for the entire process in a single GUI box, removing any intervention typically needed at key steps in the process (i.e., the incorporation of scale, leveling, and georeferencing information). This allows the user to initiate the process and leave the rest to the computer, saving time and effort. The photogrammetry pipeline used in the Full ReefShape Workflow script (Figure 3 and Figure 4) is optimized to provide efficient processing. We employ a specialized alignment process consisting of two phases. A first alignment pass is run with generic preselection enabled, an option that can decrease image alignment time by many hours without leading to loss of accuracy34. A second phase then attempts to add any remaining unaligned photos to the pre-existing alignment without the use of generic preselection, which can mitigate alignment issues caused by suboptimal overlap or caustic bands from wave lensing in shallow water. Taken together, these steps represent an efficient and powerful alignment process that frequently leads to a much greater proportion of properly aligned photos than standard Metashape processing procedures. We generate a 3D-mesh directly from depth maps, bypassing the time and resource-intensive process of dense point cloud construction. Meshes generated this way tend to have less noise and better reconstruction of areas of low-coverage, preventing the need for point-cloud cleaning before meshing as used in other methods23. In our experience, this process for mesh generation tends to be more stable, leading to less computer crashes than dense point cloud construction and meshing. Finally, we generate a high-resolution DEM that is used as the orthorectification surface instead of the mesh, as this reduces orthomosaic building time dramatically without any perceptible loss in quality.
A final challenge is the preparation, standardization, and export of data for ecological analysis. By standardizing image product resolution at 0.5mm/pixel, we ensure consistent and comparable products across plots and across time-points, enhancing future efforts to utilize AI and machine-learning for analysis. The full ReefShape workflow script provides options to export a processing report and data products in the proper format for GIS software and TagLab27, standardizing formatting and saving time and effort over workflows where this step is done manually. A region of interest (ROI) polygon, necessary for many analyses, is generated automatically using the known corner marker positions, and exported as a standard shapefile for incorporation into GIS workflows, such as generating and identifying random points across the plot to analyze benthic composition. This ROI also enables cropped and pixel-aligned outputs for TagLab, necessary for time-series analysis as shown in Figure 5. The ROI also enables the automatic calculation of a 3D surface area to planar area ratio for each time-point (protocol step 5.2.8), important for measuring reef structural complexity and its change over time.
While this protocol represents an advance in efficiency and usability for underwater photogrammetry, limitations exist. Most notably, if the photogrammetry process fails in one of the key steps, user intervention is required to troubleshoot and fix issues before continuing. While our scripted process is designed to be usable by researchers without in-depth photogrammetry knowledge, a basic understanding is important to solve issues when they arise. A few key parts of the process are most prone to issues. First, images can fail to align due to poor image overlap or due to severe caustic bands on the substrate in shallow water and sunny conditions. Alignment failure can be detected by inspecting the tie-point cloud and camera positions. If the alignment fails due to caustic bands, re-running the full ReefShape workflow with Generic Preselection unchecked can typically solve the problem at the cost of dramatically longer processing time. If image overlap is insufficient for the software to align photos, then re-photographing the site is likely the best solution. Failed marker detection can also lead to an error in mapping. This is most often the case if a marker is damaged or not sufficiently cleaned. In this case, markers can be placed on individual photos within Metashape and named manually, and the ReefShape script(s) can be re-run to finish processing. In some cases, georeferencing can fail if the GPS points are so inaccurate that they are geographically out of order or if the GPS points were assigned to the incorrect targets. This can be solved by manually editing the georeferencing CSV to appropriately match the data and subsequently re-running the script. Finally, software crashes are possible, especially when using a less powerful computer with a large plot or with higher quality settings. In anticipation of this, our scripted process will auto-save after each completed step, allowing the user to restart and adjust settings without losing progress. Further troubleshooting recommendations are provided on our GitHub page.
The primary aim of ReefShape is to simplify the collection and processing components of underwater photogrammetry and to reduce costs as much as possible so that users can focus time and effort more fully on ecological data extraction. We provide a complete process that is designed to realize high-quality data outputs that are tailored to meet the needs of common ecological analysis options. While the protocol described is highly specific, the scripted approach to processing is flexible and can handle modifications to aspects of the method such as plot size, swim/photo collection pattern, target output resolution, and specific camera system used without issue. The method can also be applied without significant modification to most underwater or fine-scale terrestrial photogrammetry projects, such as the documentation of shipwrecks or archaeological sites.

Coral reefs are one of the most biodiverse and economically important ecosystems globally and face unprecedented challenges from climate change, disease, overfishing, and other stressors1,2,3. The monitoring of coral reef ecosystems is uniquely difficult due to their often-remote locations and inherent difficulties with underwater research; therefore, reefs have historically been understudied4. Monitoring coral reefs effectively at multiple spatial scales ranging from microbial5 to archipelago6 and global7 is essential to understanding their decline, as well as planning, tracking, and evaluating intervention efforts8. A tool that has become popular for monitoring the condition of coral reef benthos at the scale of tens to hundreds of square meters is photomosaic imaging, a term referring to high-resolution maps consisting of stitched-together overlapping underwater photographs9. These mosaics allow researchers to image an area of reef that is larger than can be captured in a single photograph, hence the term large-area imaging (LAI)10. The mosaics can later be analyzed to extract relevant ecological information, such as coral cover percentage, colony size, species distribution, and benthic composition11. Advances in computing and the availability of off-the-shelf software now allow this process to be completed using structure-from-motion (SfM) photogrammetry. SfM involves analyzing photos for matching points that are used to reconstruct the three-dimensional orientation of the photos and tie-points, enabling the creation of an accurate virtual reef replica12,13,14. SfM/LAI surveys have become commonplace within coral reef research, allowing for novel insights into coral community ecology10, habitat complexity15,16, coral community responses to bleaching events17,18, hurricanes19, and coral restoration20.
Several approaches for using LAI for coral reef monitoring have been developed21,22,23,24, resulting in a diverse array of choices available to practitioners seeking to leverage the technology. However, the effective use of LAI in coral reef research is complex and demands a substantial learning effort. Proficiency in SCUBA diving, underwater navigation, underwater photography, software utilization, data curation, and management are essential. Additionally, expertise in ecology is fundamental to effectively analyzing and interpreting data products. Existing workflows tend to focus primarily on image acquisition without providing sufficient guidance for time-series protocols, metadata collection (e.g., scaling, depth, and location), or post-field trip data processing: all steps that are essential for accurate and repeatable data collection. Costs associated with LAI workflows also tend to be high, utilizing expensive camera systems and computer setups. There remains a strong need among researchers for a comprehensive, straightforward, and efficient methodology, resulting in data of sufficient quality to answer a wide range of current and future research questions. We address this by developing a robust and efficient approach for underwater LAI that reduces processing effort and complexity and minimizes costs while improving the quality of data. Our new approach allows for rapid acquisition, automated processing, and time-series alignment of imagery to provide high-quality data products for coral reef ecological study and analysis. The total startup cost of implementing this approach is about $5,000 - $8,000 USD (including camera system, materials, dedicated computer, and software), depending on whether the user can access educational pricing for photogrammetry software. Through the application of our methods, we aim to assist coral reef researchers in optimizing their data collection and processing endeavors, enabling more efficient workflows that facilitate rapid extraction and analysis of critically important coral reef ecological data.
The method described here, which we name "ReefShape," has three main novel contributions: (1) the use of semi-permanent ground control markers fixed to the substrate to enable automatic georeferencing and time-series alignment of datasets, (2) the use of a custom app-based survey to facilitate location data collection and formatting, and (3) the implementation of a comprehensive scripted process built to fully automate the photogrammetry pipeline, dramatically reducing the human labor during the processing phase that is relied on in other LAI protocols20,21,22,23. Like these other LAI protocols, ReefShape relies on the use of Agisoft Metashape25 (hereafter termed "the photogrammetry program") for photogrammetric processing and additionally utilizes the free ESRI Survey12326smartphone application (hereafter termed "the survey app") for location data collection. This protocol is designed to be simple yet robust, not requiring multi-camera systems24 or complex geodetic surveys13 while still meeting the goal of delivering high-quality data, defined as completed 3D models, photomosaics, and digital elevation models with accurate geometry, scale, and position; sufficient resolution and sharpness to visually identify benthic organisms at the species or genus level; no major data gaps or holes; accurate color; and in the case of time-series data, proper alignment between time points. The specific approach described here provides a framework for collecting and processing data to meet these goals.
Driven by advances in machine learning, we anticipate that new analysis tools will be developed for faster, more accurate extraction of ecological data from photomosaics. Therefore, we focus our efforts on the collection of high-quality underwater imagery and the automation of the photogrammetry pipeline, leaving specific analyses largely up to users of this protocol based on their own diverse sets of needs. This scripted process, aimed to be broadly applicable to the coral reef research community, includes options to export data products formatted as GeoTIFFs of varying specifications tailored for common GIS software and TagLab, a purpose-built application for rapid annotation of coral reef orthomosaics27.
Protocol overview 
The ReefShape method is broken down into two main phases: in situ data collection and data processing on a computer. The method is functional for plot sizes from ~25 m2 up to &gt;1000 m2, ranging in depth from ~1 m to 30 m. It has been demonstrated that plots of 300-400 m2 are ideal to effectively capture the coral diversity on Caribbean reefs28. However, it was found that plots larger than ~100 m2 can be difficult for novice surveyors to navigate. Therefore, a plot size of 10 m x 10 m is described in the protocol as a starting point, but we do not intend to constrain users with this suggestion. Rather, it is suggested that users choose their plot size based on their own experience and research needs. The process for data collection remains effectively the same for any plot size chosen.
When a plot is first established, the surveyor begins by permanently fixing four unique marker tags featuring coded photogrammetry targets (Figure 1D) to the substrate at each corner (Figure 2), using a dive computer to measure the depth of each marker. Coded scale bars (Figure 1E) are temporarily placed within the plot, and substrate-facing photos are collected by the diver with a single mirrorless camera and wide-angle rectilinear lens positioned 1.5 m - 2 m above the reef, swimming in a double-crossed "lawnmower" pattern, similar to other established protocols11,21,24,. The entire process (including first-time setup and photography) can typically be completed in a single dive, though multiple dives may be required for deeper or larger plots. After photography, the surveyor uses a Bluetooth GPS unit mounted to a floatation device (Figure 1C) and a smartphone to collect GPS points at the surface above each corner marker using a custom form within the survey app, which then emails the reference data to the user in a pre-formatted spreadsheet. At subsequent plot surveys, the surveyor does not collect reference data or install markers and needs only to locate and clean the existing corner markers and collect photos, streamlining the process for time-series data collection.
For data processing, a set of custom Python scripts was developed that interface with the photogrammetry program to automate the pipeline (Figure 3), normally a process that requires human intervention at several points. The main processing steps of the automated pipeline include creating a tie-point cloud and estimating camera positions, building a 3D mesh model of the reef, building a 2.5D digital elevation model (DEM), building a 2D orthorectified photomosaic, and defining a region of interest (ROI) bounded by the four corner markers (Figure 4). In this workflow, the user inputs the photos and reference data in a graphic interface (Supplementary Figure 1) at the onset of processing, rather than needing to proceed through numerous steps before manually adding reference data and generating data products, as is common in other workflows21,22,23,24. For time-series processing, permanent corner markers facilitate the automatic alignment of time points, eliminating the need for manual alignment. The use of a standardized, scripted workflow helps ensure data consistency and saves significant human effort during processing, especially in projects with many time points. A suite of standalone scripts is also included to automate various processing tasks, including calculating a 3D surface area to planar area ratio, an important metric for assessing reef structural complexity19,29.
<img alt="Coral health monitoring setup: camera, underwater housing, stand, reference tiles for marine research." class="xfigimg" src="/files/ftp_upload/67343/67343fig01.jpg" data-altupdatedat="1788304762000" data-alt="Figure 1"> 
Figure 1: Key materials required for the data collection portion of this protocol. (A) mirrorless camera with wide-angle rectilinear lens, (B) underwater housing with dome port to fit camera/lens, (C) Bluetooth GPS kickboard device, (D) automatically detectable coded corner markers for permanent plot ground control and georeferencing, and (E) coded scale bars used for setting model size. <a href="https://www.jove.com/files/ftp_upload/67343/67343fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>

<table><tbody><tr><td>0.3m length 1/2&quot; PVC pipe</td><td>generic</td><td>N/A</td><td>for GPS kit</td></tr><tr><td>1&quot; PVC table cap</td><td>generic</td><td>N/A</td><td>for GPS kit</td></tr><tr><td>1/2&quot; to 1&quot; PVC reducer bushing</td><td>generic</td><td>N/A</td><td>for GPS kit</td></tr><tr><td>12mm f/2.0 AF E-mount lens</td><td>Samyang</td><td>SYIO12AF-E</td><td>also sold under brand name Rokinon</td></tr><tr><td>2&quot; fluted shank round masonry nails</td><td>generic</td><td>N/A</td><td>for corner marker installation</td></tr><tr><td>256GB UHS-1 V30 SD Card</td><td>SanDisk</td><td>SDSDXXD-256G-ANCIN</td><td>fastest possible UHS-1 SD card recommended</td></tr><tr><td>30m tape measure (optional)</td><td>N/A</td><td>N/A</td><td>for plot setup</td></tr><tr><td>Acrylic sheet, 3mm thick, cut to 80mm x 580mm (3x)</td><td>N/A</td><td>N/A</td><td>scale bar material</td></tr><tr><td>Aluminum camera tray</td><td>KitDive</td><td>N/A</td><td>fits camera housing and aids with holding camera</td></tr><tr><td>Underwater corner marker set with printed photogrammetry targets</td><td>N/A</td><td>N/A</td><td>custom made, contact author for details</td></tr><tr><td>Duracopy waterproof laser printer paper</td><td>Rite in the Rain</td><td>6511</td><td>can be replaced with waterproof sticker paper</td></tr><tr><td>E6000 epoxy</td><td>generic</td><td>N/A</td><td>for GPS kit</td></tr><tr><td>GLO2 Bluetooth GPS</td><td>Garmin</td><td>010-02184-01</td><td>other options exist, GLO2 is the most economical</td></tr><tr><td>ILCE a6700 mirrorless camera</td><td>Sony</td><td>ILCE6700/B</td><td>a6700 camera preferred, a6600 or a6400 are low-cost options</td></tr><tr><td>Laser Printer</td><td>N/A</td><td>N/A</td><td>any laser printer (not inkjet)</td></tr><tr><td>Metashape Professional Edition</td><td>Agisoft</td><td>N/A</td><td>required software</td></tr><tr><td>Plastic card (roughly 1mm x 5mm x 5mm)</td><td>N/A</td><td>N/A</td><td>for GPS kit, can also use discarded credit card or similar, cut in half.</td></tr><tr><td>Smartphone</td><td>N/A</td><td>N/A</td><td>any reasonably modern smartphone, for GPS data collection</td></tr><tr><td>Sony A6700 Sea Frogs 40M/130FT Waterproof housing with acrylic 6&quot; Dome Port V.1</td><td>SeaFrogs</td><td>N/A</td><td>if using a6600 or a6400 cameras, replace with appropriate SeaFrogs housing</td></tr><tr><td>Super glue</td><td>N/A</td><td>N/A</td><td>for GPS kit and scale bar creation</td></tr><tr><td>Swim fitness-style kickboard</td><td>Speedo</td><td>877530050021SZ</td><td>for GPS kit, brand unimportant</td></tr><tr><td>Watch-style dive computer</td><td>N/A</td><td>N/A</td><td>for collecting corner marker depths</td></tr><tr><td>Waterproof pouch</td><td>Xuniea</td><td>W-1188</td><td>brand unimportant, to fit GPS device</td></tr><tr><td>Waterproof phone pouch</td><td>Pelican</td><td>PP048884</td><td>brand unimportant, to fit smartphone</td></tr></tbody></table>

reefshape: a system for the efficient collection and automated processing of time-series underwater photogrammetry data for benthic habitat monitoring

Reefshape: A System for the Efficient Collection and Automated Processing of Time-Series Underwater Photogrammetry Data for Benthic Habitat Monitoring

Large-area imaging (LAI) through structure-from-motion photogrammetry has gained significant traction as a monitoring tool for coral reef ecosystems, allowing for the creation of a digital model of a section of reef that can be analyzed ex situ to collect data on a benthic composition, structural complexity, and other metrics. While a variety of approaches have been used, a systematic approach to data collection and computer processing remains a need for many researchers. To address this, we have developed ReefShape, a straightforward and comprehensive workflow for underwater image collection, georeferencing, data processing, and time-series alignment. Specific camera system recommendations and image acquisition instructions are provided based on our experience. A process to incorporate real-world georeferencing using permanent ground control markers fixed to the substrate that facilitates the automatic alignment of time-series datasets is described. A set of processing scripts was developed to automate the data processing workflow, streamlining&#160;and dramatically simplifying the normally time-consuming and complex process. Our scripted approach aims to reduce the burden of data processing on coral reef researchers, increase the efficiency of the photogrammetry pipeline, and export data in analysis-ready formats for use in common GIS and coral reef imagery segmentation programs. The methods described here provide a comprehensive solution to integrate photogrammetry as a reef monitoring tool while remaining flexible and leaving the specific analyses to be conducted up to the researcher.

A time-series LAI dataset was collected using this methodology at Simms Point Reef, off the southwestern end of New Providence, The Bahamas. Figure 4, Figure 5, Figure 6 and Supplementary Figure 4 all depict results from this experiment. The reference network was established, and the first images were collected in January 2023. It was rephotographed in August 2023, during a severe marine heat wave, to assess the severity of coral bleaching. Both time points were processed using the full ReefShape workflow script, with no user intervention required in the intermediate steps. Timepoint one involved 1,299 images, all of which were aligned successfully, with an average swim altitude of 1.8 m above the substrate, native ground resolution of 0.567 mm/px (standardized to 0.5 mm/px), a total coverage area of 208 m2 (as measured by the area within the corner markers), a 3D/2D surface area ratio of 2.887, a reprojection error of 1.12 px, a total internal geolocation accuracy of 30.6 cm, and scaling error of 1.4 mm. The entire process, at default settings within the Full ReefShape script, took 8 h 23 min, using a circa 2018 desktop computer with a 6-core CPU, 32 GB RAM, and an 8 GB discrete GPU (total cost ~$1,500 USD). A 2024 desktop with 14-core CPU, 64GB RAM, and a 24GB discrete GPU (total cost ~$4,000 USD) processed the same plot in 1 h 58 min total. The second time-point, including 1,974 / 1,974 aligned images, took 7 h 45 min on the older 2018 desktop.
A zoomed-in section of the two timepoints and basic coral bleaching analysis completed in TagLab27 is demonstrated in Figure 5, showing the utility of this process for analyzing benthic habitat change over time. Results from the analysis indicated that over 90% of individuals across many of the coral species experienced complete bleaching, confirming the severity of the event, while other species experienced minimal to no bleaching, giving insight into the patterns of resilience within the coral community. Both time points (Figure 5) demonstrate high-quality imagery, with sufficient sharpness and resolution for expert identification of benthic organisms to the species or genus level. The white balance is properly set, and there are no significant areas of blurring, data holes, or other artifacts, indicating that the protocol was successful in efficiently delivering the data needed to study the dynamics of coral reef benthic ecology. Figure 6 (top) shows a zoom-in of this plot as an example of high-quality imagery as compared to a poor-quality dataset (bottom) that does not meet data quality requirements, where image artifacts hamper ecological data extraction. The poor-quality dataset was collected with improper camera settings (incorrect white balance leading to red overall hue, and too open of an aperture leading to blurring) and insufficient photo overlap due to poor diver navigation, highlighting the importance of image acquisition (protocol section 2).
<img alt="Coral bleaching comparison; top: aerial view, bottom: health status maps; reef ecology study." class="xfigimg" src="/files/ftp_upload/67343/67343fig05.jpg" data-altupdatedat="1788304762000" data-alt="Figure 5"> 
Figure 5: Representative results showing time-series imagery and coral colony outlines from a plot photographed before and during a bleaching event, showing the accuracy of automatic alignment achieved with our protocol and the utility of this data to monitor benthic change over time. (A) shows the January 2023 image for the Simms Point site in New Providence, The Bahamas, (B) the August 2023 image, (C) living coral colonies in January 2023 classified as healthy (blue), pale (orange) or bleached (red), and (D) living coral colonies in August 2023 outlined with the same classification scheme. Coral colony outlining was completed using AI-assisted segmentation in TagLab. <a href="https://www.jove.com/files/ftp_upload/67343/67343fig05large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Coral reef orthomosaic quality comparison; high vs low resolution, imagery details, artifacts." class="xfigimg" src="/files/ftp_upload/67343/67343fig06.jpg" data-altupdatedat="1788304762000" data-alt="Figure 6"> 
Figure 6: Example of a high-quality orthomosaic and a low-quality orthomosaic, with annotations demonstrating the key attributes that distinguish the two. Visual inspection of the orthomosaic and other data products during protocol step 5.4 is necessary to evaluate whether the protocol was executed properly or if a re-do of plot photography is needed to meet the data quality goals. <a href="https://www.jove.com/files/ftp_upload/67343/67343fig06large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
Software access: 
The ReefShape python scripts, installation and use instructions, and more detailed instructions on data collection and software use in the form of a white paper are available at https://github.com/Perry-Institute/ReefShape. We intend to update the scripts to address issues as they arise and make improvements. Therefore, we recommend using the latest version.
Supplementary Figure 1. Screenshot illustrating the ReefShape georeference data collection survey within the Survey123 app on a smartphone. Users can access the survey without a paid account and are emailed their data pre-formatted for use in ReefShape processing upon submission. <a href="https://www.jove.com/files/ftp_upload/67343/SI-1_Survey123.tif" target="_blank">Please click here to download this File.</a>
Supplementary Figure 2. Screenshot of the Full ReefShape Workflow script GUI. Users can enter all necessary information for processing in this interface, allowing for full automation of the photogrammetry workflow. <a href="https://www.jove.com/files/ftp_upload/67343/SI-2_Full%20ReefShape%20Workflow%20Script.tif" target="_blank">Please click here to download this File.</a>
Supplementary Figure 3. Screenshot of the Align Timepoints script GUI. Users can utilize this script to facilitate automatic alignment of subsequent surveys to the original time point, allowing for time-series change analysis. <a href="https://www.jove.com/files/ftp_upload/67343/SI-3_Align%20Timepoints%20Script.tif" target="_blank">Please click here to download this File.</a>
Supplementary Figure 4. Screenshot of a 3D model, highlighting representative results and the reference panel. All information in the reference panel is set up automatically according to the scale bar and georeferenced files input into the Full ReefShape Workflow script. The varying accuracy measures for different reference data (0.25 mm for scaling, 10 cm for depth, and ~70 cm for XY geolocation) are input automatically and facilitate preferential treatment of scale, then depth, then XY coordinates in the final location solution computed by Metashape. <a href="https://www.jove.com/files/ftp_upload/67343/SI-4_GeoRef_Example.tif" target="_blank">Please click here to download this File.</a>
Supplementary File 1: Equipment preparation instructions, camera system requirements and settings, and computer requirements and software setup steps that fall outside the scope of the protocol itself are contained within the Supplementary File. <a href="https://www.jove.com/files/ftp_upload/67343/Supplementary_File_1_.docx" target="_blank">Please click here to download this File.</a>

Presented here is a protocol for collecting and processing underwater photogrammetry data, including a significantly simplified and fully automated image processing pipeline resulting in georeferenced and time-series aligned outputs ready for ecological data extraction, analysis, and application.

Large-area imaging (LAI) through structure-from-motion photogrammetry has gained significant traction as a monitoring tool for coral reef ecosystems, ...

reefshape-system-for-efficient-collection-automated-processing-time

Research

JoVE Journal

Environment

We describe a new method for counting fishes, and estimating relative abundance (MaxN) and fish density using rotating stereo-video camera systems. We also demonstrate how to use distance from camera (Z distance) to estimate species-specific detectability.

Development of New Methods for Quantifying Fish Density Using Underwater Stereo-video Tools

Image based surveying is an increasingly practical, non-invasive method to sample the marine environment. We present the protocol of a drop camera survey that estimates the abundance and distribution of the Atlantic sea scallop (Placopecten magellanicus). We discuss how this protocol can be generalized for application to other benthic macroinvertebrates.

Methods for Image-based Surveys of Benthic Macroinvertebrates and Their Habitat Exemplified by the Drop Camera Survey for the Atlantic Sea Scallop

We provide a detailed protocol for conducting underwater structure-from-motion photogrammetry surveys to generate 3D models and orthomosaics.

A Field Primer for Monitoring Benthic Ecosystems Using Structure-From-Motion Photogrammetry

This protocol covers a four-step large-area imaging survey methodology used to extract metrics of structural complexity, community composition, and population demographics for coral reef communities. The quality of imagery collected and integrated access to the source imagery are prioritized within each step of the protocol.

Measuring the Structure, Composition, and Change of Underwater Environments with Large-area Imaging

Surface renewal is a micrometeorological method that is being used increasingly to determine energy fluxes, but its technical complexity makes it inaccessible to a broad audience. We describe the steps needed to set up and calibrate a surface renewal field station, to acquire and process data, and to correctly interpret results.

Surface Renewal: An Advanced Micrometeorological Method for Measuring and Processing Field-Scale Energy Flux Density Data

Here, we present small core incubations for the measurement of sediment-water gas and solute exchange. These will provide reliable measurements of sediment-water exchange that assess the role of sediment in influencing biological and biogeochemical processes in aquatic ecosystems.

The Benthic Exchange of O2, N2 and Dissolved Nutrients Using Small Core Incubations

Previous work suggests that the nitrogen isotopic composition of atmospheric nitrogen oxides might distinguish the influence of different sources in the environment. We report on an automated, mobile, field-based method for the high collection efficiency of atmospheric NOx for N isotopic analysis at an hourly time resolution.

Automated, High-resolution Mobile Collection System for the Nitrogen Isotopic Analysis of NOx

Three-dimensional (3D) reflection seismology is a powerful method for imaging subsurface volcanoes. By using industrial 3D seismological data from the Tarim Basin, we illustrate how to extract the sills and the conduits of subsurface volcanoes from seismic data cubes.

Data Processing Methods for 3D Seismic Imaging of Subsurface Volcanoes: Applications to the Tarim Flood Basalt

Spatial patterns of soil erosion and deposition can be inferred from differences in ground elevation mapped at appropriate time increments. Such changes in elevation are related to changes in near-surface soil carbonates. Repeatable methods for field and laboratory measurements of these quantities and data analysis methods are described here.

Measuring and Mapping Patterns of Soil Erosion and Deposition Related to Soil Carbonate Concentrations Under Agricultural Management

This article outlines a suite of techniques in light and electron microscopy to study the internal and external eye anatomy of insects. These include several traditional techniques optimized for work on ant eyes, detailed troubleshooting, and suggestions for optimization for different specimens and regions of interest.

Techniques for Investigating the Anatomy of the Ant Visual System

Learn the ReefShape protocol for automated underwater photogrammetry to monitor coral reef habitats. This method uses permanent ground control markers and custom Python scripts to streamline data processing for ecological analysis.

Reefshape: A System for the Efficient Collection and Automated Processing of Time-Series Underwater Photogrammetry Data for Benthic Habitat Monitoring (Video) | JoVE Journal

Reefshape: A System for the Efficient Collection and Automated Processing of Time-Series Underwater Photogrammetry Data for Benthic Habitat Monitoring

Explore More Videos