Introduction

Three systems were used in this test: 

• DJI Matrice M350 RTK UAV with EchoLogger ECT D24S single-beam dual-frequency echo sounder
• CHCNAV Apache 3 USV with a D230 single-beam single-frequency echo sounder
• OceanAlpha SL40 USV with a Geoswath 4 bathymetric side scan sonar

Depth measurements of a relatively shallow lake were acquired and then plotted as elevation bathymetry maps to evaluate the differences between the performance of the systems.

Data collected by

• Sergey Kucenko¹ (DJI Matrice M350 RTK UAV)
• Ilvars Liepiņš² (CHCNAV Apache 3 USV)
• Thomas Hiller³ (OceanAlpha SL40 USV)

Data processing and report by Matīss Brants¹, Alexey Dobrovolskiy¹. 

¹SPH Engineering, Riga, Latvia

²Latvijas Valsts Meži AS (Latvian state forests), Riga, Latvia

³THURN Group Ltd., Norwich, UK 

Data samples to access >>>>

Methods

Test area

The test was conducted on Lake Titurga near Riga, Latvia (N 56.874°, E 24.134°) on the 19^th of September 2024. The 8.5-hectare lake is relatively shallow, with a mean depth of approximately 7 meters and several deeper pits. It is an eutrophic lake (i.e., with increasing algae and plant growth), which continually amasses organic material on top of the sandy bottom, although it has been dredged during the last decades. The lake experiences a water flow-through due to a few inflow and outflow moats. The largely overgrown shore is low-sloping for a few meters and then has a sharp drop of around 3-4 meters. Access to the lake is very easy via the artificial sandy beach with a wooden pier at the southwestern shore (figure 1).

Used systems

DJI Matrice M350 RTK UAV with an EchoLogger ECT D24S single-beam echo sounder

The airborne integrated system consists of the UAV, the echo sounder sensor, an altimeter, and an onboard computer. The DJI Matrice M350 RTK UAV, used as the platform (Figure 2), is one of the most popular industrial drones on the market. Under the platform hangs a 2.5m long cable in a protective sleeve with the echo sounder attached at the end of it, completely submerged in the water. While the horizontal positioning of the system is enabled via the RTK GNSS sensor, accurate altitude control is done with the help of a radar altimeter as part of the True Terrain Following system by SPH Engineering. The whole system is coupled by the SPH Engineering SkyHub on-board controller, which records and combines the data from the sensors. During the test, the system was guided automatically and accurately along a route planned in the UgCS (Universal Ground Control Software). The maximum recommended speed with the echo sounder in cylindrical housing submerged is approximately 0.7 m/s. The whole system weighs approximately 8.9 kilograms.

The EchoLogger ECT D24 is a single-beam dual-frequency echo sounder working at 200 and 450 kHz frequencies. The low-frequency mode works from 0.5 meters with a beam angle of 10°, while the high-frequency mode can map shallow waters starting from 0.15 m with a beam angle of 5°. The device has a built-in tilt sensor to correct or remove data collected at a too-large angle. Data from the echo sounder is logged in NMEA 0183 and CSV formats. More information about the system can be found at the following link: https://www.sphengineering.com/echo-sounder

CHCNAV Apache 3 USV with a single-beam echo sounder

One of the two tested Unmanned Surface Vehicle systems was the CHCNAV Apache 3 (Figure 3), which is a very portable, shallow-draft marine drone suited for bathymetric tasks in shallow inland waters and coastal areas. The platform is equipped with dual RTK GNSS antennas for accurate positioning and navigation, as well as with an IMU (Inertial Measurement Unit) for navigation backup during GNSS outages. The USV can be operated manually or automatically via the integrated Android remote control, which offers route planning, video streaming and data collection. The maximum speed of the USV is 6 m/s. The total weight of the system is 7 kilograms.

The echo sounder used aboard the Apache 3 is the D230 Single-Beam Echo Sounder producing a 200 kHz pulse with a beam angle of 6.5° ± 1°, and is usable in the depth range of 0.15 to 200 meters. Hull sway is compensated via the sensors aboard the USV. The echo sounder supports NMEA SDDPT/SDDBT sentences, as well as its own proprietary CHCGD format.

OceanAlpha SL40 USV with a Geoswath 4 USV swath bathymetric side scan sonar

The second tested USV was the OceanAlpha SL40 (Figure 4) - a small, advanced shallow-water marine drone for bathymetric surveys. Instead of acquiring a single measurement below the platform like the other tested systems, the Geoswath 4 USV gathers data from multiple points across the route track, thus enabling faster surveying. Accurate positioning is provided by the RTK GNSS, and automatic route execution is available via the base station computer, but manual control is also available for more difficult locations. The built-in INS (Inertial Navigation System) records the motions of the hull for calibration purposes, which also helps in cases when GNSS navigation is limited. The USV can reach a maximum speed of 5 m/s. The total weight of the system is 42 kilograms.

The bathymetric measurements are done using the GeoSwath 4 USV interferometric sonar - the 240° wide swath at an angular resolution of 0.04° across the track and an along-track beamwidth of 0.5° provides very fast and precise data acquisition while keeping the total route length to a minimum. During this test, the 500 kHz transducer was used, which is suited for shallow waters but lower frequencies are also available. A valuable addition to a bathymetric survey is the georeferenced side scan sonar, which takes “sonic pictures” of the bottom of the water body. This greatly improves the interpretability of the bathymetric data, as well as aids in various search and monitoring missions. The GeoSwath sonar head incorporates a Teledyne Valeport mini speed of sound sensor, taking sound velocity (SV) measurements every 0.1 seconds for sonar SV corrections at the surface.

Survey execution

The survey was conducted over the western part of the lake, which has easy access to the water. To simulate a shallow water bathymetric survey, each system was operated based on its normal operating modes, which means the guidance, line spacing, and sensor sampling rate were different in all cases.

The UAV system was guided by RTK GNSS in automatic mode with 5 meter line spacing at a speed of 0.7 m/s. Its echo sounder was working in dual frequency mode (200 and 450 kHz) with a sampling rate of 15 Hz.

The CHCNAV Apache 3 was operating in RTK GNSS automatic guidance mode at a speed of 3 m/s, while the 200 kHz echo sounder was acquiring measurements at a rate of 3 Hz.

The OceanAlpha SL40 was guided manually in RC mode at an average speed of 1.5 m/s. The sampling rate was 25 Hz with simultaneous port and starboard side pings. It must be noted that the sampling rate can be adjusted for specific missions. Continuous sound velocity measurements were recorded: SV was between 1485m/s and 1487m/s at the lake surface. Unfortunately, the shipper had delivered the SL40 to their Lithuanian transport hub, not Latvia, so it arrived on-site with 30 minutes of survey time remaining. It was not possible to configure the GNSS to work with local RTK corrections, which means that the positioning data were acquired with an offset.

No sound velocity profiles were taken during the surveys; the lake was assumed to be mixed.

Data processing

All the data were processed using hydrographic software endorsed by the respective bathymetric device manufacturer. The data of the UAV-based EchoLogger ECT D24S was processed in Eye4Software Hydromagic. Only the low-frequency (200 kHz) data were used in further data processing. Most outlier depth values were manually removed, while values acquired when the sensor was tilted more than 10 degrees were automatically removed. The default sound velocity of 1500 m/s was used for calculations.

The CHCNAV Apache 3 USV data was exported using native CNCNAV software and manually cleaned in Eye4Software Hydromagic to remove spikes.

Similarly, data from the OceanAlpha SL40 USV were processed using the proprietary GS4 software, which handled all the corrections, including the continuous SV measurements and removal of outliers. Since the OceanAlpha SL40 USV was not receiving RTK corrections, it operated in INS-aided differential GNSS mode. The elevation values were calculated by subtracting the depth values from the water surface level acquired during the UAV-based survey, using the water surface as the vertical datum. The approximately 6-meter horizontal offset was corrected manually and the interpolated grid was translated into the approximate location for at least a qualitative assessment.

The data were processed to acquire the bathymetry of the lake bottom, expressed as the elevation above the local geoid. Elevation values are not affected by water level changes unlike water depth values. Bathymetric data were gridded using TIN interpolation with a cell size of 0.5 meters.

Evaluation methods

Three main methods were implemented to evaluate the results. First, a simple qualitative comparison between the data from all systems – a visual inspection to see if all three systems recorded similar lakebed features. This gives a crude estimate of the usability of the data. Second, a cross-line check to determine the precision of each system, which involves checking if repeated measurements give the same results. This value is expressed as the root mean squared error (RMSE). Third, a quantitative comparison between all systems is needed to determine the relative accuracy of the data, which includes the evaluation of the direct measurements and the interpolated data points. Unfortunately, the last step was not possible for the OceanAlpha SL40 USV data due to the aforementioned lack of accurate RTK GNSS location data.

Results

Qualitative comparison

Visual inspection of the bathymetric maps (Figures 5, 6, and 7) shows that all three tested systems have attained similar results. There is a noticeable shallow area near the wooden pier with a steep slope of ~4 meters, relatively flat ground running for around 100 meters, and an almost 2-meter-deep pit at the end. For an easier comparison, see Figure 8. The interval of isobaths is 0.5 meters in all maps.

Cross line check

The cross-line check shows how precise the results of each system are. The lower the value, the better the repeatability of the survey results. The ECT D24S echo sounder data based on the UAV system had an RMSE value of 0.09 meters. The CHCNAV Apache 3 USV echo sounder produced a RMSE of 0.04 meters, while for the OceanAlpha SL40 with its GeoSwath 4 USV swath sonar it was 0.08 meters. These values must be taken into account when making a comparison between the systems.

Quantitative comparison between the systems

The quantitative comparison is provided for 1) direct measurements, as they are the most reliable representation of the acquired data, and 2) interpolated grid points, which usually cover the largest survey area. This comparison was made only for the UAV-based system and the CHCNAV Apache 3 USV, as the OceanAlpha SL40 USV did not acquire an RTK GNSS signal due to the unfortunate time constraints on the test day.

To analyze the datasets directly, measurement points of the UAV system within a horizontal radius of 0.1 meters of the Apache 3 USV measurements were compared. The plots in Figure 9 show the differences in the calculated elevation data. Approximately 75% of the interpolated elevation points lie within a difference of ±0.10 m, and 93% within ±0.20 m. The median of the differences is skewed to the right by 0.04 meters, which hints at some slight systematic error, possibly related to the RTK GNSS. There is a noticeable spread of data at the more considerable depths, but that area also corresponds to larger slopes at the bottom of the lake, which complicates the acquisition of accurate soundings. Other outlier values also correlate to sudden changes in the bathymetry, possibly due to floral growth.

The differences or the misfit between the common area of the interpolated grids from UAV and Apache 3 USV platform data (Figure 10) show a noticeable pattern. The largest misfit (more than ±0.5 m) is at the areas of the largest slopes, which are at the shallow end of the survey area and near the deep-end pit. Interestingly, there are very small differences where the USV was weaving during the slowdown at the deepest end. Some of the larger elevation differences are most likely due to floral growth or the different beam widths of the sensors.

Similarly to the differences between the direct measurements, approximately 68% of the interpolated elevation points lie within a difference of ±0.10m and 90% within ±0.20 m (Figure 11). The median is also similar but larger at +0.05 m. These slightly more significant differences are likely more representative of the measurement errors because there are 100 times more data points than for direct measurements.

The discrepancies in the calculated elevations could be due to the RTK GNSS horizontal positioning errors and to different beam angles, which cover varying-size patches on the lake's bottom. The sonar footprints at 7m deep beneath the transducers are ECTD24 60cm, D230 40cm, and GeoSwath 6cm. However, the causes of errors should be investigated in a more controlled test survey.

Track lines comparison

The track lines of all systems reflect the inherent differences in the systems themselves (Figure 12). The UAV is very maneuverable and performs sharp cornering, which can be important in limited space conditions. Both USVs require more maneuvering space. It is evident that the OceanAlpha SL40 is able to provide the greatest data coverage over the shortest path due to its wide swath bathymetric sonar. However, this doesn’t necessarily result in the shortest overall survey time, as the system setup and retrieval time must also be considered.

Sidescan image from the OceanAlpha SL40 USV with Geoswath 4 USV side scan sonar

The OceanAlpha SL40 USV was the only tested system capable of collecting side scan sonar images with bathymetry. The nine acquired side scan mosaics reveal details about the vegetation growing at the bottom of the lake, as well as man-made objects. For example, one of the side scans shows a sunken boat-like feature (Figure 12).

Conclusions

1. All three tested systems—one based on UAV and two on USV platforms—proved to map the lakebed well enough for a qualitative assessment of the lake's bottom.
2. The median values of control measurements show that all systems can attain a precision of less than 0.10 m.
3. When comparing the UAV and USV systems, the difference in the calculated elevation was within approximately ±0.20 m for 90% of the compared area.
4. The sounding accuracy of an echo sounder can noticeably degrade when measuring the depths of surfaces with large slopes. 
5. Possible causes for errors include RTK GNSS positioning errors and varying echo sounder beam widths.
6. Errors from sound velocity: the single-beam processing used a default 1500m/s for the sound velocity, while the GeoSwath used continually measured SV at the surface (around 1486m/s).  The lake was assumed to be vertically mixed; this assumption needs to be verified.