MUSV J-5: Are you ready?
Tocaro Blue clients are using our products in their perception layer on MUSVs, and will be submitting their results of the US Navy’s J-5 Test at the end of September. As this deadline approaches, we are persistently improving our capabilities on these tests, including completing an internal J-5 Full Perception “Circles PSA Test” ourselves.
In our testing, we used an open-array, X-band radar paired with ProteusCore™. ProteusCore is machine learning radar processing software that automatically detects, classifies, and tracks radar contacts for enhanced perception and situational awareness.
We achieved a 99.8% “Track Probability” for the Medium Contact vessel, and a 99.9% for the Small Contact vessel. Sound too good to be true? Keep reading for test parameters and details...
Test Vessels
J-5 is intended to be performed on an MUSV (120-180ft) which typically have a 30+ ft (10+ m) height-of-eye for the radar. On our relatively small data-collection boat, a 30 ft long XO EXPLR, with a custom sensor arch, the radar is about 17 feet off the water line. This means our visual horizon is reduced dramatically compared to the expected radar height on a MUSV-sized vessel. We used the Furuno SCX-20 Satellite Compass for the Test Vessel's position and heading.
We also don’t have easy access to properly sized Small and Medium Contact vessels. The J-5 document calls for a 7-12m (23-39 ft) Small Contact, and a 15m to 50m (49-164ft) Medium Contact. We used a 32 ft fiberglass, Grady White center console, for our Small Contact. Our friends Towboat US provided us with a 30ft aluminum support craft as a surrogate for the Medium Contact vessel. GPS breadcrumbs were recorded using the vessels onboard chartplotters since AIS transmitters were not available during testing.
Tocaro Blue's 30 ft XO EXPLR acted as the Test Vessel. A 30ft aluminum vessel from Towboat Us was the Medium Contact vessel and a 32ft fiberglass Grady White acted as the Small Contact vessel.
The J-5 “PSA Circle Test” with full perception was performed using a Furuno USA, Inc. DRS25ANXT (6 foot open-array) radar. Standard ProteusCore configuration settings for tracking were used during the test. The Test Vessel and the Towboat USA vessel did not have autopilot, so the circles are manually driven by human captains. The Small Contact vessel did have an autopilot and used it for a more precise circle maneuver. We left the radar range fixed and ensured it covered the entire operating area.
Scoring Criteria
We read J-5 to intend that a successful detection is when a measurement (from radar or otherwise) is used by a tracker which has been determined to be fully acquired and valid. While the J-5 specifies a maximum 1.0 second logging timestep, we are presenting results on a 1.25 second interval to match the 48 RPM rotation rate of the Radar. In the timelapse video at the bottom of this blog, “trackers” appear as boat shaped icons. The J-5 test criteria considers only True Positive (TP) detections in the “Detection Probability” metric it defines but does not penalize False Positive (FP) detections nor radar Swops.
To demonstrate the natural ability of the ProteusCore software, the ProteusCore tracking was not tuned to achieve better TPs in exchange for higher FPs – we use the same MOTA-tuned tracking system that is delivered to all users as described in our product documentation.
Sea and Weather Conditions
The test was conducted beginning at 1:00pm US Central Time on Tuesday, August 4th, offshore and Southwest of Pensacola, Florida. Sea State 1 (Beaufort Scale) was reported for the duration of the test, and it was a clear day with no rain or fog.
ApolloCore™ Configuration App showing footage captured during the J-5 test.
We used our Teledyne FLIR Marine M364C to capture video throughout the test.
Test Condition Summary
The added difficulty of using a much lower mounting height for our radar, and smaller than specified contact vessels, were slightly offset by fortunately calm sea states and nice weather. The test with a full size MUSV would be expected to use a much larger “Medium” Contact vessel, and a radar mounted at nearly twice the height of our 17-foot mount. The Furuno DRS25A-NXT is a solid example sensor for the J-5 test and is likely to used by many of the MUSV teams for their X-Band radar. The specific configuration from J-5 Circles PSA test performed is the PD1 test: both contact vessels start at the West-most point in their circles, the Test Vessel holding 25 knots to represent the MUSV expected “90% RPM” speed. The test was conducted for 3 hours.
ProteusCore Performance Summary
ProteusCore achieved a track probability rate above 99% for both the Small and Medium Contact vessels for the duration of the three-hour test. Competitive MUSVs will often have a mast or other equipment which obstructs the X-Band visibility in some sectors – our testing with the Radar mounted in a prominent position showed high detection rates in all bearing sectors. The timelapse video shows ProteusCore rejecting sea clutter and radar main bang, and correctly not tracking low flying aircraft. Additionally, ProteusCore also tracks vessels not participating in the J-5 test, all without any direct intervention or constant radar tuning from an operator. Shoreline and channel markers are in view of the radar during the test creating additional complexity and noise, but are handled appropriately by ProteusCore.
Your perception team deserves the best tools available to deliver strong J-5 Test results for your MUSV. Reach out to us today to learn more how ProteusCore and ApolloCore can enhance your MUSV's perception stack!
Timelapse of the ProteusCore Configuration App during the J-5 test showing the persistent tracking capabilities of the Furuno Radar + ProteusCore radar processing.
Resources
Learn more about Tocaro Blue's ProteusCore Perception Software --> Download Product Summary
Learn more about Tocaro Blue's ApolloCore PTZ Camera Perception Software --> Download Product Summary