technologyShark Spotters: Man and Machine 

South African/Swiss Partnership Pioneering New Automated Shark Spotting Technology In Cape Town

Sharks and humans are animals with a reputation for not getting along. So much so, that this long-familiar strife has sparked the invention of various so-called shark attack prevention methods and mechanisms, many of which (despite their popularity) still have questions to answer concerning their effectiveness.

Meanwhile South Africa’s Cape Town, a well-known great white home ground, has for a long time relied on their “Shark Spotters” – a programme whose terse framework combines the use of human shark spotters and exclusion nets, research on indigenous species, and public education – not only to keep beachgoers safe but also to improve this stereotyped human-shark relationship.

“The Shark Spotters programme reduces the spatial overlap between people and sharks; essentially stopping sharks and people from sharing the same space. We do that with a spotter up on the mountain – a trained observer who uses polarised sunglasses and binoculars to scan the ocean. If they see a shark, they radio down to the person on the beach to call people out of the water. It’s a very simple system but it has been very effective.”
Sarah Waries, CEO of Shark Spotters

According to Waries, one of the things that makes this particular programme so effective is the fact that, unlike the popular but dubious aerial patrols implemented around Australia which only scan the beach for a small period of time during the day as they move up and down the coast, the spotters are up on the mountain from sunrise to sundown, 365 days out of the year.

“We have been operating now for 15 years, every single day. Since we started [the programme] we have had 2258 shark sightings at our beaches, and out of these we cleared people out of the water 52% of the time. So if the shark is in an area that doesn’t pose a risk to people, we won’t call them out of the water, only monitor the shark’s movements. In other words, it’s sustainable, doesn’t resort to lethal control of sharks, but is also effective.”

As effective as it may be, this method of shark spotting still comes with limitations: the need for elevation above the beach (being high enough to be able to see sharks coming at a distance); possibility of human error due to fatigue which, despite the intensive training and commitment from the spotters, is ever-present whenever someone sits alone looking at the sea for 5 hours at a time; environmental conditions (too much glare, turbid water, strong winds) which in turn impair the spotter’s ability to see a shark.

In the search for alternatives to enhance and improve their safety service, the programme’s gaze turned towards technology, more specifically the way a combination of computer vision and video cameras can assist with animal spotting. Although drones tend to be one of the most popular vehicles on which to implement this kind of technology, in the case of shark spotting in Cape Town there are limitations: aspects such as battery life, a drone’s inability to fly with strong south-easterly winds (something Capetonians, especially those who live around False Bay, are very familiar with), their small field of vision when compared to that of a human spotter, high costs and legislative restrictions, have all combined to prove it to be a rather impractical solution. This led the Shark Spotters to look at fixed-camera solutions – which could be placed on a pole or tower (without the need of natural elevation) and can be operated remotely – as a means to improve the accuracy of the spotting.

On February 11th, Shark Spotters hosted an event at the Kalk Bay Shark Education Centre to officially announce their partnership with PatternLab SaRL (a Swiss company specialising in applied machine learning) and their work to pioneer the first automated shark spotting system.

The collaboration came about after Dr. Krysztof Kryszczuk, founder of PatternLab and a kitesurfing enthusiast, got in touch with Waries during one of his visits to Cape Town. He wanted to find out more about the programme, eventually sparking discussions on how to develop a system of installed video recording cameras that could aid the shark spotters in their job. In other words, they wanted to use recorded images to build an algorithm that would teach the software what is a shark and thus allow it to detect one automatically.

“The way a computer vision system works, in a nutshell, is that we have to collect and process video images, chop them into frames, and then analyse every frame one by one. We then need to do an annotation-segmentation, meaning that we have to annotate ‘Is there a shark or not?’, ‘Where is it?’, etc., as well as segment the image by removing the irrelevant part until there is enough data to teach the machine what does a shark look like when it shows up.”
Dr. Krysztof Kryszczuk, Founder of Pattern Lab

A key reason to record videos as opposed to taking stills is that a framed image of a shark might look very similar to that of a piece of kelp, for instance. When that is the case, they have to teach the computer system to read the unique movement pattern of the shark in order to identify it (e.g. a kelp-like dark shape that is moving in a steady line against the waves too fast is probably not a piece of kelp). And this is where shark spotters’ knowledge of shark behaviours comes in.

After a year of attempting to collect data by waiting for sharks who weren’t showing up as often as expected, they thought of a shortcut and designed a real-size, 2m decoy model to be towed across the bay, following the previously observed movement patterns of sharks, so as to gather enough information to build a computer vision system. Pretty quickly they understood that there were factors in the shark spotting that were intuitively known to the human spotters (the way the wind changes the surface of the water, illumination, cloud cover, etc.) but hadn’t been properly evaluated to determine the reliability of the camera. Consequently, by quantifying these factors, they can then segment the images accordingly and disconsider the parts where it would be nearly impossible to make a reliable shark detection, thus more accurately developing the algorithm.

For the next 18 months, the research project, funded by a Eurostars (an international platform funding collaborative innovation that is administered by the Department of Science and Technology in South Africa and InoSuisse in Switzerland) and with the help of Institute for Communities and Wildlife at the University of Cape Town, will collect both data and feedback in order to evaluate how to build the most accurate automated shark spotting system. In the future they hope to be able to use all this data and the knowledge gathered over time to predict what a shark’s movements will be, as if it were spotted by the human observer, in a collaboration between man and machine.