Science & SkepticismPeeling Waves 

Using Remote Camera Systems and Machine Learning to Monitor Surfing Wave Quality

Surfing has, even if inadvertently, drawn global attention to coastal sites with surf breaks. Needless to say, this exposure has had adverse effects. But it has also helped shine a light on the immense value these resources have to our economies and well-being, as well as the fragile and indispensable ecosystems they host and abut.

Monitoring and establishing natural baselines of surfing resources is the most effective way of ensuring they are sustainably managed. Yet the subjectivity of surfing, together with the dynamism of the ocean and a lack of physical datasets pose challenges to developing quantitative measures that would assist in their preservation.

A paper presented at last year’s Australasian Coasts & Ports Conference and recently made available online outlines a research programme aiming to address this issue. The team installed a Remote Camera System (RCS) to monitor the surf break of Waikeri (Manu Bay), on the west coast of Aotearoa New Zealand’s North Island, collecting 1200 images every hour of daylight every day from May 2017 and June 2020. Some of the images were used to train a machine learning model to automatically detect wave-breaking patterns so they could later calculate each wave’s peel angle – the angle between the trail of broken white water and the crest of the unbroken part of the wave – one of the most important parameters used to define the quality of surfing waves. Results showed a relationship between peel angle and tide, wave height, and period. This allowed the team to establish a medium-term quantitative baseline of surfing wave quality for Waikeri, taking them one step closer to developing a standard environmental baseline for surf breaks.

https://youtu.be/H95JSH0BJwA

Surf Simply caught up with Edward Atkin, author of the paper and co-director of eCoast, a New Zealand-based marine consulting and research company,to learn more about the study and hear his thoughts on how technology can help manage surf breaks sustainably.

You have been involved with surf science since 2005, and as an environmental consultant, have focussed on surf break management, working with organisations such as the International Association of Surfing Research, Aotearoa New Zealand Association for Surfing Research, Surfbreak Protection Society (NZ), Save The Waves and Surfrider Foundation. What draws you to this topic? And what are some of the main things you’ve learned from your research so far?

Although they may not describe them in the same way, surfers are generally acutely aware of and continually assess a broad range of phenomena in the coastal space, and further afield, from deep ocean swells to activities in the watershed, sediment transport and local wind patterns. These innate observations, combined with prolonged, immersive exposure to the natural world, when compared to the average coastal environment user, put surfers in a unique position as kaitiaki, or guardians, of surfing resources. The key component is having an understanding; one of our most consistently used tag-lines is “You can’t sustainably manage resources you do not understand.”

Personally, I took the innate understanding a step further after being inspired by Multi-Purpose Reefs (MPRs) that incorporate surfing amenities with ecological enhancement and shoreline resilience. My first undergrad project considered my local reef break by mapping it and physically modelling waves, then proposing small modifications to improve surfing wave quality. My Master’s thesis considered the MPR constructed at Boscombe in the UK; it again looked at surfing wave quality but also the shoreline response. These projects provided me with the capacity to technically analyse a surf break. After moving to New Zealand I worked on many projects that considered surf breaks, but one of the most valuable experiences in being drawn to the topic of surfing resource management is my position as a technical advisor to a number of surf conservation NGOs and exposure to a range of projects associated with surf break impacts. The capability for technical analysis of surf breaks lends itself very readily to applied surfing resource management.

Your latest paper, Machine-learned Peel Angles for Surfing Wave Quality Monitoring, examined data collected from Waikeri (Manu Bay) and was the first of its kind in how it combined RCS and machine learning to monitor and evaluate surf wave quality. How did the idea for the research project come to be?

Peel angle, which relates to the speed at which a wave is breaking along its crest, is one of two metrics we use to quantify surfing wave quality, and arguably the most important. We have been using cameras to monitor the coast for a few decades now and the technology lends itself to monitoring the dynamic surf zone. A few years back we set up a number of cameras in New Zealand with the specific aim of monitoring surf breaks and characterising them, with wave quality being one of the objectives. The cameras were part of a larger research project with the overarching aims of developing guidelines for surfing resource management and establishing environmental baseline datasets. Given the sheer quantity of data available through camera monitoring systems, and having previously manually annotated and processed uncountable images to extract peel angle values, I knew we would want to develop an automated system.

Early trials of applying machine learning to the problem didn’t go so well, but I think that mainly fell over because we asked a non-surfer to explore the idea and without the innate knowledge of the surf zone it was difficult for them to pick up on the nuances of wave breaking. We did create a computationally heavy algorithm for tracking break points, which was in itself a great achievement; however, deploying it in the field would not have worked so well. The research project was wrapping up but I was convinced that the problem lent itself to machine learning, so I dived in and started training models to detect what we know as the ‘pocket’, or instantaneous point of breaking, but also where the crest was orientated as well. The overarching research project was completed, but I still carried on and it became a labour of love.

Thinking of the processes involved in the above paper… Could you please describe what a day in the lab looked like?

We use a subcategory of machine learning called computer vision. The model is trained by providing it with images that have been annotated with the objects you wish to detect. Most people will have seen the classic movie or TV drama using this, where they automatically track people with security cameras. To get the model to be able to identify the desired objects with some accuracy, the training set needs to be quite large, in the thousands, and ideally another order of magnitude. This is painstaking and time-consuming. Especially when no one has even tried the type of application before. You must look at each training image (maybe 10,000) and identify each instance of the object – in this case, waves – using pixel numbers. Again, very time-consuming, not very glamorous. Our approach to training, which will be published later this year, considered a full range of parameters, including how we think about identifying the breaking part of the wave. The annotated images are used to train the model on what to look for and once the user is satisfied with a model outcome, it can be deployed on unseen images. The model we use remains accurate despite undertaking detection analysis in milliseconds, basically in real-time.

For the analysis in the above paper, we had around a million break points. Post-processing involved setting up a wave-tracking algorithm; this took a while to develop but ultimately runs relatively quickly. This is achieved through coding and automating commands that manipulate the data. Having the break points grouped together as waves the peel angle could then be calculated. Because the peel angle tells us about the speed at which the wave is breaking it also gives us information about the degree of difficulty and usability. Peel angles of zero are closeouts, and those of 90 degrees mean the wave does not break along its length but crumbles perpendicular to the wave crest, ideal for learners.

So, what did the development and processing look like? A lot of desk/computer time!

In your opinion, what were the most surprising findings and observations of this study?

The most surprising was the small area where no wave breaking was detected. At first, I thought I had made a mistake. But the relatively quiescent zone on the point break, highlighted by a lack of break points, aligns with a persistent depression in the seabed which is shown in bathymetric data collected across the entire surf break area.

In terms of surprises from a surfing wave quality perspective, there weren’t any as Waikeri (Manu Bay) is one of the most studied surf breaks anywhere; we also live here and I surf with my kids there. There was a sense of relief though, that this ambitious approach aligned very well with previous findings and the more general, common knowledge and anecdotal evidence. The beauty is that this type of baseline data set lacks subjectivity and can be relied upon for effective management and resolving disputes.

How do you see this combination of Remote Camera Systems and machine learning being used in surfing in the future?

There are other groups using machine learning to look at resource usage, so they’re tracking people; another group tracks waves but not in the same way and they have looked at things like nearshore morphology. It sounds corny but the potential is endless, and I’m only referring to the computer vision side of things, that is using cameras. Machine learning expands well beyond computer vision and is already being used in many ways within the research space. In a couple of research papers, I have hinted at how this particular application could be used in real-time, that could be in surf competitions where judges get a live feed on peel angle so they get a sense of the degree of difficulty, or maybe just as a broadcasting gimmick. The forecasting sites all seem to have surf cameras; well, at many spots anyway. It would require some work on the cameras, as they won't be set up correctly from a technical point of view, but those surf cams could show peel angle, or just degree of difficulty, in real-time…it could just become a proxy for real-time quality, especially if an environmental baseline has been established.

What are the main challenges of using this combined approach? What sort of limitations did it present to the research?

You need a camera and somewhere to install it, a power supply, although solar is fine, and then some kind of connectivity. So, there is an overhead to all those things, but the reality is they are all becoming cheaper (well maybe not mains power!). Training is time-consuming and the main challenge. The model presented in the published work is for Waikeri; that doesn’t mean it won’t work elsewhere, but the model is trained on images with the same field of view. Training at other sites would be just as involved. However, if I could train at lots of different sites, then those training datasets could be used to explore the training of a generic identifier that could be deployed anywhere. I have a longer-term vision of having a group that all contribute. Currently, in the computer vision world, there are generic, baseline datasets that are used to train models; they have annotated images with things like apples, cats and bikes. They are a great starting point for anyone trying to train a model for more everyday type detections. Identifying the breaking part of the wave is not everyone’s cup of tea, but if a community could create a baseline training dataset that would make it easier for more people to deploy this type of technology. So, the biggest challenge is scalability. My hope though is that I can get more people on board.

I’m interested to hear more about Aotearoa New Zealand’s role as a world leader in surf break management. In the introduction, you cite that surfing resources are recognised by the statutory instruments to the point of becoming the focus of coastal planning nationwide, highlighting that the 17 Surf Breaks of National Significance (SBNS) occupy a total of 4.5km (approximately 0.0003 %) of the country’s coastline. You also mention the importance of the Resource Management Act (1991) and the New Zealand Coastal Policy Statement (NZCPS). Could you please explain how these legislations work in practice?

In Aotearoa New Zealand, the Resource Management Act 1991 (RMA) is the primary piece of legislation when it comes to matters of the environment. There are several national policy statements issued under that RMA, one of which, and the only mandatory national policy statement is the New Zealand Coastal Policy Statement (NZCPS). The NZCPS is prepared by the Minister of Conservation, who heads the Department of Conservation. Subnational authorities, like regional (this would be state in some countries), district or city councils are charged with developing planning instruments that adhere to the NZCPS. Prior to 2010, a concerted effort was made by the surfing community in Aotearoa to have surfing recognised within the next revision of the NZCPS. This was realised in Policy 16: Surf Breaks of National Significance. There are several other policies in the NZCPS that also associate with surf breaks. The outcome is that surfing is considered when planning around the management of coastal spaces is being undertaken.

Why is it that the NZ government is concerned about surf break management? And what could other countries learn from this stance?

I’m sure there are people in the ANZ government who are concerned about surf break management, but not as much as the end users. The inclusion of surfing in ANZ policy can be largely attributed to a motivated group of surfers who were exposed to historical mismanagement issues that led to the loss of integrity or complete destruction of surf breaks. Any authority with a known surfing coastline should be concerned about surfing resource management. Surf breaks have been shown time and again to provide benefits well beyond the immediate gratification of riding a wave. The most palpable I guess for people that don’t surf and/or those in decision-making roles is the economic benefits. The same goes for other resources, but it would be mad to allow resource degradation when part of the economy can be attributed to natural resource use through tourism.

Having spent a fair amount of time on the road attending conferences and discussing the learning from ANZ elsewhere it is clear that other countries can learn from the experience in ANZ. That’s not to say that we can't learn from them at the same time, though. In Peru, they have taken surf break protection to the state level. How it is administered at the local level is more complex, but it certainly leaves little room for interpretation, which isn’t the case in ANZ where authorities can interpret the guidance of the NZCPS in different ways.

Along those lines… In the conclusion, you mention that the approach used in this research can be replicated at other sites to aid in the sustainable management of surfing resources. What would be some of the benefits – both for resource management and for surfing as a sport – if all major surf breaks across the world were monitored?

Establishing environmental baselines is the most important component of effective resource management. It comes back to the line in my response to the first question: We can not manage that which we do not understand. Knowing what is there in the first place is critical to assess what the potential impacts of activities in the coastal marine area look like. The long-term benefits are that if surfing resources are effectively managed then we will still have places to go surfing! The sport of surfing won't be going anywhere fast if wave quality is compromised everywhere.

The right type of monitoring could benefit surfing performance and crowding issues. I have explored the concept of remote water quality monitoring from the same systems so that health and well-being in the water can be tracked. Quantifying and building relationships between a range of environmental parameters provide quite compelling narratives that are useful for issues such as climate change and land use.

Can you please talk a little bit about the conference you recently attended in LA and the work you’ve been doing to disseminate these management guidelines for surfing resources across the world?

A couple of years back I wrote a paper with three guys from the US looking at the utility of the Management Guidelines for Surfing Resources outside of Aotearoa and what mechanisms exist within a Californian setting that may be of use for applied surfing resource management. The article was published in a journal called Shore and Beach, which is administered by the American Shore and Beach Preservation Society (ASBPA). Because of that publication, and I guess because I provide peer reviews for articles in the journal, I’m on the mailing list for the annual ASBPA conference. The call for abstracts for the conference includes surf-related studies, and whenever I see a conference with surf-related research being accepted, I sign up. I double-checked with my co-authors if my monitoring type of work would be of interest to the ASBPA conference. The answer was yes, but the opinion was that our Shore and Beach paper would be a good fit. In terms of disseminating the guidelines, I probably could push it a bit more, but they are ANZ-centric, so a lot of the content isn’t relevant overseas. I have plans to develop more generic guidelines with an international team. The more generic guidance could then be relied on by countries that have not yet set up their own national guidance. I think that would be a good help, especially when it comes to really high-level education. As surfers, we forget that other people don’t surf and they simply don’t see the value in a stretch of coast in the way a surfer does. Getting some clear and consistent messaging and objective sources would provide value in so many different ways.

What other technologies or frameworks are you excited about when it comes to surf break management?

Monitoring is key to effective management, so the camera work is super exciting and really one of the best tools we have. The more breaks that are monitored, the faster we can expand our knowledge of surf break dynamics and what makes them tick. Combining the camera work with GPS watches and numerical modelling of wave crests in a 3D space (computational fluid dynamics), so that wave shape can be considered directly, is the pinnacle of surfing wave quality assessment. You learn so much more from that though when collecting the data required. There are the cultural, social, accessibility and usage components that all tie into effective surfing resource management. Obviously, bathymetry is key to a surf break, and at beach breaks, bars and points can change; we can pull the bathymetry using camera-based methods. This doesn’t circumvent the hydrographic surveying but compliments that type of monitoring by providing a higher sampling frequency, at a lower cost and less labour.

Having standardised methods for surf break assessment, something that the surfing research community and surf scientists buy into and get behind, is something I’m very passionate about. There have been cases of people playing surf scientist, with little to no formal experience or training, and combined with a lack of understanding within consenting authorities this can lead to conflict with surf break users and potentially detrimental impacts on the resource. This is, in part, why we put together the Guidelines for Surfing Resource Management. The guidelines are very Aotearoa New Zealand-centric, and while we have shown they have utility outside of ANZ, the next step is to produce a more globally applicable set of guidelines that can be leveraged where a local version is not available. This is certainly something that I get enthusiastic about because there is a clear requirement.

As an environmental consultant, what are your views on the future of surf break management and what roles it will/can play in the broader context of climate change?

There hasn’t been a whole lot of work that considers climate change implications on surfing resources. A good friend of mine, Dr Dan Reineman, has looked at the perceived impacts of sea level rise and how it might drown some surf breaks and create others, but focussed on California and based on anecdotal evidence. There is more to come in that space so look out for that in the coming months. There are also several studies, largely associated with the IPCC, that indicate changing wave climates, like increases/decreases in wave heights in certain regions, and the intensity and frequency of storms. These could be leveraged to undertake regional-type assessments, should they be required. You could also do broad assessments to consider a surf break's adaptability. Beach breaks and bars have the capacity to respond morphologically to climate change and SLR [sea level rise], although that’s not to say the surfing wave quality won’t change, whereas reef breaks are static and could be potentially susceptible to “drowning” – but it all depends on the bathymetry and subaerial space. So really, studying climate change effects on surf breaks on a case-by-case basis is likely to give the best insights into impacts.

You write that “surfing resources have a range of values and contribute to the natural character and landscape of the coastline, have high recreational, cultural, and amenity values, and provide a distinct sub-sector of the tourism industry.” In your opinion, are surfers doing enough to safeguard and/or improve surfing resources? What are some ways in which they can help?

In answer to the first question, probably not. I think surfers tend to have a good grasp of large sustainability issues and have some type of environmental moral compass. Buy-in from the surf community to surf-based conservation groups would probably help. As a technical advisor to several groups, it is clear they are underfunded. Yet they are generally the ones who kick up a fuss when a local surf break might be compromised.

I don’t think it is that hard for a local community group to set up a monitoring system with the right guidance; that would be a cool thing to see. Another great, individual data stream for monitoring is the GPS watches. We have been using and analysing the data from surfer GPS for years; this is something very easy for most surfers to get into. We put so much value in the surfer GPS monitoring capability that I have looked at developing a dirt cheaper alternative to the products currently available on the market. That’s a slow burner as I’m not a product developer, but the prospect of enabling a small army of data collectors and tying it into a citizen science project that is community–surf-break–driven is exciting. I think people could get into that.

**********

The author and Surf Simply would like to thank Edward Atkin for his assistance with the article. Jump on eCoast’s website to learn more about their work or follow them on Instagram and Twitter to keep updated on their latest research.