You are using Internet Explorer 11 or earlier, this browser is no longer supported by this website. We suggest to use a modern browser.

oXeanseeker: the neural network for fishing with less by-catch

7 June. 2019

Fishermen still find a lot of fish in their nets that they had absolutely no intention of catching. The oXeanseeker aims to reduce this by-catch drastically – and to contribute toward the exploration of the oceans at the same time.

On a dull, early March afternoon in Rungsted near Copenhagen, Marc Schiemann, Max Abilgaard, and Marc Bornefeld from thyssenkrupp and a handful of their colleagues are standing on a footbridge by the Øresund strait. They are eagerly following the movements of a torpedo-shaped mini-submarine, orange at each end and with a shiny gold middle section. It has just been lowered into the sea on two red ropes and is about to show them what it can do. The four rear thrusters start turning, and the autonomous unmanned vehicle (AUV), one meter long and weighing ten kilograms, slowly descends below the surface – still connected to the team on land by a rope as a precaution. Everything goes according to plan, and ten minutes later, the first test is over.

On the Danish coast, the thyssenkrupp project team is testing the orange mini-submarine in practice

On the stalk: Once thrown overboard, the oXeanseeker independently explores the world below the surface to find potentially catchable shoals of fish. This can make fishermen's work much easier- and, above all, reduce unwanted by-catch.

Next, the “oXeanseeker” has to prove that it can also operate reliably without a safety rope. A team member throws the now untethered mini-submarine from the bridge into the water, and immediately it begins to move in the Øresund, as intended. This is exactly what it will also be expected to do in the future – users are meant to be able to simply throw this vessel overboard to send it to its mission, which for example could be: “Analyze the composition of a shoal of fish near the ship.

Bye bye, by-catch

This will provide very valuable information for the deep-sea fishing industry, a sector that is under a lot of pressure. “By-catch,” in particular, is a big problem – despite sophisticated onboard technology and years of experience, captains can never be completely sure of the exact makeup of a shoal of fish near them. This means they still end up with large amounts of fish and other sea creatures in their nets that are not what they wanted to catch. However, they are not allowed to simply throw them back into the water, because the landing order requires the fish to be brought ashore and sold. As a rule, they are then used to make fish meal, which makes very little money. However, they are counted toward the ships’ catch quotas, so every metric ton of by-catch costs valuable revenue.

A simple idea, an effective application: The oXeanseeker is based on inexpensive systems which are able to search the dark, invisible sea independently for the most part. Therefore the submarine can find answers to important questions of deep-sea fishermen.

Explore before you fish

If things go as the thyssenkrupp team plans in the future, fishermen will put the oXeanseeker into the water before casting their nets and use it to ascertain first which fish are swimming in their vicinity. For this purpose, the mini-submarine’s bow is fitted with sensors that can be changed depending on the mission. Used to avoid by-catch, it will determine the shoal’s position with the multibeam sonar in the middle of the bow, enabling the oXeanseeker to move toward the fish independently. Once it has reached them, the light at the drone’s head will provide sufficient illumination to allow the built-in camera to film what is happening underwater. Next will come the crucial step – a neural network will identify the types of fish. At present, it is able to recognize the three most important European species – herring, mackerel, and cod. By the end of this year, it is set to learn more and is then be able to identify the ten most significant species. The artificial intelligence needs five to ten days’ training per fish species t achieve 80-percent accuracy.

This is the oXeanseeker’s view under the sea surface: Thanks to its high-tech equipment, the underwater vehicle automatically recognizes fish species based on their size, shape and individual characteristics – in this case, a swarm of herrings is passing by.

As soon as the submarine has analyzed the shoal’s composition, it surfaces ans sends its results to the ship by Wi-Fi. To this end, its stern is fitted with an antenna, beside which there is also a GPS receiver. “In the future, the oXeanseeker will be the fishermen’s eyes in the water,” says project manager Marc Schiemann from thyssenkrupp. “It will help them to decide based on factual evidence whether or not to head toward a shoal, because they will then know in advance both its makeup and the size of the individual fish.” However, the identification process will require skill and judgement, for the oXeanseeker will have to adapt its speed and lighting to allow it to get as close as five meters from the creatures without scaring them away. “Herring, in particular, like to swim away,” Schiemann reports.

Oceans full of mysteries

However, the intention is that science will benefit from the oXeanseeker as well as fishermen – after all, more than two-thirds of the Earth’s surface is covered by water, and a lot of what lies in the depths of the world’s seas is still unexplored. So it is no wonder that these ecosystems, some of which are extreme, regularly yield headline-making discoveries of new life forms. To put it in another way, the seas are home to the last great mysteries of our planet – and the little that we do know is well hidden. Although oil and gas companies, climate researchers, oceanographers and meteorological services gather huge amounts of information, it is spread across numerous databases, usually hard to access, and in many cases not publicly available at all.

oXeanpedia: Google of the world’s seas

The plan is to use the oXeanseeker to help create the world’s biggest realtime underwater database, “oXeanpedia.” In the future, climate researchers, fishermen and other interested parties could use it to find out about the current state of the oceans – for example, temperatures pH values, oxygen and CO2 levels, the distribution of plankton, and the topology of the seabed. The idea is built up a “Google of the world’s seas,” as Schiemann succinctly puts it. Alongside their primary role in deep-sea fishing, these mini-submarines could in the future gather additional data during their mission and feed them into oXeanpedia.

A promising idea turns into a successful marine innovation

The idea of oXeanpedia and the oXeanseeker came about at the end of 2016 and received backing as part of tkGarage, the internal group incubator that supports new talent with promising ideas. The interdesciplinary team includes colleagues from several of thyssenkrupp’s business areas. “Our core team comprises six people, but all in all, we have around 30 colleagues from widely differing areas supporting the project,” Schiemann reports.

This is how innovation begins: At the Demo Day of tkGarage 2017, the first prototype of the oXeanseeker was equipped with a commercially available GoPro camera and lamps fixed with cable ties. Now, the first market-ready version is in the final stages.

They have already met with a lot of interest in their conversations with fishermen and the fishing industry, and there is a big potential market for the autonomous submarine – worldwide, there are around 92,000 large fishing boats of more that 24 meters in length that could be interested in the oXeanseeker. If the project is successful, it could not just lead to greater sustainability in fishing, but also help uncover the sea’s remaining secrets.