Adding a Lighthouse

To test the new World Builder tools for Sailaway, I took it on me to create a worthy replica of the famous Fastnet rock complete with it’s lighthouse. These World Builder tools will be part of the upcoming release of Sailaway and will be available for all. Once published, the edits made with this tool will be visible for all sailors. Here’s my report on how I made Fastnet Rock and the Fastnet Lighthouse:

World Edit

All changes to the world in Sailaway are wrapped up in a package called a “World Edit”. This World Edit defines a rectangular area in which the actual edits will take place. The function of the World Edit is to encapsulate the individual changes to the world and to make it easier for the Sailaway system to dynamically load these edits as a boat sails by. The game only has to check these rectangular areas to know which ones should be loaded into memory.

Visible distance

So the first thing to do is create such a World Edit and define the rectangular area. It is best to keep this area as small as possible to prevent people from having to download edits that they won’t be able to see anyway. And Fastnet Rock itself is rather small, so you’d think the World Edit region can be small as well. But not in this case. The lighthouse is a tall structure and can be seen from a very big distance. The distance at which Sailaway renders any land is currently 20 kilometers. (may be we should extend this a bit). When the area of the World Edit touches this circle of 20 km, it will be loaded. The lighthouse is 48 meters tall and can be seen at a distance of 16 miles or 29.5 kilometer. And this means that the World Edit has to have a size of 9.5 x 2 = 19 km square. This roughly equals 10 minutes latitude by 16 minutes longitude. The maximum allowed size is 1 by 2 degrees, so we are well within the limits.

The lighthouse has a range of 27 nautical miles which is more than those 16 nm. This is due to refraction of the light by the atmosphere. But luckily the Sailaway system that places buoys and fixed lights will take care of this. So there is no need to extend the World Edit rectangle any further. The lighthouse won’t be rendered, but the light will be visible anyway, since this is already defined in the database of nautical objects that Sailaway uses.

Terrain height and sea depth

Screenshot of the mapFirst I am going to edit the terrain heights, because the entire island is nowhere to be found in Sailaway and the ocean seems to be 34 meters deep where FastnetRock should be.

I found this section of a nautical map of the region and will use that to edit the seafloor. I clicked on “Edit Terrain Heights” and in the empty list of terrain edits, I click the + button and adjusted the area. It’s always a good idea to make the area’s for the edits as small as possible. Both to prevent unnecessary and slow downloads and to increase the resolution of your edit.

 

Nautical chartEach edit will have 512 x 512 height samples regardless of the size of the area. So a smaller area will allow for more detailed terrain than a large area. If you are editing a port or an island it is best practice to add 1 terrain edit for the entire region in which you adjust the ocean floor and maybe the height of the terrain. And then add one or more smaller, high detail terrain edits to make the important bits. In this case I will add only 1 terrain edit because the high detailed rock is an object I modeled in Blender

Editing the water depth

Current Sailaway map

The current map doesn’t look anything like the real one. I am going to make it roughly the same as the nautical chart and to do this, the current shallow area has to be removed.

I do this with the tool “Level to” and set the depth to -57 meters, which is roughly the depth of the surrounding area. (the image is lighter blue than the previous, but it is actually deeper than it was. This is due to the dynamic map coloring to show maximum detail.)
The center area is left alone, because that will be raised anyway.

 

Changed water depths

Next up is copying the depth soundings from the chart to the Sailaway map. I use the “Level to” tool, set the accurate depth, pull the effect strength slider to the right for 100% effect and click once in the map to make the depth match the chart.
It’s easiest to start from the deepest level with a big brush and work towards the shallower depths with a smaller brush.

The sea bottom becomes a bit polka dot, but there is not much you can do about that. Except for using the “Smoothen” tool a bit.

 

Importing 3D model

In the tool “3D models’ I added a new object for the island, placed it roughly on the spot where the default lighthouse is on the map (I’ll adjust the exact location later) and pressed “Select Model”. Of course there is no island in the list of models yet, so I added a new one and imported my 3D .OBJ model of Fastnet Rock that I made in Blender. I manually selected the texture I created for it, because the importer does not import the .MAT material file.

Importing the lighthouse

The lighthouse itself is imported in the same way. But this model has 2 materials, because the light needs to shine at night. This 3rd material uses the shader “Self Illuminating Shader”. When this model is placed in the world, the light will be always on. When connected to a nautical object, it will replace that object and uses it’s light signature. To add an actual light to it, I set the light position with the sliders at the bottom. This will add the ugly Christmas star that you see, but that’s only in the editor so you can see where the light is situated.

Sailaway importer showing Fastnet lighthouse

Replace existing light

Select an existing nautical object to replaceAnd finally I place them both in the world by adding the 3d objects to my World Edit. It took a little tweaking to position them correctly. To replace the existing fixed light with this new lighthouse, I used “Replace nautical object” and select the lighthouse in the list. The lighthouse has started flashing now.

 

And this is the end result. When the new update for Sailaway is published, this lighthouse and all other edits to the world will be visible for everyone who sails in the area.

 

 

 

 

 

 

 

 

 

 

 


World Editing

Ideally the Sailaway world would be an exact copy of the real world, with the same water depths, the same landscape and the same buildings and vegetation. Of course this is impossible. The world is simply too big, the Sailaway Team is simply too small, and the available information is simply too scarce. We’ve decided to make more use of the enormous expertise that is present among the Sailaway sailors and intend to create a World Editor. This is what I am working on right now. I’d like to share my ideas with you and appreciate your input in case I forgot something or if you think a different approach would be better.

The World Editor will be a separate program in WebGL so it will run in your browser. Embedding it in the Sailaway executable would make the program and the user interface overly complex. The downside is that it will be an extra program to maintain and support.

There will be 3 different types of edits:
– terrain heights and water depths
– terrain texturing
– placing objects

World Edit Object

All edits are encapsulated by a world edit object. This world edit object will contain the id of the user, the longitudes and latitudes of the edited world section, whether the edit is published or not and a publication timestamp. If there are multiple world edit objects for the same area, they will be applied in the order of publication.

To allow 2 people working in the same area on different world edit objects at the same time, new height values are only stored when they have actually been changed with respect to the original height. This means you can work in the same region as long as you don’t edit the exact same location, because then the one published last will determine the terrain height for that position.

Of course there is the danger of someone pushing out weird shaped islands, drawing their name in the terrain, or making unintended errors. If this happens, you can disable those edits in the stack. If it was done intentionally, notify us and we’ll address the user to prevent future incidents.

Terrain Heights

The terrain heights are stored in a grid format that uses 2 bytes per sample. Since the range is -10000 to +10000 meters, the accuracy would be insufficient for detailed editing. For this reason Sailaway uses an algorithm where steps are smaller around 0 and bigger at greater depths/heights.

The current Sailaway terrain heights have a resolution of 30 meters. The world edit object will allow much higher resolutions, but the download size would become immense if even a small region like “The Netherlands” would be stored in a resolution of for instance 1 meter. It would mean a download size of 250 km x 150 km x 2 bytes = 75 GB. To prevent this, you can use multiple edits within the same world edit object with different scales. The best way to edit an area would be to start at a big scale and then create separate height edits for smaller regions where you need a high resolution. Here’s an example for the port of Rotterdam.
First you edit the whole region N51.91, E3.95 to N52.05, E 4.175. This region measures 0.095 degrees vertical and 0.225 degrees horizontal. The fixed grid size is 512 x 512 samples, which means 0.0001855 degrees by 0.0004395 degrees per height sample or roughly 22 by 22 meters.

This would show large saw teeth in the landscape at close distance.

To prevent this the next edit within this same world edit object could cover a smaller region.


In this case N51.955, E4.025 – N51.966, E4.055. At 512 x 512 samples, it would provide a resolution of roughly 2.5 meters. And you can go much smaller than that if you want to. The high resolution edits are only downloaded by users, when they are sailing at a close enough distance to see that amount of detail.

 

Terrain Texturing

In texture mode you can paint the terrain by using a blend of predefined textures that are optimized to repeat without showing any obvious repetition. The edits are again stored in a grid of 512 x 512 tiles. The region that is covered by that grid is determined by you. This way you can quickly paint an entire desert, and create additional detail by adding world edit objects that only cover a specific beach. This works the same way as described under the Terrain Heights section.

The file format uses 1 byte per tile. Each tile can display up to 16 different textures, blended together by alpha in 16 steps. 16 x 16 = 256 = 1 byte. If the total alpha of all 16 textures adds up to less than 1, the original terrain texture that Sailaway uses + earlier entries in the edit stack will shine through.

 

Placing Objects

Right now Sailaway reads the OpenStreetMap database to place buildings in the world. Plus we added some iconic buildings and structures at various locations ourselves (like the Golden Gate Bridge). To extend this and also allow for the creation of docks, jetty’s, dikes, sheet piling, trees, etc. it will be made possible to upload static 3D models. The required format will be .OBJ, which is very simple and straightforward and supported by most 3D editors. Each model will have only 1 material for performance reasons and .MAT files are therefor not supported. Instead, you will be able to upload a texture and/or specify a color in the editor. Transparency will be clipped by a cutout shader.

After uploading an object, you specify if it is a single instance object (like the London Tower Bridge) or if it can have multiple instances (for instance a tree). If it’s a multiple instance object, it will automatically become available for everyone to use after it is published. Once someone else uses it, you can no longer upload a new version of the same object. Placing the objects in the world is done by placing them on the map and adjusting the scale, rotation and color.


Selecting trim lines

The internal routines that allow the user to select a trim line have recently been rewritten in version 1.0.07. This article describes how it works now.

How hard can it be to click on something with your mouse right? This is done in every website or game. Clicking on buttons, clicking on items. Easy, out of the box functionality.

Selecting a trim line
Selecting trim lines in rough conditions

Alas, not in Sailaway. The boat is constantly moving, the trim lines are thin, close together and even overlapping each other sometimes. Using the default game engine technology to click on an object that is underneath the mouse pointer would make it really hard to select a specific line that is only a few pixels wide while sailing on a stormy sea.

Instead, a special routine is needed that figures out which line or interactive object is underneath the mouse pointer and highlight it. This means a lot of computations and therefor the whole process needs to run in a background thread to complicate things even further.

Before the thread is started, the location of the mouse pointer, the transformation matrices of the camera, the boat and each trim line is stored, because these are is likely to move during the time it takes to do the background computations.

First an imaginary ray is drawn from the mouse pointer in the viewing direction. Since direct hits on trim lines on a stormy sea are difficult to accomplish, we need to compute the distance between the trim line and the imaginary view ray.

A trim line is not a straight line in the mathematical sense. It is a mesh made up out of several triangles. For each triangle the distance between the camera and the center of the triangle is taken. This distance is then used to take a single point on the imaginary view ray. Next, the distance between that point (that is directly underneath the mouse pointer at the same distance as the trim line triangle) and each of the 3 sides of the triangle is computed. This is done for all the triangles of the trim line to find the closest distance.

Each line stores it’s own “under the mouse pointer” weight value which is gradually decreased every time the routine is started and increased with the reverse distance to the cursor. The closer the distance of the trim line to the imaginary view ray, the higher this weight value becomes.

After all trim lines have been processed this way, all of them have updated weight values. And all the routine has to do now is highlight the trim line with the highest weight value.

Because of this approach, trim lines no longer need to be exactly targeted by the mouse to light up and there is no absolute true or false hit flag anymore. All trim lines are evaluated and the one that has been closest to the moving mouse pointer on the moving boat during the past few computation rounds is selected.


Fast and smooth maps

Many of you may remember how the map would sometimes take a while to load or leave a white area while you were zooming and panning around. This was due to a series of downloads, data handling and image processing in a large number of simultaneous worker threads. One of the extra complexities was the enhancement of the contrast between deep and shallow water when the map was moved around.

The entire process has recently been redesigned. And here’s how it works:
The maps of Sailaway are not stored as 2D maps but as tiles with 900 by 900 height samples of the surface of the earth. Each tile is 0.25 by 0.25 degrees. These tiles are used to generate the terrain in the game and to deduct the map image.

The earth is 360 x 180 degrees, which means 360 / 0.25 x 180 / 0.25 = 1,036,800 tiles. If someone would zoom out to view the entire earth, roughly 1 million tiles would have to be downloaded and converted into a giant map image. To prevent such madness the data is also stored in copies with a lower resolution where 1 tile measures 0.5, 1, 2.5, 7.5, 22.5, 45, 90 and 180 degrees. When slowly zooming in from world size down to a detail of a specific port, all those levels are loaded and processed one by one, starting at the highest level of 180 degrees per tile down to the deepest level of 0.25 degrees per tile.

You can imagine this is a lot of data to download at run time and to increase the speed a bit we packed the top 6 levels of 2.5 – 180 degrees with the game, which is why the initial download when installing the game is somewhat big.

When you view a section of the map, it could be that you are looking at 1 tile, but as you pan the map, you will encounter a corner of that tile. Instead of 1 tile you are now looking at the 4 tiles that are located around that corner. In the old map window, we’d move 4 textures around behind the viewing rectangle when you were panning the map. In the new map window there are 4 static images layered on top of each other. Each will have the height map data of a single tile. If a pixel is outside the scope of the tile, it becomes transparent, allowing the image beneath it to show through. To the viewer, the 4 layered images appear as a single image that covers the viewed area entirely. This means that the conversion from height map data to a visual image (4 images) is now entirely done inside a shader and no longer processed in advance by the CPU.

Internal image layers to assemble the map
Map layers in Sailaway

Loading map tiles from the server is a slow process. Loading it from a local cached file is faster, but still not instantly. This means that the visible map will be loading at times when the user zooms in or pans around because the loader threads can’t keep up. Previously this would sometimes give white areas. In the new maps, there are 3 levels (of 4 tiles) and 1 base level (of 2 tiles) displayed all the time.
Basically this is a stack of layers with increasing resolution that each can transfer height map data into an image and may or may not be transparent to show the lower resolution beneath it. This way, the map may be blurry while a high resolution layer is loading, but it will never be white.

Bay of Biscay nautical map
Bay of Biscay zoomed out

Finally the contrast between deep and shallow water needs to be optimized. When you are viewing deep ocean together with a shallow coastline, the dark blue of the deep ocean may represent a depth of 4800 meters and the light blue of shallow water may represent a depth of 150 meters.

Nautical map zoomed in on Brittany
Brittany zoomed in

But when you zoom in on the coastline, the colors change. The deepest water is now 170 meters and the shallow water 0 meters. Previously the whole map image had to be recomputed in a background thread on the CPU. With the new map window all that happens is that new values for the deepest and shallowest point are passed to the shader. These 2 parameters are set every frame, which allows for the nice gradual contrast enrichment effect that can be seen now.

The cool thing about all this is that it took only a week to code, test and optimize this all, but it took a few months before the idea to use a special shader instead of CPU image processing to appear and evolve in my head. That’s game development: You are pushing to get something done in time, which then sort of works but is not optimal. And then later on when you take some distance and look at the same problem from a different perspective you see how you should have build it in the first place. Usually there is no time or budget for a full rewrite, but Sailaway is my passion and it fully deserves a little extra effort every now and then.


Developer Blog – Improving Sailaway’s Ocean Height Map Accuracy

Hello Pioneers,

This is Richard Knol, the developer of Sailaway. I’ve been behind the scenes more recently as I have been busy working on some big changes to Sailaway. The biggest of these tasks has undoubtedly been improving the accuracy of the map data so I thought I’d give you an insight into this process in my first developer blog!

Sailaway’s Map Data

The Sailaway maps are all rendered in real time based on the 3D model of the world that is stored on the Sailaway servers. This model was made with data from NASA and a few other sources and it holds the height of the terrain or sea floor at any location in the world.

There are a few issues with that:

  • The NASA data only provides the height of the land, not of the sea floor. It also contained measurement errors and it has a resolution of about 10m2, 30m2 or 240m2, depending on the area.
  • Most of the underwater height maps I found only have a resolution of about 240 meters
  • The underwater data I found and could use is inaccurate and incomplete with regard to coastal waters
  • Not all data sources use the same corrections for the fact that the earth is not a perfect sphere
    So all in all a rather depressing list of issues when you want to run a worldwide sailing simulator.

Luckily there are nautical charts available that give more accurate info about water depths and much more. But charts are 2D and Sailaway uses a 3D model. Basically, a chart tells you that there are regions colored light-blue where the water depth is 0 – 10 meters, and regions in another shade of blue where the water depth is 10 – 50 meters. But what happens in between?

Following that data strictly would turn the entire seafloor into a giant staircase. Additionally there are soundings on those maps that give the exact water depth at a certain location. But they don’t say anything about if that point is a peak with a steep slope or a single measurement on a horizontal and flat surface. So here’s what I did to improve the Sailaway 3D model with the US Electronic Charts as provided by NOAA…

Deciphering the data

First the charts were downloaded. All 1280 of them. Next they needed to be read, or rather deciphered. The format in which they come is called S-57 and it was defined by the International Hydrographic Organization decades ago. It dates from the days when computer scientists tried to write data dictionaries and press various record types into a highly efficient binary format. A time when JSON and XML were never heard of yet! So I deciphered the gibberish and extracted the data that I needed for Sailaway and pushed that into a JSON format. This was several weeks work, but in the end I had a neatly organized collection of 1280 JSON files and
1280 small overview images of the actual charts.

The US charts come in 6 levels. At level 1 there are large overview charts, building down to level 6 where a specific port is shown in high detail. I wrote a program that would process each of these maps starting at level1. It would download a section of the Sailaway 3D model, match the Sailaway depths/heights with the data of the chart and make the necessary corrections.

Unfortunately that technique didn’t work, because it would create a staircase seafloor after the depth regions of the chart are applied. And it would litter the seafloor with spikes or pits at the locations of the soundings in the chart.

So instead I used the charts to verify our own data and make gradual adjustments where needed. If the chart says the seafloor depth is between 20 and 200 meters, and Sailaway has 20.5 or 198, I leave it untouched. If the Sailaway model says 18 meters, the floor is lowered and also the pixels around it are lowered gradually to smooth the sea floor. This way I kept as much of the shape and structure of the seafloor intact as I could.

Then there was the problem of turning land into water or water into land when the coastline would not match the chart. Turning water into land without any information on what this land would look like will result in ugly terrain. There is no fix for this. Sorry.

Implementing the data

So now that the algorithm seemed to work (again a few weeks work) I could put the computer to work. Starting with Level 1, here is how the numbers look:

The Sailaway model is made up of 0.25 x 0.25 degree tiles. That means (360 x 4) x (180 x 4) = 1,036,800 tiles. The tiles have a resolution of 900 x 900 pixels. Each pixel is represented by 2 bytes. That means 900 x 900 x 2 = 1,620,000 bytes per tile. Times 1 million tiles is 1.5 TB of data, or 800 billion terrain heights.

To process 1 chart of Level 1 (typically a region of 20 x 30 degrees) took about 2 days to compute.

In the meantime, I read and imported all the buoys, lights, rocks, shipwrecks and other vital information from those charts into the Sailaway database. This resulted in 120,000 new objects all along the US coast.

So my computer was humming and after many days, when it was already processing the charts for Level 3, I discovered a stupid mistake. Imagine a tiny volcanic island somewhere. A beautiful thing in the Sailaway maps, where you can see the slopes and the crater clearly. But this tiny island is not big enough to have made it on the Level 1 overview charts. Instead it falls in the big area indicated as 200-2000 meters deep. After processing the chart, the entire Island has vanished and has been turned into 200 meter deep water. Of course when the charts of level 3 are processed the island is reinstated, but all of it’s characteristic volcanic shape is gone. It is now a flat surface a few meters above sea level.

As a result, the algorithm had to keep a copy of the original shape at hand. And every time it wanted to alter the current height of the land or sea floor it had to refer back to the old data and see if it could use any of the original height info to recreate some of the original shape of the terrain.

The algorithm was started again from scratch. And after many weeks it is now ready with correct calculations. Yay!! The result is a more accurate seafloor, and a very ugly coast. The last is because after processing the 2D charts, a pixel is either land or water. Imagine a nice coast with some rocks, some distant mountains, small pebble beaches between the rocks. The chart may have drawn a line over that coast, dividing it into land on one side and water of at least 20 meters deep on the other. The result is a coast of steep cliffs, that plunge into 20 meter deep water. The pebble beaches are gone, and because of the size of the pixels the coast looks like a horrible collection of zigzag cliffs.

To fix this, or at least make the coast look somewhat more realistic, is my challenge for the coming weeks. Wish me luck!

Ongoing may improvements

I started work on these map improvements early August. It is now the end of October and I am not done yet and this was only the US coast! The European and Australian coast will be even more challenging. But it is great fun to work on this and see the maps improve. Sailaway becomes more mature with every step we take.

I see players asking for more detail in the terrain often; for ports & marinas, and even trees and buildings. These would be awesome to have, I fully agree. But when you realize that the current work in progress coastline, made up of 10 x 10 meter pixels, already measures 1.5 TB of data. Just imagine the amount of data required when this includes the rest of the world! All that data has to be collected, processed, checked, maintained, refined and find its way to your computer somehow… don’t forget, Sailaway is a persistent online world.

The world is a really big place and I hope that the process behind how I implement this accurate data has become much clearer. Please do not expect the same level of terrain detail in Sailaway that you see in other games. They do not have to carry the weight of the world with them.

Thank you to all of you who have joined us in early access and support this project! I’m sure you can see from the process above, this is a passion project for me and I’m thrilled to see so many people sailing the seas I’ve created.

– Richard Knol, creator of Sailaway.

Back to Robinson’s Island

We set out the first long distance race in Sailaway as an experiment. To see if people would be engaged enough to undertake a longer event. And to test the program, to see if the waypoint system worked, see if the boats would still perform well when the user went offline and to see if everything else would work ok.

It turned out to be so much more than that. Completing the race took me 35 hours. Halfway after 17 hours I was only fighting Alvaro over 3rd and 4th place with only 3 seconds between us. Nrs 1, 2 and 3 were 10 minutes ahead of us. So close. With identical boats it all comes down to sailtrim and tactics.

Ker2x chose a southerly course upwind to Selkirk island and that turned out to be a wrong choice when the wind turned from South-West to West. It put him in 6th place.

After we rounded Selkirk island and headed back to Robinson’s Island, 2 people took a bold decision and sailed further south. When the wind turn to North-West it brought Guiseppe a solid lead and Ker2x climbed back from 6 to 4th place. Bad luck struck Guiseppe: Only a few hundred meters before the finish line his boat ran aground in the treacherous and shallow waters while he himself was offline. Frank Black, Ezra and Ker2X and sailed passed the poor guy and ended in 1, 2 and 3.

During the race you could taste the friendly competition over the VHF chat. Everyone was joking around, willing to help each other and at the same time very keen to beat everyone. It was so much fun.

 

  1.  1d. 11h. 02m.  Frank Black
  2.  1d. 11h. 05m.  Ezra
  3.  1d. 11h. 34m.  Ker2x
  4.  1d. 12h. 06m. Alvaro
  5.  1d. 12h. 52m.  CocoRhum
  6.  1d. 13h. 20m.  DK1
  7. 1d. 14h. 08m. Groove
  8.  1d. 14h. 10m.  Tsyal
  9. 1d. 14h. 26m. Mr Mustard
  10. 1d. 18h. 39m. Guiseppe Mazei
  11. (not a real) Captain Philyfil
  12. DNF  Skipp
  13. DSQ  Richard Knol

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save


One day to go

Today is April 24th 2017. One day before Sailaway becomes available to the public after 2 years of constant development.

Sailaway’s main ingredient is passion. It emerged from a passion for sailing and for sharing that experience. Sailaway isn’t just a collection of logic and data and physics. It has been written, changed, rewritten, redesigned and tweaked until it became a true sailing experience.

I hope that people will learn to sail, improve their skills, or just have fun on the water. But most of all I hope that people will catch the same fever that I have had since I was 10.

Enjoy!
Richard Knol

Save


Keep updated with Sailaway news
Direct via your inbox!

Subscribe to our mailing list and keep informed about all things Sailaway related!

Thank you for subscribing.

Something went wrong.