Atmospheres

When I started this project, I didn't think I'd be spending much time simulating air, but here we are. My planets now have atmospheres:

I looked at a number of atmosphere shaders but couldn't find one that worked well with what I already had, so I wrote my own from scratch. As with most of what Undiscovered Worlds does, it aims for superficial believability rather than accuracy, but I think it looks reasonably decent. It doesn't actually calculate Rayleigh scattering and the like, but it pretends that it does. You can adjust atmospheric thickness, colour, specularity, etc. in the appearance settings.

The clouds are formed quite simply:

(1) Several fractal maps with different levels of detail are made and then combined. Then these are converted to the range -1 to +1, and then turned into their absolute values before being inverted. This gives the effect of wiggly lines. Finally, the resulting map is warped. This gives us the fairly large blobs of cloud with some stringy bits that you can see in the images.

(2) Then, another fractal map is made with a much smaller granularity. A map with larger granularity is used to remove large sections of this. This produces the smaller, puffier-looking areas of cloud.

(3) These two cloud maps are combined using yet another fractal, so that they merge into each other in a varying way.

(4) A simple swirl effect is added in places to give the effect of weather systems. Of course we make sure that these swirl in the correct direction, depending on hemisphere and overall planetary rotation direction.

(5) Clouds are removed in areas of low precipitation, to ensure that you don't get great big weather systems over deserts and the like.

Plus I've reduced the effect of the normal map, and made wetlands and large rivers visible (and specular) from space.

As always, it's not perfect, but I think it's looking reasonably decent. I shall probably continue to tinker with the clouds, but for now I'm happy to move on to the next bit, which will be some boring refactoring. I'm using a lot of three-dimensional vectors now, which are slow, resulting in annoyingly long planetary generation times. So I want to rewrite them all as one-dimensional ones to see if that speeds things up.

(Bonus shot: ocean worlds can now look quite dramatic:)

Shiny!

There's not been much activity here recently, but things have been moving behind the scenes. I've mostly been trying to work out how to improve the visuals for my planets. I was just using OpenGL to display them, and I wrote last time about wondering whether to change to a less low-level approach that might make it easier to get a few nice visual effects in. (I was particularly missing the fake 3D effect of my old maps.) After a lot of time alternating between digging deeper into OpenGL and trying alternatives, I decided to use raylib, which is not an engine so much as a library that keeps things pretty low-level, but without quite as much fuss as doing everything in OpenGL directly. Also it's easy to use ImGui with it, and I wasn't going to redo all the GUI at this stage.

There's still quite a lot of getting one's hands dirty with this. I've spent a lot of time tinkering with shaders, and have managed to generate some fairly simple effects, but I'm quite pleased with them:

Getting the normal map to work properly was very fiddly. For some reason I don't understand (a glitch in my cubesphere object, or a glitch in my shader?) the shadows on the top and bottom faces of the cubesphere vary incorrectly across the faces, even when the normal map is definitely correct. So I had to mess about with my normal map creating routine to vary them in the opposite way, to counteract this effect. It's still not perfect, but it looks mostly OK, so I'm sticking with it for now. It's certainly nice to see shading on my mountains again (even though, at this scale, it's probably too exaggerated).

Next I need to think about how to improve my shader - ideally by adding some basic atmospheric effects, as well as clouds, if possible. And then I'll need to think about how to integrate the regional map level into this system, which is where things will get tricky.

Another decision point

Time (at last) for a new update!

I have, more or less, managed to convert all of my existing terrain and climate routines to work with spherical planets. This has been quite fiddly to do. Some could be converted fairly straightforwardly, while others could not. In some cases I had to rewrite them from scratch, and in extreme cases I had to simply alter them to work with a giant 2D map which subsequently gets wrapped onto the sphere. In some cases I found that the rewritten versions were better than the original - e.g. my continent-generating routines now seem to throw up more varied terrain more often, including archipelagos, meaning that I don't need to use the Aegean-creating function.

I was particularly held up by a subtle but nasty bug in my routines for moving from one point on the sphere to the next, which caused inaccuracies when moving from one face to the next. Much of my time was spent writing workarounds to deal with the effects of this bug until I finally found it, after which I had to go through and remove some of those workarounds.

But it's done! Mostly. All four of the planetary types (small continents, large continents, oceanic, and non-tectonic) can now be in spherical form, and in all three planet sizes too. Most features such as mountains, dunes, lakes, wetlands, craters, etc. are present, as are the variable climates (including exotic climates). There are still some issues. Artefacts occasionally appear at the edges of faces, and occasionally the whole thing crashes. So there's still some work to be done smoothing it all.

Here's a typical continental world, showing some of these features:

Here are a couple of non-tectonic worlds that I thought looked interesting:

Some new terrain features are possible now. For example, rarely, ocean ridges on small worlds can create planet-wrapping archipelagos and islands:

This was unplanned, but I like the effect, so I've left it in as an occasional possibility.

And you can of course view things like rivers on the globe:

- and climate types:

So now there are two problems I need to consider.

First, what engine should I use to display all of this? At the moment I'm just using OpenGL directly. This is fine for displaying simple objects such as the facets of my globe, but doing more elaborate effects is probably beyond my abilities. So far I've not been bothering about this as I wanted to work out whether I could create the terrain etc. of the planets first, and worry about how to display them later. But obviously with this method I can't do things like apply shadows, which means that they don't look as nice as my 2D maps that have the faux shading effect. So I'm wondering whether to use an existing engine such as Irrlicht to render the globe, which might allow me to do more. But then of course I'd have to learn how to do that...

Second, what about the regional maps? As things stand, I haven't implemented them at all yet in this new version. The actual generation of them shouldn't be too tricky. (I know I will regret typing that! But at the moment I think it should be fairly straightforward to convert the existing regional map creation routines to work with the spheres.) The problem is how to display them. One option is to keep the system of a zoomed-in mode, where the region is displayed as a 2D map, just as it is now. That would be very straightforward. But consider again what each region represents:

Each tile here is a region (supposing this is an Earth-sized world). Now suppose we zoom in on one in the middle of the "face" that is facing us. All is fine - it can be displayed in the zoomed-in screen, and we can scroll around to north, south, east, and west. But suppose we keep going north and find ourselves on the "face" that's on the top of the cubesphere. Scrolling around here won't be taking us north, south, east, and west any more, at least not in any consistent way. Suppose we now scroll to the right and go down the "face" to the right of this image. When we started on the first face, the top of the map was pointing to the north. But now it's pointing east. Things get even worse if we try to navigate around one of the points where three "faces" meet.

All of this comes from the fact that the world isn't really a sphere at all, but a cube, and if we display the regional map in 2D form it's going to be very hard to conceal this fact.

So the alternative is not to do it that way at all. Instead, we could allow the user to zoom gradually in on the globe, generating the regional maps as they do so, and then display those maps directly on the globe itself. So as you zoom in on the map you would see the regional-level details popping in once it's close enough.

There are two problems with this. The first is that it's hard! We'd have to be able to generate regional terrain on the fly as the user is moving around the globe, and keep track of which regions have been generated, which ones need to be generated, and be able to paste them onto the right bit of the globe. All of this is theoretically possible but could be a nightmare to implement.

The second problem is that some tiles would get distorted. Look again at the image above, and in particular the regions at the corners. We get away with the distortion at these points at global scale, but I worry that at regional scale it would look pretty bad. Mountains and rivers, in particular, would look unnaturally stretched.

So I'm not sure what to do about this. The next job is therefore probably to convert the regional map generating parts of the program and try pasting the generated maps onto the globe to see how they look.

The blue marble

I've not had much of a chance to post any updates for a while. This is partly because I've been very preoccupied with real-life distractions (finishing the first draft of a book 😱) and partly because the process of translating everything into spherical form is pretty time-consuming.

Much of the difficulty comes from having to deal with multiple coordinate systems at once. The cube-sphere, as described in my last post, is really good for creating terrain that looks reasonably good from all angles, without the distortions you get from just wrapping a rectangle directly onto a sphere. From any given point on the cube-sphere you can easily move to the neighbouring points in a grid fashion, just like on a 2D plane (except at the corners of the faces, where it gets tricky). However, if you want to move due east or west, or north or south, it's much more complicated, because the grid doesn't line up with those directions unless you're right on the equator. To work those out you need to know the longitude and latitude of the points, and converting from cube-sphere coordinates to longitude and latitude (and back) is both fiddly and resource-intensive. And being able to specify directions of this kind is essential for the climate simulation.

Still, I've made some progress. I've converted most of the standard continental terrain generation to the spherical form, and made a good start on the climate model. Temperature and basic rainfall are in place, as well as wind directions and sea ice. Here are some pictures - they are all of the same world, which hopefully gives a sense of how the globe looks from different angles:

Here's the south pole, so you can see it's possible to have polar continents without distortion:

And here are the wind bands (red: westerly; cyan: easterly), which aren't very realistic, but I'm not going to try to change them:

There's a still a lot to do. I've been having a lot of difficulty getting the mountain generation methods to work well with the globe - in fact I've had to create an entirely new way of doing the mountains along continental edges, which is based on the functions I originally created to import user-generated mountain maps. But there's more work to be done with them (the world shown here is one of the most mountainous I've been able to generate, and it still looks very flat). Then I need to add the monsoon routines, before getting to grips with rivers and lakes, which is never much fun. But I think I've reached the point where it looks like converting this whole thing to spherical worlds can actually work, so I'm hoping the final results will be worth the effort...

Cubular balls

After the last release version of Undiscovered Worlds I'm more or less at the point I was nearly four years ago, when I got as far as I could with the original version of this project in AGK. Back then I wondered which of these three things to do next:

(1) Create a new level of detail, so one could zoom in on the regional maps and expand them.

(2) Start populating worlds with procedurally generated life forms.

(3) Rewrite the entire thing in a more suitable language.

I went for option (3), and the current state of UW is the result. It's much better than what I had then - faster, fewer errors, more options and variety, and much more detail and realism (within reason). So what next?

Options (1) and (2) remain on the to-do list, and I was initially going to get cracking with (1). This would add "local" maps, which would have the same relation to the regional map that the regional map does to the world map. But before doing that I thought I'd try a bit of more fundamental experimenting with the program.

One obvious limitation of UW as it stands is that it deals only with flat maps. I wrote right back at the start that creating spherical worlds is theoretically possible, but I wasn't going to try it as it's too difficult. But I've always wished I could have done it, and this limitation is more apparent now that worlds with different climates are possible. That means that there's no guarantee that any given world will have ice caps at the northern and southern edges of the map, which in turn makes it harder to hand-wave away the terrible warping that happens if you try to wrap one of these rectangular world maps onto a sphere.

So I thought, having come this far, I might as well at least try to see whether I could rewrite the program to work with spherical worlds. If it doesn't work then I can just give it up and continue with the flat maps as before, but it might be interesting to see what can be done.

So after a fair bit of tinkering, and wrestling with the horrors of OpenGL textures, I have this:

This is my existing diamond-square algorithm, which forms the basis for a lot of the global generation, but rewritten to map onto a sphere. This wasn't easy, but it works!

The basic idea is to represent the world as a cube. Each of the six faces of the cube has its own terrain map. When the cube is displayed, we warp each face to form a sphere shape:

This works because each face of the cube is not a single plane object but a grid of 16*16 of them. We warp the cube by taking each vertex of each plane object and transforming them using the extremely useful formulae found here. Here's what it looks like if we show all the sub-faces:

So our sphere is actually a polyhedron made up of 1,536 squares, which are distorted to varying degrees. As you can see, this projection keeps all of the subfaces roughly the same size as each other, but distorts the shapes towards the edges of the faces and especially at the corners. An additional irregularity is that where most of the squares have eight neighbours (including the diagonals), the ones at the corners have only seven.

The visualisation is useful because each of those squares also corresponds to a single regional map. Exactly how the regional map will work with this remains somewhat unclear, particularly when it comes to navigating round the corners, but I'll cross that bridge if I come to it.

In any case, the distortion involving in warping the cube into a sphere is much less than you get if you try to wrap a rectangle onto a sphere. In the latter case there are two points of distortion - the north and south poles - whereas with the cube there are eight - the corners. That shares out the distortion more evenly so that it is much harder to see.

The next job is to see if I can adapt the rest of my global terrain (and climate!) generation algorithms to the spherical maps. I'm fairly sure it should be theoretically possible, but that doesn't mean it's practical to do, so we shall see... If not, back to the flat maps!

Update: version 1.0 now downloadable

The updated version of Undiscovered Worlds is now downloadable on the first page. I've also updated the instructions for use there, and the screenshots. Since the previous version was rather unpolished compared to this one, I'm retroactively numbering it 0.x and calling the new one 1.0!

Note that saved worlds and settings from previous versions aren't compatible with this one.

The new version has all the features I've written about since the original upload, plus a few that I haven't. Here's a summary:

UI and presentation

• Proper installer application
• New app icon (courtesy of u/Iron-Phoenix2307)
• Nicer font (Roboto)
• Clearer organisation of information
• Automatic update notifications

World generation

• Three sizes of worlds now possible
• New terrain types: non-tectonic and ocean worlds
• Greater variety of tectonic terrain
• New features: craters, mud flats, beaches, mangroves
• Much larger sandy deserts
• Variable planetary information: orbital eccentricity and obliquity, gravity, etc
• More robust, detailed, and flexible climate modelling, allowing for exotic climates

Custom worlds

• In addition to importing maps, custom-generated ones are now possible with user-defined settings
• Ability to specify all planetary variables, including extreme settings

And of course various bugs have been eliminated, and many exciting new ones added.

The source code has all been updated too, so everyone is welcome to poke around in there too.

Water and sand

In my last post I talked about (yet another) new way of making coastlines better: by creating a mask for them using a simple diamond-square algorithm, and then filling in the terrain afterwards. It occurred to me that something similar could work for lakes, too.

Lakes are currently created at regional level via a horrendously complex process. Originally, I treated lakes as simply part of the regional terrain, using the diamond-square algorithm to generate the lake beds using the actual elevation values taken from the global map. This didn't work well, so I changed this to a more complicated method that traces the edge of the tiles of the lake and then disrupts it, to create a mask, and then generates new lake bed to fill it. This works fairly well most of the time. However, sometimes it goes wrong, for reasons I can't diagnose, and produces lakes with square borders or leaves them out entirely.

But it occurred to me that I could just adapt my new coastlines approach. If I'm generating a mask for the shape of the lakes before calculating the elevation of their beds, I can make that mask using the diamond-square algorithm, just as with the coastlines. The key realisation is that this algorithm doesn't have to be seeded with elevation values from the global map. It can be anything. This allows me to give it values that tend to produce good lake shapes, and then fill in the depths afterwards.

This gives results like these:

This method has several advantages over the previous one. It's much simpler and requires far less code. It also seems not to go wrong, which also helps. Moreover, this method can render lakes of any size. With the old method, for each regional map, UW would create the entirety of any lakes that were only partially on the map, as in the last image above. This meant there was a limit to how big they could be (and even the larger ones within that limit tended not to work properly). This new method doesn't do that, however. It just creates the bit of lake being shown and ignores anything outside the borders of the current region. This means we can have very large lakes. It also means that we can have very large ergs (areas of sandy desert) and salt pans, because the program treats these as weird lakes. So where before these couldn't get very extensive, we can now have immense sandy deserts like this one:

Indeed there's nothing to stop us now from having Arrakis-style planets with ocean-sized deserts of sand. As with everything, we don't want that to happen too often, but anything that (believably) increases the range of possibilities is good. Some more adjustments to this new system are needed, but it seems to be fairly robust so far, so I'm quite pleased with it.

Sinking the extra islands

In my last post I mentioned an issue with excessive islands appearing around coastlines. Happily, it's proven to be a very easy problem to fix (at least I think so...), but I thought it worth explaining.

Here's an example of the problem:

In itself there's nothing wrong with this, but it's a problem when all coastlines look like this - and as you can see in the last post, they do. But why? If you look through earlier posts on this blog you'll see mostly much less busy coasts. Somewhere along the line, I've made some kind of change to tackle some other problem, and it's had the unexpected side effect of replicating the Hebrides everywhere. Which is fine on occasion, but we don't want it everywhere. It's another illustration of the difficulty of making changes to what is now such a complicated piece of code, as it has so many moving parts that a small tweak in one place can have unlooked-for ramifications elsewhere.

How are coastlines made, anyway? Well, the terrain at the regional level is basically done by seeding the corners of the tiles with values taken from the corresponding cells at the global level, and then applying the diamond-square algorithm to fill in all the rest. Where terrain higher than sea level meets terrain that's lower, we get coasts. There are then endless tweaks to make them look better, but the diamond-square is at the heart of the process.

I think that what's happened has something to do with the changes I made a while ago to the sea bed, especially the addition of continental shelves. This means shallower seas near the coasts on the global map. Shallower seas mean more islands on the regional map, because the diamond-algorithm basically makes the terrain bumpy, and if the terrain is near the surface of the water to start with, lots of those bumps will stick out of it. (This is especially so given that the algorithm is set to tend to bump land up, in order to emphasise river valleys.) The solution, then, is simple. I add to this diamond-square algorithm a check to see whether a point at the corners or on the side of the tile is below sea level. If it is, we make it a lot lower. As a result, the algorithm tends to produce lower land around the sea, and far fewer islands are produced.

But doesn't this result in undesirably deep seas around the coasts? No, because the undersea terrain that this algorithm produces isn't actually used for the final map. Instead, there's another diamond-square-based function that creates undersea terrain. UW calls the first function first, and works out what's land and what's sea, and the heightmap for the land. It then calls another function to create seabed, and then pastes that onto the sea regions of the original heightmap. This allows me to make changes to the land terrain generation without messing up the seabed, and vice versa. So in this case, we can artificially lower the seabed in the way described above during the initial terrain generation, and it won't affect the final seabed - but it will affect the coastlines.

Here's what the above map looks like after doing this:

This looks much more plausible. Some islands still remain, and of course we don't want to get rid of them entirely. I've set it so that the island-reducing tweak varies in strength. It's weaker in areas that have higher roughness, and much weaker in arctic-type areas where one might expect fjords. So crinkly little islands are more likely in such areas. As always, variety helps to make the maps more believable, at least in theory.

(Also, notice how in the first picture there is one river that doesn't reach the coast. I think this is because the land has been extended out into tiles that correspond to ocean cells on the global map, and the river simply doesn't reach that far. There are functions that are supposed to deal with such cases by extending the rivers out to the coast if need be, but they seem to have failed here. In any case, you can see in the second picture that the coast has retreated inland and the river now reaches the sea properly, so that's a bonus from this tweak.)

Mud and sand

Having grown up by the coast, I'm very aware of the need for a terrain generator to be able to handle different kinds of coastlines. So I wanted to try to include some in Undiscovered Worlds.

Since I currently live near the Bristol Channel I'm particularly aware of the importance of mud flats, so I had a go at incorporating those first. The basic idea depends on a feature that has been present in UW for a long time: inlets. The program removes land around river mouths, depending on factors such as the size of the river and the tidal range at that point. It occurred to me that it's easy to keep track of where these inlets have been created, and we can simply lay down mud flats around them - again based on factors such as the size of the river and the tidal range. Mud flats, of course, are represented by an array of bools within the region object.

At this point things got very complicated as I decided this would only make sense if we also kept track of how much silt every river is transporting. I wrote a new system to simulate this and got it mostly working, but annoyingly it wasn't quite right. This was a pain as the maps looked quite good with river colours varying from bright blue to dull mud, depending on their silt content, but the fact that sometimes this didn't work right (with small stretches of river inexplicably showing too much or too little silt) made it not really worth the effort - especially as it was largely a visual effect that didn't really make much difference to the world generation.

So after scrapping all that, I went back to the inlets. In this map, inlets are marked in magenta:

These are all river mouths where the program has cut away land to make it look like the rivers have eroded the coastline. Most are for very small rivers, but some are for larger ones.

Now we create a temporary array to hold silt levels. Going through the map tile by tile, we lay down silt levels on coastal cells that are within inlets. The levels of silt are determined by the size of the river and the tidal strength of the area. Then, we go through the map again and spread the silt out from these points, both inland and into the sea. Any cells that have a certain level of silt or higher are then turned into mud flats, which also involves setting their height to 1m above sea level, and possibly creating some saltwater wetlands nearby too.

Only a few inlets actually have enough tidal strength to generate mud flats big enough to show on the map - bearing in mind that each pixel is a kilometre, so the mud flats have to be quite extensive to show up. Here is a region that has quite a lot of them:

And on the relief map, they blend in fairly subtly with the wetlands around them:

We can also add mud flats to river deltas, too:

Now we want nicer beaches too! Namely, sandy beaches and also shingle ones. We can make these in exactly the same way. With the beaches, though, the tidal range is more important than river size, so we tend to get mud flats around large rivers and sand or shingle around small ones. And where mud flats tend to spread inland along the rivers, beaches tend to spread along the coast. Finally, we set it so that mud, sand, and shingle can all mix with each other, making for varied coastlines.

The resulting effect is quite subtle, because again only really large beaches can be seen at this scale. Here, you can see a few spots of yellow or brown along the coast where there is enough sand or shingle to make a large beach:

There might not seem much point adding such minor features, but one of the new planetary variables is lunar gravity. Worlds with weak lunar gravity have small tidal variation, and relatively few mud flats and beaches. But worlds with strong lunar gravity can have extreme tides, with correspondingly extensive beaches and mud flats:

There are a couple more tweaks we can add. First, mud flats get a chance to generate barrier islands along the ocean-facing edge. This results in areas like the Wadden Sea, where a string of islands protects a large intertidal zone. Second, in tropical areas, mud flats (and salty/briny wetlands) get mangrove forests on them. You can see the darker green coastal areas here (the user can turn off this effect if desired):

So that's that for coastal zones, at least for now. The next job is to try to work out why all my coastlines have acquired far too many little craggy islands and get rid of most of them (they never used to look like that!). After that, I'll aim to release the updated version.

Warp speed

I liked the effect of domain warping at the global level, and I wondered: might it help at the regional level too? With the regional terrain, there's a constant danger of the underlying tile pattern becoming apparent and making the terrain look like a grid. So I'm always looking for ways to disrupt this.

Originally, I intended every tile at regional level to be calculated entirely independently of the others, so that the whole map could be scrolled tile-by-tile while keeping every feature the same. I eventually gave up on that. Now, although many features are created on a tile-by-tile basis, many bleed across into their neighbours. You can't scroll tile-by-tile now: each regional area always has the same 32x32 tiles in it. So I thought: suppose I were to apply domain warping to an entire regional area. Might that make it less grid-like?

One complication is that the warping needs to be faded away towards the edges of the region, so that it joins up neatly to neighbouring regions. This wasn't hard to do. I then spent a great deal of time tinkering with various ways of doing it. One problem was that rivers can't be warped in this way. Or rather, they can, but they look terrible. I did find that warping the mountains worked well to make them a bit less gridlike - but for this to work, I had to warp the normal terrain too, because otherwise the mountains might move away from the higher normal terrain that normally lies beneath them. Since I wasn't moving the rivers, warping the terrain around them did leave some rivers travelling over what were effectively aqueducts if the land around them was lowered. I tried making it so that warping is only done if it would raise the land in question. This solved the river problem, but now the high terrain under and around mountains tended to get stretched across the landscape, creating broad plateaux, which looked weird. (There could also be a similar effect near the edges of the map if the natural variation in warp synced with the reduction in warp strength at the borders, creating stretch marks.) The only way to get around this that I could find was to reduce the strength of the warp effect, but doing this enough to overcome that problem meant that it had little overall impact.

So eventually I gave up on warping the terrain on land. However, I did find a similar effect worked well on the rifts and ridges on the sea floor. Because these feature a lot of parallel ridges, they can look a  bit like they've been drawn with a ruler. But a bit of warping really helped here, since there are no rivers or anything to get in the way. Rather than simply warp the whole undersea terrain after the ridges have been drawn, I generated the terrain normally but warped the coordinates that the ridges would be drawn to, and then drew them to those coordinates. This meant that the ridges themselves don't look stretched, but they are forced away from the underlying tile structure.

Compare this before shot:

...to this after one:

It's a shame I couldn't get this to work equally well on land, despite all that tinkering, but I think this is still a nice improvement, minor though it is.