Improving river simulation

I'm continuing with the process of converting the old region-drawing functions into the new globular context, and refactoring and improving them where possible.

While doing this I decided that the old way of tracking river flow wasn't accurate enough. At the moment, UW stores the January and July temperature and rainfall for every point on the global map. When needed, it can calculate the temperature and rainfall for other months by extrapolating them from that stored information (along with other relevant information such as the planet's obliquity, eccentricity, and so on). So far I've been doing the same thing with river flow: for every global cell, the program stores how much water is flowing in January and in July. The problem is that it's not possible to extrapolate the information for the other months from this information. This is because river flow is determined by more than just information about that point on the map. The water comes from elsewhere, and quite different climactic conditions might prevail in those other places. So we can't say (for example) that because there's less rain in March than in October there should be less river flow, because perhaps that isn't true of the place where most of the water is coming from.

So I've updated the program to track and store river flow for all twelve months. When the rivers are calculated, it goes through every cell on the globe and calculates how much water will be flowing for every month, and then adds that, month by month, to every downstream cell from that point. This means that the variation in flow can be shown quite accurately.

Here, for example, is a world with a very long river that travels through mostly desert and tundra:

(I've removed the clouds so it's easier to see.)

The information in the graphs to the top right are for a point on this river roughly in the middle of the image, where it passes through a pretty cold desert. There's significant variation in temperature, and no precipitation at all, but the river has a roughly constant flow throughout the year because its sources, far to the north, are in places with fairly even rainfall throughout the year.

Now compare this image:

You can see another large river which joins the first one near its southernmost point, in a darker area that indicates extensive wetlands. This river has its sources further south in the tundra and arctic mountains to the bottom right of the image. The graphs here are for a point on this new river near where it joins the first one. The temperatures and rainfall are similar to the previous one, but here you can see extreme variation in river flow: none at all for much of the year, and then a deluge in the brief summer as the snow melts in the sources of the river far to the southeast.

And, finally, we have this image:

This shows information for a point on the river near its mouth. Here, there is a bit more rainfall (not much!), but you can see that the variation in flow of the river follows an opposite pattern. For most of the year the river is moderately sized. This is the same flow that we saw in the first image. Then in the summer the river floods extensively, thanks to the meltwater coming from the second river that joined this one.

Since UW now displays variation in the seasons, the plan is that the sizes of rivers at the regional level will be shown dynamically, so you can see them swell and shrink as you move from month to month. Now that the program is storing flow information for every month, rivers like the one above should show this kind of variation in a way that adds a bit of believability to the simulation.

Zoom!

It's been quite a while since the last update. This is partly because there has been even more RL than usual to deal with, and partly because creating regional-level detail on spherical planets is hard!

To recap, in the previous version of Undiscovered Worlds you could view a world map, and then click on a point on that map to bring up a regional map, in much higher detail. The program creates the regional map afresh every time you visit it, but procedurally, so it's always the same. This creates the illusion of a vast and highly detailed world. But of course only one regional map is displayed at any time.

With the new version, we want to display regional maps directly on the globe. So instead of having separate global and regional displays, we just have a single display that shows the global images if you're far away and the regional ones if you're zoomed in, fading neatly between them. This means displaying multiple regional images at a time, since the user might be only halfway zoomed in and therefore able to see several at once. It also means being able to dynamically create the regional images as the user moves around, ideally without any gaps or pauses.

Well, I've got the basics of the system up and running. Although the new regions are 2D, like the old ones, it's still quite a lot of work to translate the generation functions given that they're now extrapolating information from a 3D global map rather than a 2D one. Along the way I'm trying to refactor the functions to be as efficient as possible, since speed is now even more important than it was before. So far I've done just the basic elevation with mountains, and rainfall and temperature. No lakes  or rivers or anything else fancy yet, but it's enough to give a pretty good idea of how it all works:

That's several regional areas all being displayed at once. Here's a video of the zoom (though it's a bit blurry!):

There are still a lot of bugs, as well as all of the other features to add, so it will be a while before this is done, but progress is being made!

Alpha update

Since the program is now running in a mostly stable way, I decided this might be a good time to update the code on github. It's definitely not ready for a proper release so I've not uploaded a built executable, but some people might like to have a look at how it's working. So consider this a sort of alpha preview.

It's uploaded here. The previous version of Undiscovered Worlds, with flat maps, is now Undiscovered Worlds Classic and can be found here, including the executable and instructions.

You don't get the jazzed-up GUI though, because those are changes I've made directly to the ImGui code, so I can't really upload them as part of this project.

Parrots everywhere

My spherical worlds are mostly working pretty well now (although for some reason I haven't been able to make river deltas work on them - I don't know why). So it's time to try to get regional-level maps working with these worlds.

As I've explained before, this will work differently from the original, flat-world version of UW. In that version you could view the global map, select a point on it, and zoom in to see the regional map of that part of the world, in a separate screen. But this wouldn't work very well for a spherical world, where we will instead display regional maps directly on the globe itself. As you zoom in, the details of the regional maps will become visible. So it should all operate seamlessly. That's the theory anyway!

This means we need the ability to generate and display multiple regional maps at once. It's not possible to store regional-scale information for the entire planet, since a large world has 16 x 16 regions on each face, making 1,536 regions in all. We can store only a fraction of these at once. So what we need is a system whereby the program can dynamically create new regions as the user moves around the world, deleting them when they're no longer visible.

After many false starts I've got the basics of this system up and running. We have a stable of threads dedicated to generating regions and drawing their maps. As the user views the globe, these threads work in the background to create regions roughly where the user is looking, with unused ones being discarded. Meanwhile, the main thread displays the regional textures where they exist, and the lower-resolution global textures where they do not. This way, if the user is zoomed in enough to make out details at the regional scale, they should be able to see only regional textures visible on the screen.

Now I haven't yet started translating the actual region creation functions so that they work with spherical worlds, because I wanted to get this system up and running first and then plug in the region creation functions as I convert them. So at the moment the region creating threads are just applying a placeholder texture (of parrots, because why not). But here is a pretty rough-and-ready (and somewhat nausea-inducing) video showing the system in action.

As you can see, as we move around the planet (or rotate the planet itself) the more distant regions are deleted and replaced with closer ones. If you zoom in closely, it creates the illusion that the entire world is covered with nothing but parrot images, because you can't see the edge of the patch of textures.

Now all I need to do is to convert the existing region-creation functions from the old version of UW. For the most part this should (I hope) involve changing them so they take their raw data from the spherical world object instead of a flat one. This will be tricky at the edges of faces, because the regions as generated are a bit larger than what you see. This is because some elements of the tiles that compose the regions overlap from tile to tile, so when we generate a region we need to generate the tiles along the edges of its neighbouring regions, in case anything from them overlaps onto our region. In a flat world this is fine, but here we're going to have regions along the edges of faces of the cubesphere, which means we have to take into account the fact that tiles from the next face along might be rotated relative to the active face, and that complicates matters.

That probably makes very little sense. Still, I'm glad to have the framework for regions set up now, so I can focus on getting back to map generation...

Sprucing up

My plan to create, replace, and delete regional maps on the fly means I need to use multi-threading, where a program basically splits itself into multiple concurrent programs. This is so that Undiscovered Worlds can get on with generating regional terrain while the user is moving around and viewing the world. I've never done this before, so I've been learning how, and fortunately it's not too hard, at least in principle. To practise, I've added a progress bar for world creation - one thread creates the world while another updates the progress bar - and it works quite nicely.

Adding the progress bar made me think again about the GUI. I'm using Dear ImGui, which is a very widely used library, but mostly for internal use rather than user-facing interfaces, because it's pretty simple to incorporate into one's code but doesn't look very fancy. (Also it's easily recognisable!) I certainly don't want to change to a different GUI again, but I thought it would be nice to try to change the appearance of Dear ImGui a little to make it a bit more distinctive and attractive.

So I've had a go at tinkering with the code of Dear ImGui. I've been shamelessly copying the style of NanoGUI, another library that I was using for a while in an earlier version of UW, but abandoned as it proved too tricky to port to Windows. My very rough hacks can't really reproduce its appearance, but I think the results look quite nice:

Hopefully it doesn't look quite so generic now. There is still some tweaking to be done, of course, but the next focus is on getting some version of the regional detail up and running.

Global details

It's time to think about adding detail to the planets. The ultimate aim is to create regional maps similar to those in the old version of Undiscovered Worlds, but wrapped onto the sphere. My initial thought was to generate an entire globe's worth of these while the user is looking at the globe, adding them on as they are generated - but that is too much detail to hold in memory at once. We need, then, a more dynamic system, which can generate new regional maps and delete old ones as the user moves around the globe, so we only store the ones we actually need.

This creates a problem though. Here again is how the cubesphere is constructed:

If this is a large world, then each of the small panels corresponds to a single regional map. (With small or medium-sized worlds, it's still constructed from the same number of panels, but a single regional map would be spread over several panels - four each for a medium-sized world, or sixteen for a small one.) So as the user moves around the globe, each panel will display either its portion of the global map texture or its own regional map, depending on how close it is to the viewer.

Clearly, for this to work well, we want to display regional maps only when they are pretty close to the viewer. Otherwise we would have to be generating masses of them every time the user scrolls around even a little bit.

The problem is that the global texture looks quite blocky quite quickly as you zoom in. It's just not highly detailed enough to let us get away with generating only very close regions.

The solution is to make the global texture more detailed. Rather than making the world object (that is, the data structure that holds all the information about the world) more detailed itself (as I did once before), I've created a much simpler data structure that only holds information needed to generate the map textures - basically elevation, temperature, rainfall, river flow, and one or two other things. Since it's much simpler than the "real" world object, it can be four times as detailed without taking up too much memory. Once the world is generated, we populate this "simple world" on the basis of the "real" one, mostly using a simple diamond-square algorithm similar to the one used for regional maps. Mountain ranges and rivers are drawn more directly on, to maintain their detail. Then we just use that to create the main world textures. Since the "simple world" is only used for generating those textures, and doesn't contain information that the user can access otherwise, we don't have to worry about details such as carving out river valleys and the like, so it's very quick to generate.

The "simple world" has a resolution of 4x4 the "real" world, which means that the global textures can now be 16 times as detailed. This only applies to the main textures, showing the world pseudo-realistically - the other textures, showing information such as elevation, temperature, etc., remain in the old resolution. So for example, here are some shots of a single area showing elevation, rivers, and basic terrain colours, all in the "true" resolution:

And here is the same area with the higher-detailed texture plus fancy shaders (and clouds dialled right down so you can see the details more clearly):

It's a pretty simple technique, and there are certainly a fair few wrinkles still to be ironed out, but I think the initial results are quite decent:

With this in place, we can now zoom in a fair way before the map looks too blocky. That means that, in theory, we can generate regional-scale maps and apply them to the appropriate parts of the globe only when the view is quite close - so we don't need to display many of them at the same time. Whether it will actually work like that, though, remains to be seen!

Stars and suns

I've been doing a fair bit of tinkering with the model, focusing on improving the algorithms connected with monthly changes. Seasonal temperatures and rainfall are now more realistic (more or less), and so is river flow. You can also pull up monthly river flow charts alongside those for temperatures and rainfall, to get a better idea of seasonal changes for any point on the globe.

Alongside this, I've removed some bugs with specular maps (rivers now reflect sunlight correctly) and improved the appearance of clouds.

After doing all that, I decided that my planets looked a little lonely floating in the void, so I've added procedurally generated starfields, galactic dust and nebulae, and some bloom effects on the sun, as well as simple lens flares. All of these can be adjusted in the settings. I wrote all the shaders from scratch, as I couldn't find anything quite like what I wanted. This was a pretty tough learning curve, but I'm glad to have managed it and am quite pleased with the final effects.

Here are a few shots showing how it looks now:

All of this means I've more or less run out of things to do before tackling the next big thing, which is adapting the regional map model to the new spherical layout. As I've mentioned before, this is inherently tricky. The ideal solution, if I can manage it, is to generate regional maps on the fly while the globe is displayed, and display them on the globe in the appropriate places as they're generated, if it's zoomed in enough to make it worthwhile. So this would involve keeping a lot of regional maps in memory at the same time, with hopefully a system intelligent enough to generate precisely the ones that are needed, depending on where the user is looking, and deleting the ones that aren't.

That's the plan anyway! If I can't get that to work then I may have to go for a less ambitious system where the regional maps are displayed in a different screen altogether, as with the 2D version.