{"id":46,"date":"2014-02-20T22:03:00","date_gmt":"2014-02-20T22:03:00","guid":{"rendered":"http:\/\/oneiricworlds.com\/en\/index.php\/2014\/02\/20\/world-of-thieves-goes-open-world\/"},"modified":"2016-02-20T17:15:50","modified_gmt":"2016-02-20T17:15:50","slug":"world-of-thieves-goes-open-world","status":"publish","type":"post","link":"https:\/\/oneiricworlds.com\/en\/2014\/02\/world-of-thieves-goes-open-world\/","title":{"rendered":"World of Thieves goes Open World"},"content":{"rendered":"<p>Hi again folks&#8230;<\/p>\n<p><b>[WARNING: This is a crazy technical article. Pursue only if you&#8217;re mad]<\/b><\/p>\n<p>As promised in my last post, here is a pretty technical article to show you the kind of problem I run into and to give you a hint on how I spend my days losing my hair. This one is a pretty complicated one and kept me occupied for the last 2 weeks, because it forced me to change a lot of things in the code of the game even if I though &#8220;no problem, I planned it well, everything&#8217;s gonna be okay&#8221;. How na\u00efve&#8230;<\/p>\n<p>The <b>final goal<\/b> is to have a <b>continuous world without any loading time<\/b> between zones.<br \/>\nWhy an open world you may ask? Because, at the beginning, I set myself 3 major guidelines about my game:<br \/>\n&#8211; freedom<br \/>\n&#8211; humour<br \/>\n&#8211; oneiric world<\/p>\n<p>And every choice I make at any step of the development follows these 3 &#8220;rules&#8221;. This ensures me that the game, even if not perfect, will have some coherent content and some kind of art\/feel direction. Thus&#8230; having an open world helps a lot for the freedom feel. Plus it&#8217;s a crazy challenge, and I&#8217;m a crazy guy :).<\/p>\n<p>As you may have seen in the <a href=\"http:\/\/home.oneiricworlds.com\/search\/?q=label:WorldOfThieves+label:Demo\">demos<\/a> and <a href=\"http:\/\/home.oneiricworlds.com\/search\/label\/Video\">videos<\/a>, the world of my game is a big ocean with various islands on it (yes, just like Zelda Windwaker). At some point the player will have the ability to travel on the ocean (on a turtle&#8217;s back \ud83d\ude09 ) and can go pretty much everywhere he wants. This means islands\/levels must be loaded dynamically according to the player&#8217;s position\/direction in the world.<\/p>\n<h3>I &#8211; Unity Limits (Yes I finally reached some)<\/h3>\n<p>I use Unity to create my game and (luckily?), Unity provides 2 functions to load a new level &#8220;in the background&#8221; so that you don&#8217;t notice any lag:<\/p>\n<ul>\n<li><b>LoadLevelAsync<\/b>: loads a new level in background. Once loaded, the new level replaces the current one.<\/li>\n<li><b>LoadLevelAdditiveAsync<\/b>: Same thing, but adds the content of the new level to the current one. This is obviously what I&#8217;m going for here.<\/li>\n<\/ul>\n<p>But this is theoretical only. Unity is a great software, but some points are still under heavy development. These ones are. And it impacts the game in a way I didn&#8217;t think about: after using LoadLevelAdditiveAsync, all the IA agents of the new level crash.<\/p>\n<p>This occurs because the IA uses a &#8220;<b>NavMesh<\/b>&#8221; (= Navigation Mesh) to represent the walkable areas in the level. The IA can only walk\/move\/search a path on the NavMesh. Problem is, Unity only authorizes 1 NavMesh to be loaded in memory at a time, and you can&#8217;t load a NavMesh using LoadLevelAdditiveAsync. Technical limitation. I can&#8217;t argue.<\/p>\n<div style=\"clear: both; text-align: center;\"><a style=\"margin-left: 1em; margin-right: 1em;\" href=\"http:\/\/3.bp.blogspot.com\/-SIXRJn8i_CM\/UwZp6vAIGRI\/AAAAAAAAHo4\/VyCYYkHfB1U\/s1600\/20140220_navMesh.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/3.bp.blogspot.com\/-SIXRJn8i_CM\/UwZp6vAIGRI\/AAAAAAAAHo4\/VyCYYkHfB1U\/s1600\/20140220_navMesh.jpg\" alt=\"\" width=\"640\" height=\"371\" border=\"0\" \/><\/a><\/div>\n<div style=\"text-align: center;\"><i>The NavMesh: enemies can only walk on the blue zone. The computation of this 3D NavMesh is a quite complex task&#8230; <\/i><\/div>\n<p>&nbsp;<\/p>\n<h3>II &#8211; Time for Hacks<\/h3>\n<p>I found a hack on a <a href=\"http:\/\/answers.unity3d.com\/questions\/303007\/dynamic-levels-navmesh.html\">forum post<\/a> : using LoadLevelAsync (which actually loads the new NavMesh in memory) and tagging all objects of the current level not to be destroyed (a cool feature I discovered while reading the forums. Great community by the way).<br \/>\nThis is supposed to do the trick but it rises 2 more problems:<\/p>\n<ul>\n<li>LoadLevelAsync is not actually a background task. It really freezes the game for a few ms, and it IS visible.<\/li>\n<li>Cool, the NavMesh of the new level is OK, and the IA too, but what happens if I go back towards the 1st level (which is still visible but with no corresponding NavMesh and no IA)? If I wan&#8217;t to reload only the NavMesh of the 1st level&#8230; I can&#8217;t without reloading the whole level, which may result in objects flickering during the reload.<\/li>\n<\/ul>\n<p>At this point, I&#8217;m faced with Unity bugs I can&#8217;t fix and I&#8217;m left with a few options:<\/p>\n<ul>\n<li><b>Wait for a bug fix<\/b> from Unity about the NavMesh+LoadLevelAdditiveAsync problem. I don&#8217;t think it will come before I release my game, the Unity guys have loads to do and this is not a priority.<\/li>\n<li><b>Use a 3rd-party library<\/b>. I must find one that does NavMesh generation and path-finding, is real-time, and dynamically loads levels.<\/li>\n<li><b>Recode everything<\/b> that is not working. Not impossible (I already coded a real-time A* path-finding algorithm for <a href=\"http:\/\/home.oneiricworlds.com\/search\/?q=label:WorldOfNinjas+label:Demo\">World of Ninjas<\/a>, but it works only on a 2D grid)&#8230; but hardly realistic. Good guys spent months developping systems much more reliable than anything I could do in a few weeks.<\/li>\n<li><b>Find another workaround<\/b>. I didn&#8217;t find any when I spent a few hours on forums and faqs.<\/li>\n<li><b>Give up<\/b>. This is a serious option. I can perform the navigation part on a 2D map where you click where you wan&#8217;t to go. All islands would be accessible too, and it won&#8217;t change the gameplay on each island. Maybe I&#8217;ll even consider that if I achieve to create\u00a0 a &#8220;real&#8221; open world but it&#8217;s not fun to explore.<\/li>\n<\/ul>\n<p>But before giving up I heard a lot of good things about a 3rd-party library implementing the classical A* path-finding algorithm: Aron Granberg A* path-finding library.<\/p>\n<div style=\"clear: both; text-align: center;\"><\/div>\n<div style=\"clear: both; text-align: center;\"><\/div>\n<div style=\"clear: both; text-align: center;\"><\/div>\n<div style=\"clear: both; text-align: center;\"><\/div>\n<div style=\"clear: both; text-align: center;\"><\/div>\n<div style=\"clear: both; text-align: center;\"><\/div>\n<div style=\"clear: both; text-align: center;\"><a style=\"margin-left: 1em; margin-right: 1em;\" href=\"http:\/\/2.bp.blogspot.com\/-ICg8nAOpF14\/UwZs5dTiPrI\/AAAAAAAAHpI\/kpNqbHCg_gs\/s1600\/20140213_172417.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/2.bp.blogspot.com\/-ICg8nAOpF14\/UwZs5dTiPrI\/AAAAAAAAHpI\/kpNqbHCg_gs\/s1600\/20140213_172417.png\" alt=\"\" width=\"640\" height=\"360\" border=\"0\" \/><\/a><\/div>\n<div style=\"text-align: center;\">\u00a0<i>Lots of levels loaded together!<\/i><\/div>\n<p>&nbsp;<\/p>\n<h3>III &#8211; A new lib: A*<\/h3>\n<p>Cool! A new library full of promises. But before commiting to this, I have to test that every basic IA feature already provided by Unity is implemented in this library.<\/p>\n<p>I start with the free version of the library, which means&#8230; there is no NavMesh generation available (only on the full 100$ version). Of course, I can buy the full version, but I&#8217;m not sure this library solves my &#8220;dynamic loading&#8221; problem. I must first test it on this specific point.<\/p>\n<p>I know that <b>Unity can generate NavMeshes<\/b> (I used them before). So I write a script to convert Unity NavMeshes to the library NavMesh format, which enables me to use the library path-finding on NavMeshes from my real levels.<\/p>\n<p>But there is already a problem: the path-finding behaves weirdly and sometimes IA makes huge detours to get to some point. It seems to be a known issue&#8230; This is because of the NavMesh topology: a &#8220;good&#8221; NavMesh for the A* library is supposed to have some kind of grid pattern on it to avoid big triangles next to small triangles. Unluckily, NavMesh generation in Unity doesn&#8217;t expose some &#8220;grid size&#8221; or &#8220;max edge length&#8221; parameters, which means I can&#8217;t test that the path-finding behaves correctly with a &#8220;good&#8221; NavMesh.<\/p>\n<h3>IV &#8211; Another new lib : RAIN<\/h3>\n<p>I heard about another path-finding library: RAIN. Totally Free, but sparse documentation. I tested it mainly to assess its NavMesh generation algorithm and hurrah! It can generated &#8220;grid&#8221;-NavMeshes. So I write another script to convert RAIN NavMeshes to A* NavMeshes, with very few documentation&#8230; Tough time! And&#8230; I get a few errors during the convertion but the NavMesh seems to be generated anyway. I test it with the A* path-finding, and it seems OK! The IA behaves well.<\/p>\n<p>Now, by combining 2 external libs, I have some basic IA behavior. I am at the same point that with Unity path-finding before.<br \/>\nI must now tackle the REAL problem: dynamic NavMesh\/IA loading.<\/p>\n<h3>V &#8211; NavMesh dynamic loading<\/h3>\n<p>It seems the A* library provides a way to export a NavMesh in a text file to load it dynamically at run time. Exactly what I need (theorically \ud83d\ude09 ). After a few tests, it seems to work at least for &#8220;little&#8221; files. But of course loading a new NavMesh gets rid of the previous one. I have to be cautious while activating\/deactivating IA agents. But this means I need to write another script to convert the A* NavMesh in a text file during level generation.<\/p>\n<div style=\"clear: both; text-align: center;\"><a style=\"margin-left: 1em; margin-right: 1em;\" href=\"http:\/\/3.bp.blogspot.com\/-w9pdyuPfX-E\/UwZuoqHTWII\/AAAAAAAAHpU\/9sBsfWyxJlA\/s1600\/20140220_220709.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/3.bp.blogspot.com\/-w9pdyuPfX-E\/UwZuoqHTWII\/AAAAAAAAHpU\/9sBsfWyxJlA\/s1600\/20140220_220709.png\" alt=\"\" width=\"640\" height=\"307\" border=\"0\" \/><\/a><\/div>\n<div style=\"text-align: center;\"><i>The usual test level for enemy\/IA behavior<\/i><\/div>\n<p>&nbsp;<\/p>\n<h3>VI &#8211; Integrating everything<\/h3>\n<p>OK. Every test I&#8217;ve made until now was of course on temporary\/separate IA agents. I must know rewrite the code of the real enemy IA in my game to make them use the new A* library. And of course, I have a few problems because the library doesn&#8217;t provide exactly the same callbacks\/hooks for various states (path is computing, agent has arrived at destination, etc&#8230;). But finally, the IA works just like before, and I can dynamically load a new level with correct IA behaviour.<\/p>\n<h3>VII &#8211; Final surprise: progressive activation<\/h3>\n<p>But&#8230; for larger levels, the loading seems to lag. How it this possible? I made all this to finally realize that the Unity fonction LoadLevelAdditiveAsync lags? Did I do something wrong?<\/p>\n<p>And indeed, after a few tests, it seems that the loading itself doesn&#8217;t lag. It&#8217;s the activation and start scripts of all the loaded objects (IA\/vegetation\/animals&#8230;) that occur on the same frame that makes the game lag!<\/p>\n<p>So I have to disable all the loaded objects, and activate them one by one on each frame. But this leads to 30 seconds to load a 1800 element level. So I optimized this to load many objects on one frame if they are light (a simple crate), and only one if it&#8217;s a complex one (enemies). I dropped down to 3 seconds to load the same 1800 elements.<\/p>\n<h3>VIII &#8211; Making it automatic<\/h3>\n<p>Cool! It works on a few levels that I placed &#8220;by hand&#8221; on the global world! But, in the end, I&#8217;ll have many levels, some of which may change in location. I have to set up a pipeline to ensure that every step I manually made is correctly and systematically handled for every new level.<\/p>\n<p>I set up a special &#8220;world&#8221; mesh file made in Blender to precisely locate every level in the world. Each individual level is stored in a separate Blender file and centered on a (0,0,0) position.<\/p>\n<div style=\"clear: both; text-align: center;\"><a style=\"margin-left: 1em; margin-right: 1em;\" href=\"http:\/\/1.bp.blogspot.com\/-fco-vNu-xVo\/UwZ4p0N7Y2I\/AAAAAAAAHp4\/dYYW6JPEs_8\/s1600\/20140220_worldMesh.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/1.bp.blogspot.com\/-fco-vNu-xVo\/UwZ4p0N7Y2I\/AAAAAAAAHp4\/dYYW6JPEs_8\/s1600\/20140220_worldMesh.jpg\" alt=\"\" width=\"640\" height=\"352\" border=\"0\" \/><\/a><\/div>\n<div style=\"text-align: center;\"><i>The simple world mesh locating all levels in Blender. The big and small spheres respectively represent the loading and activation zones.<\/i><\/div>\n<p>When loading a level from Blender in Unity here&#8217;s the (almost automatic) process:<\/p>\n<ul>\n<li>Find the final level position in the &#8220;world&#8221; mesh<\/li>\n<li>Move the whole level to the final world position (while converting coordinates conventions)<\/li>\n<li>Convert every Blender object in a &#8220;smart&#8221;\/&#8221;scripted&#8221; Unity object<\/li>\n<li>Create the NavMesh using either the Unity or RAIN NavMesh generation<\/li>\n<li>Convert the Unity\/RAIN NavMesh to A* NavMesh<\/li>\n<li>Plug the generated A* NavMesh into the A* path-finding Object<\/li>\n<li>Convert the A* NavMesh to a text file<\/li>\n<li>Deactive all objects in the scene so that they won&#8217;t be activated simultaneously after a dynamic load<\/li>\n<\/ul>\n<p>&#8230; And that&#8217;s pretty much it&#8230; For the offline level edition part.<\/p>\n<p>At run-time here&#8217;s what happens:<\/p>\n<ul>\n<li>If you enter a level zone, the level is dynamically loaded, but nothing is activated yet. Only basic (and huge) island meshes are visible<\/li>\n<li>If you get closer, the text file NavMesh is loaded, object activation starts and within a few seconds the whole level comes to life.<\/li>\n<li>When you leave the activation zone, all objects are deactived, but the NavMesh is still in memory (in case you want to come back \ud83d\ude09 )<\/li>\n<li>When you leave the level zone, the whole level is destroyed.<\/li>\n<\/ul>\n<p>I had to carefully study the distance at which each loading\/activation occurs. Because I don&#8217;t want to start activating very far away levels, but I still want them to be visible at a fair distance. I must also care about the distance between the islands: if 2 or more activation zones overlap,\u00a0 many levels are loaded at the same time, and this may seriously slow down the game.<br \/>\nIn brief&#8230; all this things are to be tuned and balanced.<\/p>\n<p>Sooooo&#8230;. This was a huge journey through incredibly complex features, but I now have an open world running at 50 fps in average and at least at 20 fps during loading. Now, you know why the game hardly changes between 2 releases \ud83d\ude09<\/p>\n<p>For the brave guys who read until here, here&#8217;s a video of the dynamic loading of the levels. I use a super-graple enabling me to move from one island to the other, even if the aiming is sometimes a bit difficult&#8230; And you may notice the distance between islands is perhaps a bit long.<\/p>\n<div style=\"clear: both; text-align: center;\"><\/div>\n<div style=\"text-align: center;\">\n<p><iframe loading=\"lazy\" title=\"World of Thieves - Open World\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/9JiAX03hLXQ?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<\/div>\n<p>&nbsp;<\/p>\n<div style=\"text-align: center;\"><i>To avoid seeing islands popping from nowhere when the player enters a loading zone, I added fog (classical trick in video games).<\/i><\/div>\n<p>&nbsp;<\/p>\n<h3>IX &#8211; Even more problems<\/h3>\n<p>For the sake of clarity I didn&#8217;t talk about every problem I had, but for the guys who would like to set up a similar structure, you have to know:<\/p>\n<ul>\n<li>The A* lib has some cache information about the NavMesh, and it&#8217;s sometimes necessary to &#8220;rebake&#8221;\/&#8221;rescan&#8221; the NavMesh after loading it. But this is absolutely not real-time friendly.\u00a0 I have to make additionnal tests, but it seems to be necessary only when playing in the editor when you modify a preexisting NavMesh. In the release, this may not be necessary.<\/li>\n<li>LoadLevelAdditiveAsync is absolutely not async in the editor. It freezes the game. You have to make a release exe to truly test real-time loading.<\/li>\n<li>Loading lots of level simultaneously totally messed up with all the automatic triggers I used to launch dialogs\/cinematics or whatever. I had to fix all those things happening at the same time while I was still far away from the actual islands.<\/li>\n<li>Because I now dynamically load levels, I also must dynamically save all the local modifications of the player: if he takes a pickable item, unlocks a door or a chest, I must keep track of it even if the level is unloaded before he gets to a save point.<\/li>\n<li>A few objects don&#8217;t support activation\/deactivation at all: clothes. This breaks the physics simulation in the best case, crashes in the worst. I had to find a workaround consisting in only disabling the cloth component instead of the full object&#8230; which makes my code look like crap.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<div style=\"clear: both; text-align: center;\"><a style=\"margin-left: 1em; margin-right: 1em;\" href=\"http:\/\/1.bp.blogspot.com\/-HsOnzxS24pM\/UwZviQ5H8NI\/AAAAAAAAHpc\/7t8-GfT48q0\/s1600\/20140220_221057.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/1.bp.blogspot.com\/-HsOnzxS24pM\/UwZviQ5H8NI\/AAAAAAAAHpc\/7t8-GfT48q0\/s1600\/20140220_221057.png\" alt=\"\" width=\"640\" height=\"307\" border=\"0\" \/><\/a><\/div>\n<p>&nbsp;<\/p>\n<div style=\"text-align: center;\"><i>I used clothes simulation to add huge flags above the Thief Guild (the only graphical change for 2 weeks&#8230;)<\/i><\/div>\n<p>That&#8217;s all for the crazy stuff. See you next time! Peace!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hi again folks&#8230; [WARNING: This is a crazy technical article. Pursue only if you&#8217;re mad] As promised in my last post, here is a pretty technical article to show you the kind of problem I run into and to give you a hint on how I spend my days losing my hair. This one is [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":227,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14],"tags":[],"class_list":["post-46","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dev"],"_links":{"self":[{"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/posts\/46","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/comments?post=46"}],"version-history":[{"count":1,"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/posts\/46\/revisions"}],"predecessor-version":[{"id":228,"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/posts\/46\/revisions\/228"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/media\/227"}],"wp:attachment":[{"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/media?parent=46"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/categories?post=46"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/oneiricworlds.com\/en\/wp-json\/wp\/v2\/tags?post=46"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}