MyWorld Meetup – 4/13/19
Discussion:
- Q (FordLady): As you are working on MyWorld how do you know what to work on next?
- Answer: That’s a question I (Daniel) have to answer each week as I work on the code, and it can be a difficult question on when working on something the size of MyWorld. There are many different systems within MyWorld, many of which are interconnected and have to cooperate. These often have to be developed side-by-side.
- You can also consider systems like MyWorld to be built in “layers” – a data storage layer, a server simulation layer, a network layer, etc. Layers will depend on other layers, but they build on top of them and (if designed well) aren’t as tightly coupled. This gives a natural separation point and priority to different parts.
- This last week I’ve done a lot of major work in the data storage layer that will accelerate development moving forward.
- Similarly, MyWorld’s custom UI system (which has been very time consuming to build) is oriented towards fast and clean development. Tradeoffs often have to be made between spending more up front time to build something that will be very fast and powerful later and keeping initial development time shorter to stay on track for alpha release.
- You can also consider systems like MyWorld to be built in “layers” – a data storage layer, a server simulation layer, a network layer, etc. Layers will depend on other layers, but they build on top of them and (if designed well) aren’t as tightly coupled. This gives a natural separation point and priority to different parts.
- Answer: That’s a question I (Daniel) have to answer each week as I work on the code, and it can be a difficult question on when working on something the size of MyWorld. There are many different systems within MyWorld, many of which are interconnected and have to cooperate. These often have to be developed side-by-side.
- Q (FordLady) How long have you been working on MyWorld? Has the design changed much in that time?
- Answer: I’ve been working on MyWorld for around 6 years. The design has improved substantially over the years – it’s very different from what I started out on, and far more powerful. I set out to make virtual worlds better, but that improvement was initially very much at the code level – writing code that was cleaner, less bug-prone, and more efficient than current virtual worlds. As I worked on it, I became increasingly aware of some pretty severe design flaws and limitations in current virtual worlds, and began making design changes to address those. One thing led to another, and today the design is substantially more flexible and powerful than anything out there right now.
- One thing that’s different is the data model – older virtual worlds defined a set of database columns to describe a specific thing (a region, an object, a user, etc.) and if you ever want to add more data about something (or change something) you have to make many changes to both the database schema and the codebase. MyWorld is far more flexible about data storage. I’ve developed a theoretical model that starts with some basic “building blocks” and lets many different systems within the server flexibly handle very different tasks on the same common platform.
- The theoretical model is very similar to how mathematicians and physicists build logical models to describe physical or mathematical phenomena. This model (MWOS – MyWorld Object System) treats everything from entities to physical sections of the virtual world as generic “objects” and defines a common data storage and set of operations for all “objects.” This means we can quickly add or change the data we’re interested in storing or the operations we want to perform without touching the data storage layer.
- One thing that’s different is the data model – older virtual worlds defined a set of database columns to describe a specific thing (a region, an object, a user, etc.) and if you ever want to add more data about something (or change something) you have to make many changes to both the database schema and the codebase. MyWorld is far more flexible about data storage. I’ve developed a theoretical model that starts with some basic “building blocks” and lets many different systems within the server flexibly handle very different tasks on the same common platform.
- Answer: I’ve been working on MyWorld for around 6 years. The design has improved substantially over the years – it’s very different from what I started out on, and far more powerful. I set out to make virtual worlds better, but that improvement was initially very much at the code level – writing code that was cleaner, less bug-prone, and more efficient than current virtual worlds. As I worked on it, I became increasingly aware of some pretty severe design flaws and limitations in current virtual worlds, and began making design changes to address those. One thing led to another, and today the design is substantially more flexible and powerful than anything out there right now.
- Q (FordLady): Can you work on multiple different things at the same, or do you have to completely finish one thing before starting another?
- Answer: Yes, I often work on multiple very different aspects of MyWorld at the same time. For example, for quite some time I was building the server and scripting engine at the same time. These are two very different codebases, and to this day have not yet “touched” each other. (Integrating scripting is one of the things left to do before alpha – they were developed separately to keep the scripting language generic and not tied just to MyWorld.)
- Answer: Yes, I often work on multiple very different aspects of MyWorld at the same time. For example, for quite some time I was building the server and scripting engine at the same time. These are two very different codebases, and to this day have not yet “touched” each other. (Integrating scripting is one of the things left to do before alpha – they were developed separately to keep the scripting language generic and not tied just to MyWorld.)
- (Daniel): This brings me to another big change to the scripting engine that we’re exploring before release. Scripts run in a “virtual machine” – a special program that runs the programs written in the scripting language. There are three ways to write these virtual machines: (1) AST interpreters (easy and fast to write, run scripts slowly), (2) bytecode interpreters (significantly harder and slower to write, but run scripts much faster than AST interpreters), and (3) JIT/optimizing interpreters (bytecode interpreters + some extras – very hard and slow to write, but often run code much faster than the others).
- MyWorld’s scripting language is currently built on a completely custom bytecode interpreter custom designed just for MyWorld. As bytecode interpreters go, it’s quite fast and efficient and fairly well optimized. We’ve done some informal benchmarks against common LSL virtual machines such as Xengine, Phlox, and Second Life’s Mono-based VM, and MyWorld’s engine is consistently 3x faster (and in some cases up to 16x faster) than any of them.
- However, compared to industrial-grade language runtimes, it’s still a bit slow. We can do better.
- Up until recently, small teams could really only take on AST or bytecode interpreters. JIT/optimizing interpreters were out of reach to smaller teams because of their extreme complexity. However, over the last decade Oracle (the company behind Java) has been working on an exciting research project called Graal that’s finally coming to fruition.
- To cut a long story short, Graal gives scripting language developers the best of both worlds – we get to write AST interpreters (which are fairly simple and easy to write), but Graal automatically converts the (slow) AST interpreter into an optimizing JIT interpreter! Even better, Graal watches the scripts as they run, and as runtime conditions change it constantly adjusts and re-optimizes the running code into the most efficient form possible. This means that for the first time in computer history, single developers or small teams can, over a matter of months, take large, industrial languages and develop interpreters that are competitive with (and in many cases, much faster than) the fastest interpreters in existence now.
- We’ve done experimental work with Graal in MyWorld before, but kept our current design due to some specific requirements that are difficult to handle in Graal. Now that we’re closer to release, we’re revisiting Graal again to see if we can work out the complexities in a reasonable timeframe.
- In addition to making MyWorld’s scripting language faster than it is today, Graal also presents some very exciting opportunities for using existing professional grade debugging and optimization tools for MyWorld scripts, saving us a great deal of time and giving scripters a professional toolset from day one.
- Note/disclaimer: This is still tentative, and we may have to stick to what we have now for the time being. Graal is definitely worth exploring, and we’re hopeful that we’ll be able to use it, but there’s still a lot that needs to be done before we can definitively settle on basing MyWorld’s script engine on Graal.
- MyWorld’s scripting language is currently built on a completely custom bytecode interpreter custom designed just for MyWorld. As bytecode interpreters go, it’s quite fast and efficient and fairly well optimized. We’ve done some informal benchmarks against common LSL virtual machines such as Xengine, Phlox, and Second Life’s Mono-based VM, and MyWorld’s engine is consistently 3x faster (and in some cases up to 16x faster) than any of them.