Open-Source Software tends to invite a sense of adventure and discovery, which is why we have been actively testing all sorts of new solutions in the past. Apart from DDoS mitigation and CDN based asset delivery we also explore the bare metal side of things. One of the biggest arguments for OpenSim has always been its more cost-effectiveness and we are taking that idea further.
One idea is to isolate customers on their own virtual machine, this makes resource allocation simpler and can improve security. However, most of these systems trade overall low overhead for hard resource restrictions. As a result customers may still be able to overdraw on their assigned resources. The only way to counteract this is to isolate them on physical hardware. Sounds easy enough, but in practice cost-effective hardware for just a single simulator is difficult to find. This is why we are exploring technologies like ARM and specifically the Odroid line of small ARM-boards by Hardkernel. While ARM is a great technology and certainly capable of running OpenSim, ARM can be a bit tricky to deal with due to lack of support for some of the software we use alongside OpenSim. Another solution is the APU-board line of products from PCENGINES. These book-sized boards offer the more widely used x86 architecture and thus have all the usual software available to them. Size is key here, after all space inside datacenters is not cheap. There are many factors that will ultimately have to be evaluated before we can move ahead with these solutions, but that does not make them anymore less exciting to explore.
Apart from tinkering with hardware there is more general testing going on as well. From reproducing issues our customers encounter, to building new systems, requires testing and that brings along cost associated with running long term tests. To reduce this cost and to allow for a versatile testing environment capable of replicating very complex systems, we opted to run dedicated hardware locally at our headquarters. This allows for hands-on configuration and makes adding or removing parts very easy and above all affordable. Since some of you might be interested in the specifics of this system the next paragraph will go into some detail on what we have set-up. Most of this will not be overwhelming, but it does not need to be to provide adequate performance for testing.
Our main testing is done on an ESXi v6 cluster with various virtual machines. The cluster sits on an HP Proliant DL580 G5 with 4 Intel Xeon X7350 processors, 128 GB of Memory and 800 GB of drive space. Most of the ARM testing is done on the two Odroids with one of them serving as an external proxy to make navigating the testsystem a bit easier, because remembering ever-changing IPs is not easy. The Odroid U3+ ARM Cortex A9 1.7GHz, 2GB Memory all within a creditcard formfactor, the X2 has similar specifications and is pretty much the bigger, uglier brother 🙂
The PCENGINES Apu is more designed as a pfsense router, but the formfactor makes it a contender for stuffing a bunch of them in a server enclosure to reduce hardware cost. It features an AMD G series GX-312TC 1GHz quad core with 4 GB of Memory and three gigabit lan ports, all with a mere 12W TDP. Excluded from the above graphic are the backups, since they are just some older HDDs that happen to be spares, one of them has failed already. If any of you want to know more feel free to ask.
Now that all the geeks and sysadmins have had their fill back to something more easily comprehensible. We obviously also test on the actual systems that will be used to deploy these solutions on. This forces these solutions to be solid enough to be deployed on any hardware or system. It also helps expose problems caused by conditions we cannot simulate on our internal systems. Only after that should solutions be allowed into a production environment. Now, we are not completely free from pushing solutions without doing thorough testing on them, this is something that cannot be helped sometimes. We would be lying if we did not admit that this has happened, but frankly this is the nature of IT. We aim to keep service interruptions to a minimum and that usually causes such solutions to be pushed out quicker than we would normally like. Rest assured we always go back and re-evaluate these and make sure it was the right thing to do and make changes where needed. Fortunately for us with nearly a decade of experience between everyone at Zetamex Network we have become fairly proficient at this and our customers profit greatly from that.
Obviously we will be expanding on our testing equipment and will continue to conduct research on all things OpenSim. At the same time we are looking onward to what the future holds and with the rise of VR gaming and increasing interest of the education sector we hope you are as excited as we are for what this will evolve into. We would not be here if it was not for our pursuit of improving what we have and create new and better things, but there come times that this road has been rocky and innovation brings with it a sense of uncertainty. We hope that our approach of being transparent and sharing some of the technology and ideas gives you a greater sense of understanding our goals.