Thursday, 31 October 2013

Thursday BETA DAY

A Jack Daniels Day

It is days like this I feel entirely justified to buy myself a little treat by way of a small celebration drink (Bottle -Ed). Call it dutch courage for the ordeal ahead of me, known only to those souls who have released software into the world and bracing themselves for the whirlwind.  Despite my repeated disclaimers, notes and warnings, I will still get feedback of the most vehement nature. But that's fine too, it means people are passionate about their game creation, and quite rightly too!

The Launch

The official BETA1 launch occurred just before 9AM Thursday morning GMT and gave Brits and Euro types a few hours head start on any Americans not pulling an all-nighter. Such advantage is now the stuff of history as the downloads are proceeding at full kilter, and slowly melting our two file servers :)  The great news is that despite a few trips and stumbles, the server is holding up well and has braved the worst of the day one weather. My compliments to our server guy, Paul.

The Video

It was always my intention to create a whistle stop tour of the software once the beta was out the door, and so I have spent some of the day creating just that. As I write this, I have handed the raw footage over to Rick who will process it into something more presentable and should be with you soon. It's not comprehensive, but it does address questions put to us at this early stage in the beta testing process. Hopefully it will be a nice intro to what it is you've been downloading all day.

The Remainder

As I have a few hours of today left, I plan to take a tour through the blog comments, you tube commands, forum comments, twitter comments, Facebook comments and intermittent emails that constitute the reactions so far. I may also find time to fix one or two small bugs that have been squarely placed in front of me.

The First Patch

I have absolutely no doubt I will be releasing some sort of patch to BETA1 users before too long, so I have set my stall up to prepare for it. I am keeping away from adding new media, and focusing on producing a better executable. Tweaks, bugs, stability, compatibility, usability will all be under the spot light for the next few days. We'll deal with fancy new assets and features once we've absorbed the early bumps associated with any substantial release.

Signing Off

All in all, we're quite pleased with the launch so far, and we also know full well that the work has really only just started. On receiving the beta you are now enrolled into the official development team and your comments and feedback will now shape the future direction of this project. Of course we have ideas of our own, but as far as possible we want the community to decide the order in which these happen. We have some ideas how to organised that and you will know more once we've investigated and created the framework we have in mind (watch this space as it's going to be cool). In the meantime, tinker and play, create and destroy, get a feel for the current version and get those blue sky minds working to imagine what amazing new features you want to see in what has now become YOUR game creator.

We have a good start, and a LONG journey ahead of us. As they say, it's not the destination but the journey that's important. If that is indeed the case, let us never reach or final destination and continue on into the infinite horizons of possibility. Before such grand adventures though, let's make sure we have everything we need and most importantly, not leave anyone behind. This is your software now, so please let us know if it does not do what you need it to do. We'll take care of the rest ;)

Wednesday, 30 October 2013

Wednesday Eve Of Battle

Well, We're Nearly There

A tale of two halves, with a meeting at one damnably early end of the day and lots of installing and testing at the other end. The goal; to produce a beta installer that stands up to the many PC systems attempting to use it throughout Thursday as the world wakes up in stages.

It's been quite a blur since March when we started putting the prototypes together and figuring out what we wanted from this new tool. I'm very pleased with where we are right now, and I am eager to get cracking with more tweaks, fixes, optimizations, new features and more betas.

The Meet

The meeting today was simply to confirm that the beta existed, and to double check we did not miss anything. Much time was also dedicated to the next large items on our list, and the order we should tackle them. Still top of my own list is performance, and I have a few fiendish ideas to implement in the weeks to come.

LOD BLOB - This idea involves creating a few patch-work quilts of low-LOD static entities using a single vertex buffer and texture image. These LOD blobs are then used to replace distant single entities used during shadow rendering and also for the far distance of any rendered scene. The upshot is that the draw call count is vastly reduced, and we also ensure only low-LOD meshes are used in the more distant shadow renders meaning less polygons overall. They cannot replace moving entities and transparent ones, but my guess is that there will be more static than dynamic, so we will see a saving.

TERRAIN POLYS - Another obvious target, and more immediate to me, is the strange behavior of the terrain render which on the face of it seems to be rendering 350,000 polygons for a rather flat boring (and empty) scene. It might be right, but I am going to investigate. Even though modern graphics cards consider 'polygons as free', for older cards the polygon count and fill rate play a role in the final frame rate score and non more so than one of the alpha testers who is experiencing some serious woes, even with effects switched off. Strangely though, my three Ultrabooks demonstrate a decent 30fps when extra features are switched off which suggests I might just get away with running Reloaded on integrated hardware with more optimizations and newer laptops.

SERIOUSLY LOW-END - Just to make sure the Reloaded software ran on the worst of the worst, I dug out a ten year old machine running XP and an AGP graphics card for my 'silly test'.

As you can see, a blinding performance metric there for my computer :)  Alas, my wonderful AGP card could not exceed to shader model 3.0 and the terrain and entities just rendered white but amazingly everything else worked fine.  

As the bloom shader also rendered blank, I could not see the frame rate inside the Reloaded test game, but it's perhaps a good thing I did not.  I did consider for a brief moment supporting shader model 2.0 and REALLY notching down the demands on the 3D chip, but I think it makes sense to define some borders to our low-end specification.  A ten year old XP + AGP rig is probably not within that scope :)

The Test

As I write this, I am doing all sorts of boring and necessary things like formatting Ultrabooks, downloading drivers, installing stuff, testing stuff, making small tweaks and trying installers again.  All necessary for a satisfactory beta roll-out, and good guidelines are being made for future releases too.

I have signed the final installer and I can report on a fresh Windows 8 install, you are asked for Administrator privileges and then the software installs without a real hitch.  As the DirectX web installer is silent, my WiFi driver was not installed and it skipped it, making the software report a DirectX error. I am now creating a new installer which uses a non-quiet DirectX install so you can see the connection issue and do something about it. I have also uploaded the web installer to Rick so he can host it alongside the beta file, just in case you want to install DirectX separate from the install process (or after it).

Right now I am installing the drivers back onto the Lenovo Ideapad Yoga 13 after the fresh Windows 8 installation which should allow my older installer to finish the DirectX web install properly, and then I will either confirm compatibility with the dreaded Windows 8 or not.  I will then move onto other areas of testing to ensure the file I select as the final beta file is the one that won't fall over.  Last minute changes can be SO dangerous!

Signing Off

As a reminder for those that do not know, on Thursday we will be releasing the first ever BETA of the FPS Creator Reloaded product to all pledgers of the project. It is thanks to your contributions and support that we've been able to get this far and I am excited to continue working towards making your game making dreams come true.

Don't worry too much about performance on your particular system at this stage as I still have plenty of speed-up ideas I can do to the engine, and as we get real world reports we can narrow our energies toward the most recurring performance drains. It will also be apparent which systems are too low-end for our software, or whether we need to scale back the visual ambitions of the software to allow such low-end compatibility.  Stay tuned to this blog to find out about my trials and tribulations in hunting down more performance gremlins.

I also have an inbox slowly filling up with non-performance items, such as a snazzy looking weapon and health screen HUD, but I think we all agree we need to crack overall speed, stability and usability before we go for the finishing touches.

Also a small note, the FPM level files are still in flux and will change for the next version so don't be too precious about any levels you create with this beta as they are unlikely to port to the new version. I will be adding a version control system to allow backward/forward compatibility with level files, but right now it's still got some old stuff in there than needs slopping out. 

If any readers want to pledge now and get ready for the beta download, you can find the pledge page here:

Enjoy the BETA and for the first week just see how far you can get with what we have now, and from your feedback we can produce a new version which responds to those individual concerns. It there is anything seriously unworkable, and your specifications are not too low (see above) then we will make those our top priority. No sense moving too far ahead if we're leaving some of our pledgers behind.  Can't wait to see the screenshots and YouTube videos of what you come up with!!  Be sure to share your exploits with the community so we can all see what you're creating :) Until Thursday!

Tuesday, 29 October 2013

Tuesday Installations

What A Day

It was so simple on paper. Clean up the beta files, make an installer, test it on a few test systems I have here, get it ready for a small demo Wednesday morning ready for a Thursday launch.  Damn paper!

Half The Day

In reality, I spent about 5 hours getting the sky to work again, and then to get the guns working again. Small almost incidental changes had the knock-on effect of causing the reflective sky to disappear, and for the gun geometry to fly all over the shop.  The solution was to write a sky shader and use some older gun models to get back on track.

The Other Half

Fear not however, as my hurdles jumped, I cleaned up the files so only Reloaded specific files are used, and I created a nicely clean installer which also uses a DirectX web installer to reduce download size and also runs the VC redist installer in case you are missing any DLL files the software requires. During our alpha testing, the full zip file was around the 500MB mark.  Thanks to file cleaning, and a better compression level thanks to NSIS, the new beta installer is around 190MB.  This should please everyone, not least our server which is going to get hit HARD on Thursday!  To make sure the download is not agony for you, we have created a silent mirror system so you will have two servers serving up your much anticipated beta file.

Signing Off

I have yet to start my compatibility testing with the new installer, but I am going to dig out an XP machine as I have received reports that all is not well on that operating system at the moment.  It's also prudent to test an installer under different conditions to make sure you see what we see when it's finally on your PC.

As I won't have much time left on Wednesday to actually code, I am dividing my time this evening and tomorrow evening either hiding or finishing functionality demonstrated in the editor and engine. One example is the weapon and health panels, which although functional, are currently using placeholders. Even though I have the actual art now, it came in a little late and my current task is installer testing.  Maybe if I don't tire later tonight or same time tommorow, I can add them in as they do look rather sharp!

Appeal For Rick

Rick has asked me to retract his email as the official place to send your woes to, but you can keep an eye out on the forum for news about how and where to post your findings. For this beta, I suggest you avoid submitting bug reports about the stuff that clearly is not working (as we already know about them), and make your feedback more general about the stuff that appears complete and whether we're heading in the right direction.

Appeal For Testimonials

Rick is looking for some quotes we can put in our 'Beta Press Release' about how amazing the product looks so far.  If you can email him directly at it will save him bouncing back to this blog every ten minutes.  Much appreciated!!

Monday, 28 October 2013

Monday March

Only A Few Days

There is some comfort that this version is not going into a retail box, being duplicated 10,000 times or going on stage at E3. It's also comforting that the intended audience are going help and feedback back, rather than criticize and condemn thanks to the pledge system of development.

Had a good day today of performance hunting, and found many islands of slow which I took great pleasure in sinking.  Everything from shadows, post processing, too many duplicated object rendering, redundant rendering and small improvements in code layout all contributed to the FPS creeping up from what we had a few days ago.

As you can see above, I can get 200fps on an empty terrain level. Also bear in mind this was an old shot and I have made improvements since.  Of course you will not need such a frame rate but it was necessary to get accurate measurements of each module in isolation which means switching everything off.

Graphic Options Panel

Thanks to the options panel, each user of the beta can also tweak which parts of the engine should be active, and in some cases the degree of support is provided such as the Reflection mode which when set to zero completely removes the reflection from the water. When set to 1-50 the sky is reflected but no objects or terrain. When set to 51-100 all reflection activity resumes. As we proceed, I will add more fidelity to these graphic option modes so that you can tailor the engine experience to suit your hardware, and ultimately your intended audience.

VSYNC Option

A timely request from the blog comments saw the introduction of a new flag in SETUP.INI file which allows the engine to boot the game in VSYNC mode ON or OFF. This means you can cap your frame rate to the refresh rate of your graphics card for super smooth zero-tear screen rendering.  By default I might keep this off for the purposes of the beta so we can see how high some users will get with their performance metrics.

Keeping The Sky

I also allowed the sky box to remain, even if the camera distance was pulled in to an extreme degree. I am not sure how to solve the issue of FOG vs SKY yet, as the two rather clash but I will let the beta process resolve that one :)

Alpha Users Beavering Away

Another fine shot from our sound guy, this time deep in ice mountain territory and close to the site of a crashed saucer.

It is my fond hope that I can do justice inside the engine to the ideas and imagination you guys are about to unleash onto the world with Reloaded. It's not just about getting performance, but getting it in the kind of situations your game will find itself.  Imagine the scene above as the site from which you will sniper the guards patrolling the complex.  Just imagine how much AI the engine will need to govern to show you the 20-30 enemies you have secreted around the base.  I have my work cut out, and why I think the employment of LUA (or other super fast script) and DarkAI over a multi-core solution is vital to create the kind of scenarios your games will eventually demand.

Signing Off

For the remaining few hours, I am going to re-align the AI with the various frame rates the engine can produce and get things like movement speeds, animation speeds and AI work-load balanced out. This should then allow the character behaviors to seem more normal and produce a good foundation for moving forward with AI. Tuesday will see the cleaning up of the beta files, production of an installer and testing on various Windows versions to ensure a fresh install works out of the box.

On Wednesday I have a meeting to look at the beta version from Tuesday evening so there will not be much happening on that day except talking and recovering from talking.  Just enough time and energy to upload the file to the server and batten down the hatches for a huge amount of bandwidth sucking activity come Thursday.

We don't have a bug base set-up as I could not find an appropriate one for the beta, but you can use the Reloaded Forum to ask questions about how to do things, but specific bug reports should be sent to until further notice (that is, until we have a more formal bug reporting system in place).  Avoid sending them to me directly as I will be putting filters in place so that all reports to me have been confirmed by at least two people, and a full reproducible report creating so I can focus on fixes rather than attempting to reproduce bugs.

Sunday, 27 October 2013

Sunday Slicerthon

Priority Performance 

As you know, Lee has gone paranoid on performance, and looking at the engine as one big speed making machine.  To that end, I have been working through the ideas I have to speed everything up. You can think of it akin to a statement from the previous blog comments which I quite liked:

Project Manager: "Amazing, how did you get a 1000% speed increase?!"

Coder: "Simple, I just switched everything off!"

To that end, I have added a new slider panel called "Graphic Options" which allows me to selectively switch of hungry sub-processes, and eventually allow different 'modes' of operation yielding various speed vs benefits to choose from.  Right now they just switch stuff off gracefully in real-time.

I have also extended the performance monitor a little more with polygon and draw call readouts, as this will also be vital in understanding whether objects are being rendered needlessly.

I also got a tip that the IDE was draining performance in the background, and on investigation it turns out a rogue mouse update function call was (and this you won't believe), reading a file for two strings and producing a formatted string from them. The code then proceeded to do nothing with that string! Amazing.  Anyhoo, removed that code and the IDE went from 25% usage down to 0% usage when in test game mode :)

Signing Off

My latest screen shot shows 144fps, which is what I get with an empty terrain and grass level with no reflections, shadows or light rays.  Turns out the test game was also being capped at 80fps which I have now fixed, allowing the engine to run as fast as it can.  Next is top investigate each of the 575,000 polygons and 191 draw calls to see what they are contributing.  Once I have stripped everything back to the bare walls, and introduced Graphic Option controls for them, I can start putting everything back in an optimized way.  

Not been brave enough to touch the NSIGHT tool yet. I also have the Intel GPA tool on standby too, so plenty of back-up if things get sticky.  I figured before turning to these assistants that I should proceed to implement my own tried and tested techniques for speed gain (that is, to remove all the stupid code) ;)  I have also managed to obtain a low-end dedicated NVIDIA card over the weekend (under 1000 score) so will be able to conduct some remote testing with NSIGHT a little sooner than anticipated!

Friday, 25 October 2013

Friday In Study

Small Blog Day

Nothing supreme to report today. Spent most of Friday studying NSIGHT and it's various profiling capabilities. It has the very cool feature of being able to remote debug, allowing for a true reading of performance on the application. The downside is that I installed the NVIDIA card in my development machine, so now I need to buy another card to install into the target machine for true profiling. No small inconvenience except that with the time constraints any remote debugging will have to be conducted after the beta release. In the meantime I am sure the 'local debugging' mode will be revealing and should vastly improve the performance of my GPU antics.

Signing Off

As we're in the final gallop to the beta, I will probably make a special weekend blog to reveal more of my discoveries. We have already determined that 90% of the slow-down is due to 'rendering stuff' which means NSIGHT has a vital role to play in identifying those areas that need the most attention.  Until then, please continue providing the performance shots as they are a great indiicator to me how low I should be setting the graphics card bar. Right now it looks like a shade over the Intel HD3000 benchmark, which is painfully low and requires me to change my views on what constitutes a mid-range card ;)

Thursday, 24 October 2013

Thursday Super Tired

A Pile Of Emails and Plenty Of Zzzz's

I set off this morning at 8:30AM and did not get back until after 5PM, driving both ways and utterly drained. The mission was a new pair of glasses for my continued coding, and of course they will not now be ready until November so back to square one almost.

I started going through emails but suddenly found my head loll right into the keyboard while reading and responding to emails.  I have dealt with the urgent ones, and look mournfully at the 14 remaining ones. Fortunately all juicy FPSC emails, but each one requiring some effort that does not directly contribute towards performance profiling.

Profiling Tool

I also had a recommendation from an alpha tester for a piece of software that compares your graphics card with every other published graphics card in the market and it's quite an eye opener where you find yourself sitting in the grand scheme of things.

I am curious where you guys and gals sit amongst the grand scheme of things, and if anyone would like to share a screen grab of a card they feel should be able to run Reloaded, comment a link here so I can see. The software is called PerformanceTest 8.0 from PassMark:

I am running a GTX 650 Ti Boost, which I consider a middle of the road card capable of running Reloaded once the performance tuning has been done. You can use this litmus test to see if your card is anywhere near this benchmark so you know how your system compares to the one used to develop Reloaded.

Signing Off

I am posting this blog a little early (and short) just in case I fall asleep at the keyboard and miss Thursday's posting.  I will continue chasing engine speed for as long as I can, and hopefully report some more improvements at the weekend. I am still quite happy that all my attention will be on the GPU activities as I have a bit of experience in this area and have a few ideas already.

Alpha Tester Sneak Peek

As you may know we have a few alpha testers producing screen shots in top secret to see what can be created with the early version of the software, and I received a few of them in the inbox, one of which I could not resist but show.

Notice the attention to detail in the mountain sculpture, and the stepping effect. All my mountains so far have been smooth blobby hills, and this shot from Mr H. really showed me what could be done with a little care and patience with the sculpting tool. All the more impressive as we have not finished adding all the sculpting brush modes yet!  Nice work dude.

Wednesday, 23 October 2013

Wednesday Battle: LUA vs FPI

Performance Day

Thursday I will be out of the office getting my eyeballs replaced, but today was all about performance profiling and answering emails, and then some more performance profiling.

The good news is that I have gone from 21fps to 45fps. The bad news is that I had to switch off all shadows to do it ;)  Better news is that I discovered that some file map communication code written to obtain Windows mouse coordinates from the parent IDE application was a serious drain, and upon removing it (only needed for actual TAB page views) rescued some substantial performance back from (or should that be for) the engine.

I have also extended the division of metrics within the main engine so I can see at a glance more fine grain detail of what the engine cycle is doing.

As you can see, apart from Misc2 (which I am still investigating but suspected to be a rogue value), the main energy draws are from the rendering step which is the GPU stall while it produces the visual you see above, and a little bit of terrain and AI processing. When I include the shadow rendering, all other performance considerations shrink and it seems the performance is entirely dictated by the amount of 'rendering and shadow rendering' being done by the engine.

In a way this is good, as I can focus plenty of effort on a single module for instant speed-up. I also have some great tools to assist me, and I know for a fact I can create multiple techniques within the shader to suit different system specifications (for LOW, MEDIUM and HIGH).  It's pretty good fun performance profiling as you have nothing to lose and everything to gain, and can be quite a buzz when you gain a few FPS.  Much more to come!

The Big Debate: FPI vs LUA

Ever since the idea was proposed, the topic of which scripting language is better for Reloaded has been pretty hot of late and I have been asked to step in and clarify the situation as to where we are.

I wrote the FPI (FPSC Instructions) language to quickly allow for simple conditions and actions within the game world. It was never meant to be a language as such, and a million light years away from something like DBP. It was ideal for quick one liners, such as turn on a light when you enter a zone, but became almost laughable when you asked it to coordinate the mental abstractions of an ally character following you around a combat scene. I would even cringe at having to apply a state value half way through the script, so that the subsequent conditions where skipped, only to set he state value to the intended value at the end of the script. If you read that in pseudo-code, you'd severely admonish the student responsible:


The FPI language scores marks for being one of the smallest language on earth, it's syntax almost entirely comprising of a colon and semi-colon. It is perhaps this simple format that has earned it such popularity, and when you start to study the built-in scripts, you're guaranteed to learn the logic within a few minutes.

The basic problem with FPI is that the underlying 'interpreter engine' which processes all the commands is written in DBP, and so each command is actually a trigger point for a multitude of other commands to fire before returning to the main thread. Multiply this with the many commands found in the script, and then by the hundreds of entities that are firing off said scripts and you have a substantial drain on overall performance. Indeed, one of the complaints levied against the classic version of FPSC was that too many scripts in a level would start to depreciate frame rate and force the the creator to clear their level of heavy scripts, divide up their levels or worse, scrap their ambitions for more attainable goals.

When we arrived at the decision to scrap the segment system and re-imagine a new way to construct large buildings and structures, it gave us the confidence to ask similar questions about other parts of the system. As fond as we are about FPI, we also know it is the silent assassin of many large and ambitious levels, and if we are to aim high we must look critically at each part of the whole.

Understand that we have not decided unilaterally to drop FPI and instate LUA. We still have to do some prototyping and profiling to see whether the integration of LUA is viable and improves performance of the engine. There is also the possibility of moving FPI into the core engine, executing the instructions outside of DBP. Therefore, you can rest uneasy in the knowledge a decision has not yet been made, and every effort will be made to work out which solution is best for Reloaded. To that end, you are welcome to continue this discussion, and hopefully arrive at a consensus as to ease of use and suitability.

A Question Of Ease

FPI is easier than LUA. LUA is much nicer than FPI. FPI is simpler than LUA. LUA is easier to read then FPI. The mystery is that all these sentences are correct. Based on who you are, you will find a sentence you can agree with. I think each has their charms. Unfortunately I have been in the situation of supporting two parallel paths of near identical functionality before and I must confess that supporting both can be agonizing and demoralizing. It was this past experience that lead me to conclude there should be only one. It was this decision that fired up the debate we are all enjoying.

I was fortunate enough to have spent a few years teaching at Masters degree level, and I learned a few things about 'ease of use' or more accurately 'ease of understanding'. Just as secret knowledge becomes commonplace once explained, so to does a script or language. If you learned FPI, you can learn LUA. If you know LUA, you can learn FPI. The underlying lessons are identical! If THIS happens, do THAT. I'm afraid that's what programming boils down to. If THIS, then do THAT.

From a personal point of view, and this is from a coder who's regrettably forgotten more languages that he presently knows, the syntax of LUA is very reminiscent of BASIC, which remains my all time favorite language. The idea of IF X THEN Y ENDIF really makes sense to me, and is the principal building block of any large electronic edifice. My prediction is that there will be more scripters for Reloaded than we ever had with FPI, not least because the scripts we will provide will become readable!

A final note addressing Reloaded ease of use. We are designing the software so that scripting by no means a requirement of the creative process. In fact, I expect only around 5% of users should ever need to break out into the world of scripting. The remaining 95% will find the combination of entity properties, sliders for in-game adjustment and level configuration more than sufficient to create a unique experience for your users. The scripting is there for those brave souls brazen enough to think they can create better scripts than TGC :) Rest assured that we will provide a wide variety of LUA scripts, assigned by default to relevant assets, dealing with all the game functionalties you require from opening a locked door, to commanding a battalion of soldiers to follow you into hell. You only have to look at our history of products to know that 'ease of use' is one of the goals we constantly strive for, and we understand that many of you have no desire to program, and why you use FPSC in the first place.

What's Next

You will want to know what happens now. What is the decision! For this I will need to create one or two prototypes, principally to test how the integration into the main Reloaded engine will be, and also to check whether the performance benefits are worth the transition. I also want to study the existing FPI language (which has doubled in size since the modders took over) and understand the scope of maintaining and/or moving it to a more internal engine that can process at higher speeds. 

Aside from the professional reasons, I also want time to play with the LUA syntax and get a feel for how it might work inside the Reloaded universe, and whether it is a good fit. LUA is not the only scripting language out there, and my researches might turn up something newer and more shiny! I also promise to try and resist the urge to create a new language for the occasion :)

Signing Off

Hopefully I have brought everything up to date, and hopefully I can bring you more performance improvement news Thursday evening, when I explore the deeper GPU realms and return with some worthy finds.

A Bit Of Fun

Rick decided to film an impromptu video as we concluded our meeting on Tuesday, and then put it on YouTube (bless him). Just in case you missed it, I have hosted it here so you don't miss out on any of my mindless waffle.

Once Reloaded is in a state where I can show my face in public, I am pretty sure there will be plenty more videos for you (and me) to cringe over.

Tuesday, 22 October 2013

Tuesday Talkathon

The Day

Starting at 8AM, I started talking and did not stop until 2:20PM. After a cup of coffee, I had a conference call and talked intermittently for another 95 minutes. After 8 hours of continual chatter, I pretty much exhausted my tank for self expression!

The Meeting

A productive meeting, which clearly trivialized some items on the project list and magnified others. The top priority agreed by all was the matter of engine performance, overruling pretty much every other feature consideration. The demo itself performed well enough, with only smoothing and tweaking to bring it up to scratch, but the lack-luster performance clearly affected the whole experience. My PC is no slouch, not a high-end unit (three years old at least), but I should be getting MUCH better numbers from the frame rate.

The conclusion was that I should spend several DAYS just on performance, and keep drilling down until every lazy function and process has been chased from the software.  To this end I began my profiling activity.


The art and science of profiling is the methods by which a programmer can determine which individual component of a larger more complex system is causing the greatest burden on overall performance. The identification of the most serious bottle-necks will invariably lead to rapid increases in speed, but finding them can be tricky.

To hand I have a profiler (all be it a simple one) in the engine already, tracking the time taken to execute each of the main processes. Right now the values returned do not make a whole lot of sense, showing 'AI and Physics' taking almost no time and 'terrain and misc' consuming over 80% between them. I also have the NSIGHT GPU profiler and compatible graphics card which will allow me to dive deep into the cost of the shaders and video memory I am using.  I also have a full Intel developer suite, allowing me to make some very detailed profiles of the CPU and threading activity of the engine too.

I have the basis of some insanely precise profiling, all I am requiring is some fresh eyes and daylight.  Having the green light from the management team to post-pone adding more assets and tweaks, and focus entirely on getting the engine as fast as possible is music to me, and I am sure that everyone on the list to receive the impending beta will also delight in this pursuit.

Signing Off

As I promised a screen shot, and I am sure the idea of a mini-game might have tantalized unduly, I attach one here for your consideration.

It shows all the slider panels the engine has so far, and as you see I am getting carried away with them. The backdrop shows a scene from my little game, just before the compound entrance and nasty enemies beyond. Past the buildings is a dock and freedom (well, a win zone).

What I really want to enter into evidence is not the completeness of the game or my make-shift scene, but the present FPS count of 21.  My hope is to bring you another screen shot of much the same scene but a considerably higher number.  All my energies this week will be to that end, and hopefully have some interesting stories for you as to where the bottle-necks where hiding.

Monday, 21 October 2013

Monday Prep

Monday Meeting Prep Day

Prepare for a very short blog, as I have spent the day preparing for a meeting on Tuesday and have left no time to blog about the days activities, just enough time to write this brief note and then a few hours kip.

A Small Game Goal

My goal today was to get the makings of a very small game in place, which meant adding workable AI, physics, health, damage on both sides, a small level to combat within and so forth.

The good news is that I managed it, but I must stress the 'small' and it has also highlighted some more areas I need to look closer at as well. I added some nice touches though such as gun flash for the enemy and player, which really makes the action pop!

Signing Off

A more generous blog will follow on Tuesday with some shots. Most of the day will be 'meeting mode' which basically involves me talking for seven hours, so I won't be fit for much that evening, but I will certainly see if I can polish off the edges of what I have done today and show you some peeks.

It is also very likely priorities will be changed and tasks shifted around as a direct result of the meeting, so be prepared for anything in Tuesday's blog!

Friday, 18 October 2013

Friday Guns N Ammo

Triggers Resumed

After a fun time yesterday with sliders, it was back on the main highway towards completing the main modules of the IDE and engine. The last IDE element was the trigger zones which had been mostly completed in the editor side but did very little in the engine side.

You will be pleased to hear that the trigger zones are now being monitored and as soon as the player enters one of them, they are triggered. At the moment, with no scripting in the engine, they don't trigger very much, so I have simply deactivated them when you enter one. It's a job of five minutes to connect this to something practical when the time comes.  I have also made sure everything resets properly so you can do it all over again during the many test games the software will go through.

Guns N Ammo

After a few editor tweaks requested by the powers that be (well, Rick) I was full steam ahead on my next mission which was to get weapons and ammo collectible within the game and have those collections properly trigger the gun to appear and arm accordingly. This allowed me to reduce the test game loading time as only weapons actually dropped into the level should be loaded.

I then spent a few fun hours adding guns and ammo entities, and of course guessing what ammo I had.  This of course led to the need for a status panel so I could see what weapon I had and the ammo and clip count for my selection. It was then I had a brain wave. What if I re-used the slider panel system for my ammo readout. It was all ready and waiting for as many panels as I wanted, so why not.  So I did.

Weapons Panel Is Born

Obviously I need to provide some new graphics for the status HUD, but the upshot of this code re-use is that I had my ammo readout with the name of the weapon above, but I could also TAB to the slider screen and 'MOVE' the panel as it was in edit mode. When I returned to the game view, the ammo panel was in it's new place of course.

I occurred to me quite quickly that this ability to shuffle about the ammo panel, and eventually health panel, lives panel, radar panel, local map panel and whatever else panel we add in the future is a superb way to keep your games customizable.  Gone are the days of recognizing an FPSC game because it uses the same health and lives status graphic in the top left corner of the screen.  Any user will be able to shift the positions of these panels within minutes of their first exploration of the tab views!  How cool is that.  

We would of course have to add some more panel controls so you can change the panel box skin, fonts, gadget positions, e.t.c. but building this into the panel system as an editable function means you can not only customize your status HUDs but the debug panels used to edit your game as well. I won't go as far as to say you can edit the editor, but we're heading in that general direction.

New Reloaded Website and Newsletter Launched Today

As some of you may know, we had the official launch of our new website today and we hope you will like your new home. It is crammed with the latest web technologies, and it is geared up to provide a flow of fresh information as it happens.  It's all centered around Reloaded and just like the software, we will be adding to the site as new content, features and requests come in.  For those who missed the launch here is the link:

You can also first the first ever Reloaded newsletter in there as well, which will be used to reveal juicy insights into the deepest corners of the engine each month.  For the first few months however we will simply be covering the gran unveiling of all these amazingly new features of the engine, and providing an overview of where it fits into the whole.

For the moment my blog will remain 'above ground' and the concept so far is that Rick and friends will mine my blog for newsworthy items and post them on the site and newsletter.  This means those users who are short on time can skim through the main site and those who want ever juicy detail can return here and read about it from the horses blog.

Early Third Party Art

"It had already worked out the existence of income tax and nice pudding before anyone managed to turn it off". That's right, even before the beta got released do we see the first radical customization of Reloaded from our sound guy, who not satisfied with desert and lush terrain went away and created himself 'Iceworld'.

The more I see how a few terrain textures can completely transform the whole scene, the more I think we will be releasing terrain packs as well as model packs down the road.  I also think there is much more to do with our beloved terrain system, and I am looking forward to seeing how far we can take the system in the months and years to come.

Signing Off

It's now the weekend, which means I can have the option of resting my tired eyes for two days, but we both know that is not going to happen. If the weather is nice I might dig a few holes, but rest assured I will be returning to the warmth of this project and cracking on with the next urgent thing.

As I survey my lists the next urgent thing is hacking in the trigger zone functionality for win, sound and story (basically, exit the game, play a sound and play a video).  As I don't have a script engine yet, I will hack in the code and choose the behavior based on the hard filename of the script. I have already done this to great effect with weapon.fpi and ammo.fpi. This allows the engine to be completed to a functional level without having a script system, and then the script system can be introduced in stages and tested to ensure behavior is not affected (whether it be FPI or LUA).

Once those basic functionality items are in place, I return to character AI and to see if I can make them less 'silly'.  Right now they silently fire six bullets then walk backwards before reloading, all very silly.  I will begin with the basics and adding obstacle avoidance, physics for the player and infinite ammo for the enemies to get some decent combat happening.  Should be even more fun!!

Thursday, 17 October 2013

Thursday Thunder

Where Did The Day Go

Fear not, my title was not reference to weather effects, simply the noise Thursday made as it thundered past me.  I set off with the best of intentions to add some sliders so I did not have to waste to tinkering with various color values to get the perfect night time color balance.  In the end I wrote the complete slider-menu module, which is now capable of producing multiple control panels across several pages, employing such gadgets as sliders and glass tube readouts, plus the cool feature of being able to drag them around the screen to suit your own custom edit layout.

Here is a shot of the current 'slider' panel system. I need the smaller fonts to be redone and perhaps give the panel edges a modern blue tint twist if there is time, and perhaps blue/grey sliders to match the Reloaded theme, but they are fully functional.

In the above shot I was attempting to create the night time color factors by drawing in some dark blue fog, set the ambiance to blue and the surface specular to blue.  I also added a yellow dynamic light for fun. I am still not 100% satisfied my shaders are where they need to be though for effective night time color play, but it's a small thing to tweak and I wanted the slider panels working to make the remaining process a little more pleasurable.

It's 3AM In The Morning

Literally so, and I WAS gong to add a few editor features for Ricky baby but time beat me.  The good news is that I can delete the 'ambient night effects' email which has been in my inbox for over a week, and I can soldier on with editor and IDE functionality work come Friday.

Proofing The Website

Also had the pleasure of reading through the new Reloaded website this evening, which Rick and friends have been slaving over for weeks. It's really come together nicely and uses the newest and latest web technologies to give the whole site experience a modern and relevant feel to it.  The real meat of the site however will come over the next few months when we start adding some seriously nice content and news feeds to it. Our plan is to make the site a hub of information, so that each time you visit you will get the latest and greatest state of play from the land of Reloaded.

Signing Off

I know I did not make it much past sliders, but the panel system is pretty versatile now, allowing me to conjure up as many panels as I want, putting any values of readouts inside them, and spreading them across multiple TAB view pages.  The ability to arrange the layout to suit your needs, and eventually have the editor save those preferences for all time will make it a very useful addition to your game making toolbox.

A Community Question - LUA or FPI

An interesting comment was posted as to whether the FPI language would be improved, and from previous feature requests the idea of using LUA as a substitute was aired.  My question in yesterdays blog, and continues here. Do we throw away the FPI language (and subsequently all scripts ever created for it) in favor the language LUA which is a more mature language offering faster processing, but means we have to start our script mountain from scratch.  I have no problem continuing to support and grow FPI, but just like we did with the 'segment system', before we launch into another five years of glorious FPI scripting, should we be thinking bigger?  I've only had the briefest exposure to LUA (I stopped learning new languages as it tends to push out old ones I still use) but by all accounts it has a range of debugging tools available for it, and there is a good chance I can compile and run the LUA script internally instead of through DBP code.  The upshot is that you can not only have it running 'significantly' faster than current entity logic processes, but it can be made to run on multi-core systems for even greater speeds with thousands of entities. You must also consider adding LUA is a substantial piece of work, so saying yes will also mean scripting capability will be delayed, or that the feature will delay other needed components such as the 'Room Blob' system.  Something to think about though!

Wednesday, 16 October 2013

Wednesday Waypoint Zones

Another Bottle Please

Yes indeed, thanks to your combined eyeballs we pushed past 1500 daily views and got to 1599!  My thanks for your interest and all them double visits ;)  My goal was to hit this number, but I did not realize I would get there so soon, so I am setting a truly high goal now of 2000 daily views.  It's a fun little game!

The Progress

Today was trigger zones, but not just any old boxes this time.  As the terrain and entity placement presented a whole bunch of new challenges, we had to make sure the trigger zones could cope with the new non-grid world.

As you can see, the new trigger zones (also called waypoint zones) can be contoured to any shape to allow any kind of perimeter to be defined for your game.  We removed the hurt and heal zones which we felt where a little silly, leaving Win Zone, Trigger Zone, Story Zone and Sound Zone.  Even though a single zone could handle all functionality through scripting, color coding general concepts will help when your levels get larger.

The white circle in the middle of the zone is a handle you can use to drag the whole region around, and right clicking will enter a properties menu where you can specify the particulars of what that zone will do.  The little yellow stars, which might be under the graphical axe, allows you to move the edge of the contour, add more control nodes and delete nodes.

Next Things

Now I can create, delete, save and load these new trigger zones, and the whole markers menu has been cleaned up, Thursday will be about making these zones functional within the engine itself.  The biggest of them of course will be triggering an AI character to respond to your entry into the zone. I think I will have some fun with that!

I also did some other things like change all the assets in the markers folder so new graphics and geometry are used to give it a Reloaded feel.  You don't really want to see any throwbacks from the classic version hanging around.

Signing Off

It's just gone midnight now, and right on cue my eye efficiency has dropped. Before I clock off though, I want to quickly try a few night mode lighting changes. I think the dynamic lights will really pop if the day night cycle really brought in the darkness.  It won't be anything new or magic, just some changes to things like ambient color, fog distance, sky details, and anything else hanging out the edges of the current shader.  With some luck it should look pretty good as the night comes in and gradually changes the whole scene. I like to think you could get two different looking games from the same set of assets, simply by adding night elements.  Shots on this to follow soon.

Tuesday, 15 October 2013

Tuesday Alight

Dynamic Lights Are In

Yesterday was the creation, placement, editing, deletion and control of the AI characters via waypoints. Today was the addition, placement, deletion and shader integration of the dynamic lights.  A particularly popular request for many moons, and thanks to a little infinilight update work and a lot of agonizing, they are now working in the editor and engine.

Before you launch an attack, I 'deliberately' went OTT on the color levels here to illustrate the use of multiple dynamic lights and their reaction against the terrain and entity normal maps. You probably won't see this type of extreme lighting again until we do a SciFi pack, where such two-color styles can really make your levels pop!

Something To Discuss

The system has the natural barrier of only really having a limited number of what I call 'hardware' lights. That is, only three lights actually exist inside the shaders, but the engine can make use of them to create the impression of many more lights in the scene.  The downside is that the user has to place them and space them so the light transition as you move about does not look too jarring. I know you want infinite lights in the engine, but trying writing a shader that can accept 10 or 20 lights, for each pixel rendered by that shader, and you will come to know what performance hit really means.  I have started with three fixed lights as that is enough to demonstrate the infinilight system and a good starting point for discussion. I also don't want to go ape just before performance tuning so a sensible number had to be selected, and three allows two lights to always be in force, and a third light allows the transitional play of the third and forth closest light to the camera.

One really cool byproduct of 'real-time everything' is that when you place the light down in the editor, it actually lights everything as it would in the real game, and the effect of the normal maps against the dynamic lights is something to behold!  I think you will like!

Other Visual Touches

Our artist Mark came through for you guys again and went ahead and improved the bloom shader to include SSAA (screen space anti-aliasing), and I can testify that zooming into a screen grab the pixels are indeed anti-aliased. It seems like dark magic to me, but it works so I ain't knocking it!

As the first shot was a little dark, so I provide another. I must confess to having a little too much fun creating these poses.  Hopefully you can see the anti-alias at work if you zoom in enough.  Not sure if Google blogger is squashing these shots or not, but the original size is 1360x768 or thereabouts.  You will find anti-alias around the edges of things and attempts to remove what are called 'jaggies' from the scene as a whole.

Signing Off

I am also pretty sure that these 'backup glasses' I am being forced to wear at the moment are giving me premature blurry eyes as after about 8 hours of keyboarding, everything gets fussy.  I am also the proud owner of one headache per evening which seems to coincide with these damn specs. I am also the proud owner of a moved appointment so my new eye test only happens on the 24th.  How crazy would it be if I finished off the beta half blind. At least I would have the perfect excuse in almost every quarter ;)

Monday, 14 October 2013

Monday Wayfinding

The Final Mile

Well not quite, and very probably far from it, but it's interesting to look at the next two weeks as a sort of ultimatum. Plenty of lose threads, each as vital as the others and all needing attention.

Today I worked on the 'obvious' missing elements from the IDE, starting with the way point system. Currently these are the last toolbar icons with no functionality so I focused on this. As part of the work, I first ensured the character AI entities retained their position and rotation state between test games so that when I eventually assign them paths via way points, they will start in the right direction and animate as they mean to go on.

Waypoint Mystery

You would think this feature should be a simple cut and paste, considering the outward functionality already existed in classic, but for all the clever and creative lines of code, I could not coax the waypoint objects from appearing. It was as though a mighty void swallowed their geometry the moment it was created. Turned out waypoint objects are based at 0,0,0 and their mesh coordinates are absolute, and also still based in the negative Z space. The bottom line is that I finally managed to get them to appear, and subsequently had them following the height of the terrain.

Next Steps

Now I have the waypoints being created, edited and deleted as normal, the next step is to tie them into the AI system which will require some new AI code to assign waypoints to DarkAI paths.  From there, it should be a simple matter of assigning paths to entities and we should see characters patrolling custom waypoint routes for the first time.

Other incomplete features include adding dynamic lights, contour marker zones and checkpoint marker zones.  All very exciting and will bring the front-end to almost full functionality (from a buttons point of view).

Beyond the IDE aesthetics, I also need to add weapon, ammo and health pickups pretty promptish as launching test game for the first time loads ALL the guns which is a little needless during these early test runs.

I also have a small mountain of new art and asset pieces to integrate, which is also exciting. One asset set includes brand new water splashes, ripples, decals and other material to create a truly lovely triple A water effect for when the player walks through the water, shoots into it or throws something in. Having studied a few YouTube moments of what happens when game meets water, I think we have the graphics required to pull off something similar.

Signing Off

I spent two hours digging a rather large hole in the garden and after some food and a small snooze, I am pretty zonked.  I am going to make myself a brew and perhaps put another hour in and then call it a day. At only 8:30PM it's the earliest I've stopped work in many moons but I think it will put in squarely in the day shift for the next slew of days and allow me to work in TGC normal hours during this critical crunch month.

You will notice the lack of a screenshot today. I think we all know the visual level we are at right now, and rather than throw another Christian into the ring, I'll save my next visual for something more in line with the feedback received so far. You may be pleased to know I've knocked the bloom level down from 0.4 to 0.6 threshold and reduced the light ray value by half, so the next shot should be a little easier on the eye when it comes to the phenomena of 'Oblivion Bloom'.

Saturday, 12 October 2013

Saturday Continued

As I Was Saying

My Friday blog was cut short due to a little fatigue on my part, but I did have more to say. I resume my monologue in a rare weekend blog post and I want to start with a nice screenshot of what I have been working on in the last 36 hours.

As you can see, I have been working on blending the character shader into the rest of the visual feast.  In doing so, I found it convenient to bring in the AI and Character physics module as well, which went in a little too smoothly. It was this lack of pain that caused me to carry on and run out of steam.

I returned to a decent place in the code, and could carry on with the AI some more. Right now the character can initialize and immediately connect with the DarkAI brain that was running in the prototype. Shows my plan to create prototypes that also directly plug into the main engine was the right one.

The new code is not without it's challenges though. The character seems to stretch some internal limbs, hides the in-hand weapon and has a strange glitch which is very subtle to detect, thus fix.

Bring In The Bullet Ray

As these glitches are vague, I thought I would connect the other necessary part of the AI system which is the ability for the player to fire the weapon and have the AI character respond to this impact.  This will allow the creation of a video which shows running around a small scene and knocking over characters. Almost there, just a few more lines and tests and it will be in.

As I copy, paste and correct the legacy 'entity hit' code from the classic FPSC, I respect the fact it's quite huge now, and probably not highly optimized. The current code uses an entity loop and checks every entity against a ray from the players gun. A bit crazy really. Not too bad in overall performance, but this idea of an entity loop multiplies out and I cannot shake the idea that there is a better way, something more spacial in nature might be better all round.  For now I can keep the current approach, but I will be looking for other options.

I did do a little optimizing though and the main character animation and combat system runs on a smaller loop, only including characters in the level.  

You may notice I also worked on the character shader, cleaning up the normal map shader code and more significantly added self-shadowing for them.  As character animate 'all the time' the initial shadowing artifacts from the depth camera resolution made the end visual pretty bad. By tweaking an epsilon for shadows close to the casting pixel I made it a little nicer at close range. I still have to do more distant shadow tweaks, but it's a good start I think.

Signing Off

I am just finished off the ability of the editor/engine to allow the test game to kill the characters but restore them when you are back in the editor, and the ability to dive back in and shoot them again. This will allow more testing and start the process of my 'multiple test game' nodes which will allow you to go into the test game as an editing mode. Things you do in first person view can remain in place and saved back in the editor. At one point the characters did just that, but it's not quite right that your well placed level assets start walking off :)  Again very pleased the character AI just dropped in, which means I can dive straight into the other million connections required to complete the character system.