Saturday, May 30, 2015

NA Merge on for June 2, plus New Engine Screenshots!

As we finish ironing out the kinks from this month's EU merge, we are happy to say that the NA merge is on schedule and set to go on June 2nd! The Citadel launch has seen a large increase in activity from both new players discovering APB and old vets returning to stake their claim, and we are excited to do the same for our NA players.

NA Server Name Leaked?

Many of you that are active on the forums saw a leaked name for the new server - "Liberty". This was a result of some of tests that involved momentarily putting a new server on display, which at the time was using Test server name. Good or bad news (depending on what camp you are in) the new Live server name is not Liberty, but something else entirely, which will be revealed on launch day!

New Engine Screenshots

Our team is rapidly making progress on the long-awaited Unreal Engine upgrade toward both the Console and PC releases going live this Summer. Here is another set of screenshots that share a glimpse of the dramatic visual improvements you can expect when it arrives:

As we get closer to the engine release, we will be spinning up a new OTW (online test world) for the PC game, where we will invite select community members to test the new game engine. More information will be released on the forums as we get closer to the first OTW test. The initial engine test will require that your system has DX11 (including a DX11 capable graphics card) and Windows 7 64-bit or later. Once these tests are completed we are still anticipating making a back-port to support DX9/DX10 cards, but the game will likely at minimum require 64-bit only systems going forward. 

Making the game 64-bit only solves several long-standing memory management and performance issues that we are happy to finally put behind us. But it does mean that a small percentage of machines currently used to play APB will need a new OS in order to continue playing.

Final Thoughts

We are excited about the upcoming North American server merge as we think it will be great for all NA players. Also we are very excited to see the return of many players after the EU merge was completed last week as well as the return of some players that took a brief GTA-V hiatus in the past month (which may have included some of our own team that went to check it out). Also a huge thanks to those who continue supporting the game through in-game purchases, giving us the ability to complete these great upgrades and continuing to deliver the unique creature that's APB. 

Wish us good luck on the NA server merge on Tuesday, and we are thrilled to finally being that much closer to the PC engine and the Console versions nearing the completion of their very long development journeys. As soon as we can share all release dates we will, the timing of which is in part driven by when our publisher announces the final Console release date.

See you in San Paro!

Thursday, May 21, 2015

Merge Cleanup and Final NA Merge Date

Hey San Paro!

As players continue to merge characters into the Citadel, our team has been working hard to tie up loose ends this week.

The TL;DR version; while 99% of characters were moved successfully during the EU merge (you can read more here), the remaining 1% had various issues, which means our dev team has been deep into sorting out the unique circumstances that lead to these conflicts. The team is updating the merge code as they go along to avoid those same issues during the NA merge. To give them sufficient time to sort out the current customer support tickets and updating the code, this unfortunately means we have to push the NA to Tuesday, June 2. It really does suck we have to move it, but this will allow for the smoothest possible NA merge.

Current Citadel Merge Status

Here is a a list of merge issues we have either fixed, or are working on fixing:
  • Not being able to merge certain characters, or unexpected behavior during the character select screen 
    • Fixed
  • Mail in inbox being reversed, from oldest to newest
    • Fixed
  • Unable to restore deleted characters from EU1/EU2
    • In progress. While we still store deleted character info, retrieving and transferring this data from the old servers is a complex task. We are building a set of tools to streamline this process.
  • Clans being renamed
    • In progress. Some clans that contained banned characters had their names mistakenly changed during the merge process. We are working on fixing these clans.
  • Unable to create new clans
    • Fixed. We temporarily disabled this feature while investigating an issue where new clans could potentially take the names of pre-existing ones from EU1/EU2. This is now re-enabled, and un-merged clans from EU1/EU2 will have their names reserved until they merge.

NA Merge Date

As mentioned earlier, our goal is to make sure that all the kinks are ironed out before we kick off the Colby/Joker merge, and we are now set to start the NA merge on Tuesday, June 2. More details will be posted as we approach this date.

Thanks for your patience, and we look forward to another successful server launch!


Tuesday, May 19, 2015

Welcome to the Citadel!

[Note: the merge has been completed, and you can follow the "merge clean-up" thread on our forums here:  !]

After weeks of planning, building and testing and we are pleased to announce that the new EU server - the Citadel - is now open for you to play in!

A few weeks ago, we let you the players vote for the name for our new server, and Citadel was an overwhelmingly popular choice. We agreed, and thought this was a fitting choice - the Citadel is now APB's largest server, boasting 3.1 million characters, 193845 clans across 2.5 million accounts. 

What to do Next

Just log into APB!

As we mentioned in an earlier post, most characters will be merged on-demand as you log into the new server for the first time. This will only take a few seconds in most cases, but some accounts may take longer, especially if you are the first member of your clan to login. Once you see your characters in the selection screen, the process will be finished and you can start playing.

Earn Your Special Treat for Being an Early Bird!

Simply log back in to the EU server to automatically merge your accounts before June 30, and you will receive the special Early Merge Bonus Package, featuring 3 X Joker Mystery Box 17's, 5 X Premium Days and ONE Exclusive Title! 

Later This Week: Get The Juggernaut Strike Pack Season Pass

As a special treat, the eagerly-awaited Juggernaut Strike Pack will be available this week. Compared to our previous bundles, this season-pass style pack will bring you more, exclusive content than ever before from May through August. 

The Juggernaut pack will include:
  • Bighorn/Mammoth Vehicle
  • Frenzy Assault Rifle*
  • Strike & Blend Weapon Skins*
  • Urban Strike Clothing Set
  • Manic Submachine Gun
  • Rabid Light Machine Gun
* denotes pack exclusives

As an added bonus - players who pick up the Juggernaut pack will get all of the Bonus items for that phase and later!

Happy Hunting!

We sincerely hope you enjoy playing in the new Citadel and look forward to the intense matchups, new friendships and clan activities from here on out.


Monday, May 18, 2015

EU Merge Countdown

Here we go! We're ready to start the EU server merge on Tuesday, May 19th at 200 AM BST (May 18th 6:00 PM PDT). The EU servers will remain down until the server merge has completed, once it has completed we'll post on the forums and social media to let you know! We'll also be creating a separate thread for EU players to post after the merge should they have any unexpected problems. We're hoping that everything goes smoothly and the thread won't be needed!

Joker, Colby, and Han will go down for maintenance at 8:00 AM BST (12:00 AM PDT) and be back in service at approximately 11:00 AM BST (3:00 AM PDT).

Once the EU merge is completed we will determine the best day to execute the NA merge based on the initial merge results (and if there are any new cleanup tasks the team has to perform after the EU merge).

Next Up; Server Name Unveiling and the Juggernaut Strike Pack info.

Keep your eye out for a new post once the merge process is complete, where we will unveil the new name of the EU server, and where we will share the details and preview images of the upcoming Juggernaut Strike Pack that arrives later this week.


Friday, May 8, 2015

The Big APB Roadmap Update; new EU hardware, server merges, engine upgrade, console games and surviving GTA-V (again)

(note: read about the latest server merge schedule here)

A mighty northern wind...


As we recover from the storm that blew in across our northern border from Los Santos two weeks ago, down here in San Paro we are still plugging away on the jam packed schedule we are facing over the next three months.

First of all, a huge thanks to everyone who stuck around in APB Reloaded during the last two weeks, and to those who simply decided to come back after checking out some of the great storyline gameplay in GTA V.

Second, a very warm welcome to the slew of new players that found APB during this heightened interest in urban city combat! We admit we may also have wandered over to that "neighboring" city of Los Santos for some exploration, before we returned to APB with great ideas for how to keep APB unique and different, especially by focusing on and refining APB's multiplayer combat both on PC and Console.

Also, it has been interesting to see the competition step out of the safe confines of the console world with their new online PC experience, and attempt to deal with the crazy challenges that come from being a PC online game. We wish them the best on handling those situations, as it seems they have had their hands full the last few days.

On our end, as a small team juggling a lot of new releases and changes, we wanted to get you up to speed with a new roadmap post, particularly as we depend on your ongoing support to keep APB running, for many more years to come.

The Roadmap...

We are really excited to share the new roadmap, with its list of really important releases being worked on right now. We will also make a few individual blog posts for some of these key features as we near each launch.

May-Aug 2015 Roadmap:
  • Completion of all EU District Server Upgrades (NOW LIVE!)
  • Joker Mystery Box 17: Urban Legends (NOW LIVE!)
  • EU PC Server Merge
  • US PC Server Merge
  • Espacio Season Pass
  • Refer-A-Friend
  • CRISP: Community Moderation System
  • New Matchmaking Release
  • PC Engine Upgrade Release
  • PS4 Launch and Xbox One Launch

Live (May 1): All EU districts have been migrated to the new server hardware

So after what feels like thousands of iterations, all EU action districts are now running on the newest upgraded district server hardware. So far the hardware performance has been great. Of course we have seen problems with some malicious individuals still DDoS-ing things (making things feel slow), but when there are no network storms going on, we are now running a steady 33ms frame times on all action districts. That's exciting and should be a great experience improvement for all players.

Live (May 6): Joker Box 17

We are excited to release the next Legendary weapon model series, the Urban Legends, as part of the next Joker Mystery Box. Each of the 3 fully-automatic secondary weapons (Bloody Mary, Jersey Devil, and Hitchhiker) comes with their own unique modifications and deluxe skin designs. We hope you enjoy using them as much as we did crafting them!

Going forward: 

US and EU server merges

After massive amounts of testing, the very hard desicion was made to move the EU server merge from May 6 to the week of May 18-22 (the same week as the US server merge). More details here.

Aside from the ongoing engine update work, completing the merge preparations is currently our very top priority, not only to prepare for the upcoming console releases, but also to make sure that everyone will have a great experience on the PC version going forward. The merged servers will effectively double the current population, but will also set the stage for the "Server-Wide" matchmaking system that is being worked on. In short, the server merges are a critical cornerstone to a massive amount of upcoming new launches.


Since releasing the Heavy Duty pack in December, our art team has been crafting a bold new body kit design for the Espacio vehicle, along with matching clothing items and weapon.

We are planning to release the Espacio Pack at the end of May, but with a bit of a twist. Since we release a pack about once every 3-4 months, we are planning to release the upcoming Espacio pack as a "Season Pass," where for the same price as a normal content pack you will get even more content over a period of time: pack exclusives, Joker Boxes and other items every month.

The earlier you buy the more bonus items you will get. The goal is to make new content and extra bonus items available to you each month as a Pass holder. And also to pave the way for a fixed predictable long-term release schedule for these large content packs.

CRISP Community Moderation

Whether in missions or hanging around the Social District, APB is at its core driven by constant and positive social interaction between each of the players. Each day our team strives to create an inviting atmosphere so you will not only feel welcomed among other APB players, but also invite your friends to join you here. Still, we realize that our small GM staff is not always able to properly address all the harassment, scamming, and offensive language that takes place and much more needs to be done. So over the next couple months, we are introducing two major initiatives that will both tackle this toxic behavior head on while bringing the next generation of players into APB.

First, we are nearing completion on integrating a sophisticated community moderation solution called CRISP Thinking into APB. CRISP works seamlessly in the background to automatically detect and moderate toxic players anywhere in the game, then take instant action as necessary. Unlike conventional filtering systems, CRISP is able to look beyond 'bad' word usage and examine the context of entire phrases. Harassment, hateful speech, and even scam attempts can be detected and acted upon in real-time while keeping positive players protected. We are aiming to introduce CRISP this Summer.


We are finishing up our referral system that will award prizes for bringing your friends into APB! While there are other referral systems out there, APB's Refer-a-Friend is the deepest, most rewarding and most convenient system devised in a game like this.

As you invite friends and help them achieve milestones in APB, you will earn points that you can choose to spend across 31 different prizes at launch! After over a year of development, designing and balancing, we are expected to unveil the Refer-a-Friend system at the end of May.

Revised Match-Making

This summer we are looking to launch our new matchmaking system we first described here, internally named Dedicated Automatic Matchmaking (D.A.M.). By introducing the ability to match teams with equal ratings located on different servers, the new system was designed to promote more balanced, frequent, and engaging matches across all districts so that you can truly enjoy the unique multiplayer gameplay that sets APB apart from the competition.

Combined with a more concentrated population, D.A.M should vastly improve your chances of having those elusive, exhilarating and insane "APB moments" that will stick with you for a long time.

PC Engine Launch

It feels like an eternity since we began the complete rebuild of the underlying APB game engine. When we started the project back in 2013 (in fact if you go back to our October 17, 2013 blog post we shared that the engine upgrade project had begun July 17, 2013), we were wildly optimistic that we could complete the project in about 9-10 months.

That was insanely optimistic for a whole host of reasons. First we didn't count on the amount of other ongoing work APB needed just to stay operational day in and day out, so when the team was pulled away to solve other problems, the engine work slowed down. But most of all we hadn't anticipated the ungodly amount of work required to undo years of custom systems that would no longer be part of the new engine.

The Console game actually helped the team refocus on the engine update since the upcoming Console versions absolutely had to use the later version of the engine.

Below are screenshots from the current game (left) and the updated engine (right) that we are currently running in our office. Clearly the new look will be great for the game, but more than the visuals, the simplified workflow will dramatically improve our ability to actually make new meaningful content going forward for all three versions of the game.

Our current expectation is that the PC engine upgrade will go live either concurrent with, or just after the release of the Console games. It's been an insanely long road to get here, and when the engine and the console versions are launched we will finally breathe a sigh of relief, and then we will go right back to building new things for the updated game!

The upcoming Playstation 4 and Xbox One versions of APB Reloaded

Finally, we are also getting closer to the launch of our first ever console versions of APB Reloaded! While we don't yet know the final Console launch date, all the teams involved, i.e. our internal team, the external console development partner's team and the publisher's team, have been working around the clock to get the Beta and Certification versions readied for submission.

Unlike the PC game which can basically be launched "when ready," the Console versions require a long series of approvals from Publisher, Microsoft and Sony before anything can go live.

As soon as the formal launch date has been finalized, our publisher will be sharing that information through their channels, and after that we will be able to share that information here as well.

Thank You

Finally, it’s an awesome privilege to work on a game with this kind of longevity and ongoing community support. I am thrilled with all the changes that are coming, the big leap it will mean for us in adding new content with the new systems, and I cannot thank you enough for your continued support of the game. See you in San Paro soon again!


Tuesday, May 5, 2015

Merge Update

Hey San Paro,

Just a quick update on our progress as we prepare for our massive server merges. First the TL;DR version; after a lot of testing and simulated merging even through today, we have decided to postpone the EU merge this week and are moving it to the same week as the previously scheduled US merge (the week of May 18-22), with the EU merge likely occurring at the start of that week, and the US merge occurring during the second half that week.

The fundamental concern; if this merge runs in to any issues, it would be nearly impossible to reverse once it's set in motion, and any post-merge repair job could take a very long time to complete. At this point we don't have enough test data showing us that the merge will succeed in all cases.

Therefore the decision was made today to give the first merge at least 10 more days of QA and testing time before we pull the trigger.

An insane number of merge simulations...

These server merges may be the riskiest operation we have ever undertaken with the game, other than maybe taking over the game in the first place, since it's going to be nearly impossible to undo or reverse once it's been kicked off.

Over the last two months we were busy writing the merge code and revising database systems to handle the size and complexity of the merge, while also running simulations of every type of merge we could conceive of.

Our first approach "worked," but yielded merge simulations showing the game might have to be offline several days and nights to complete the merge! Therefore over the last couple of weeks we reversed course and designed a new type of merge that lets us bring the game back online in less than 24 hours.

Old Plan: Merge 5.7 million characters in two rounds. 

Since APB:R launched in 2011, a LOT of characters have been created by you and your friends - over 5.7 million characters to be exact. For each one of these characters we store each character's customized appearance, vehicles, symbols, themes, inventories, friends lists, in game mail and a whole host of internal parameters that relate to each character's relation to all other characters in that world, like clan membership, the "author" of any items in your inventory you bought from the auction house etc. 

Suffice to say, reliably moving this data is a colossal undertaking in terms of both scripting, hardware configuration and sheer processing time. As you might expect, the most time-consuming pieces to migrate are related to character customization and inventory.

On the technical side, each of our 5.7 million characters "own" a set of around 30 distinct, long XML blocks, and each of these must be converted, validated and potentially modified as they move to the merged database.  This is why our last tests using this method showed it would take a colossal 3 days (72+ hours) of continuous downtime to merge the EU servers, and slightly less than that to merge the US servers. This would simply be an unreasonable amount of time to keep everyone out of the game this week.

New Plan: Merging On-Demand active characters.

Over the last couple of weeks our lead engineer in charge of the merge team devised a clever new way of reducing the 72-hour downtime down to a more manageable 8 hour downtime window, to fit in an "extended maintenance" period. Rather than merging all characters at once, the team implemented a new system to merge a lot of the preliminary data first, and then merge each character in real-time as players log into to the new world for the first time. This means that you will only need to wait 2 seconds for each of your characters to "arrive" before hopping right into APB again.

While this plan is certainly faster, it makes the upfront development effort significantly more complex, since we now have to consider the edge cases around the fact that characters will essentially be created on the fly on the new server at different times. In essence it became a live "character migration" in real time. The new issues that arise are things like - what does this mean for clans with members arriving at the new server at different times? What about mail that was sent by a character that doesn't yet exist, but might "arrive" at the new server eventually?

After the last few weeks of development we are finally confident we have a good solution, but now we need to spend significantly more time testing all these edge cases.

How long before it goes live?

The merge is something we have been working toward for a long time, and we never like pushing these dates back. So while we can't merge this week simply because we don't yet have all the test data showing us that we have a green light, the new plan is to complete both the EU and US merges the week of 5/18.

Thanks for all your patience. I assure you that it will be worth the wait!

Stay tuned for another post later this week, where I'll be going over the brand new roadmap (including the Engine Update) and unveiling the six new names for the six going forward servers for PC and Console for both EU and US!