Don’t show me the money

This started out as a quick summary before I left the office, and then I’ve written more and more as I was dabbling with the odd bit of code here, looking at logs there, and hanging out on Befallen with Toto and Ramp.

Rick’s bouncing grenade STO wants for the originating player not to see the STO, something that STOs hadn’t done yet. Sounded easy enough, but on closer inspection I realized that I’d very heavily optimized the generation of data for STO announcement, which are frequently being sent to hundreds of players.

Typically teulkit requires you to marshall the outgoing data for each destination you are sending to. This isn’t an oversight but it provides a very efficient code path for delivering packets which have to be tailored for each destination. Its very much less efficient when you are sending a single packet to multiple destinations.

Last year I added a small interface class, TeulBroadcast, which uses hooks into the standard packet sending system. I looked at the send routines and snipped the primary function in half, so that marshalling and transmission are (transparently) now separate functions.

When you instantiate a TeulBroadcast instance, it invokes the standard teulKit marshalling process, and then you call send with an array or STL iterator-pair of destinations. The TeulBroadcast::send function then repeatedly calls the second half of the teulSend process with the prepped data.

STOs announcements (transmission to players in the cells around where an STO has been placed, vs introductions which are transmissions to players who have just moved into range of a pre-existing STO) are TeulBroadcasted to all the players in the cell-group.

Which means the STO creator too.

In the end I extended the entire STO architecture to support this and other objects that might need similar functionality, basically by tracking the network endpoint that triggered the STO creation.

That gave me two options: a filter function to selectively skip the endpoint that matches the originator of the STO during the TeulBroadcast::send iteration across the endpoint list; or I could special case Ordnance STOs before calling send and remove the endpoint handle from the list if it was in it.

Not wanting to loosen the send loop, I went with the latter. Then I tried to be clever and do the remove trick which basically replaces the entries you are removing with entries from the end of the vector. Since I’m removing a single endpoint address, it would should simply move the last entry in the vector (if there are 2 or more entries) into the vacated slot.

Sadly I didn’t remember the other half of the remove trick; or rather I misremembered it. I know that remove is one of the worst functions in the STL because it doesn’t do you’d expect it to, but I forgot how generous a statement that is.

If you have a vector with the integers (1, 2, 3, 4, 5) and you remove(vec.begin(), vec.end(), 3) then your vector will now contain (1, 2, 5, 4, 5). It “removed” the value of 3, by moving entries from the end of the array to fill the space. 3 is no-longer in the vector, but it wasn’t “removed”. It was substituted. The vector is still has 5 entries.

You still have to call erase to finish the job. I’d only remembered one part of the trick. The real meat of the trick is that remove will allow you to limit erase to the end of the vector which means that it can just reduce the entry count without actually adjusting the memory allocation/doing any memory operations.

erase(remove(vec.begin(), vec.end(), 3)) ;

But, we now have bouncing grenades, 1st and 3rd person. Which is really unnerving. My first grenade bounced back at me. I hadn’t given “bouncing” itself any actual thought, so it went off before I understood what I’d just seen :) Rick’s still tweaking them I believe, but they’re looking very good. It’s going to be highly disconcerting for a while. “Oh, that didn’t go where I wanted. Oh, wait! I can still run away. Doh, too late”.

Tested out the netcode2 between cell hosts which had a couple of problems; there’s no way with teulKit to tell when an IP address is in host or network order, so some of the functions kept complaining about no corresponding connection until I realized the value I was being given was in the wrong order. But with that solved, as it has done persistently, netcode2 just worked and worked superbly.

NC2 differs from teulKit in a number of ways: its threaded, so the network activity can be offloaded from the primary cpu/core; it uses UDP for transport with its own reliability layer so it uses slightly less packet overhead; it aggregates packet-payloads (including retransmits) into single network packets where possible; it can deal with fragmentation. (And a bunch of stuff I can’t discuss)

That all adds up to a much better suited communications stack for gaming than tcp-based netcode1 can ever be: Of the last 5000 Netcode2 connections to the live server, 2% of them have disconnected without closing normally – i.e. that includes alt-f4s and crashes along with CTHLs, of which 50% (i.e. 1% of the connections) are 3 guys, two of whom I know have been crashing a lot.

Yes, 2% or even 1% is unacceptably high. But we’re still testing netcode2 and its only being used in a limited fashion, and we’re deliberately testing it with people we know have major problems playing the game with netcode1.

And since netcode2 runs in parallel with netcode1 I could probably add some trickery such as if no packets flow over nc2 for 10-15s, sending a packet via nc1 and seeing if the nc2 pipe has somehow become stuck (so far a large number of our network issues appear to arise from people who can’t cope with various sizes of packets over 576 bytes).

That leaves me with only one minor task I’d like to get in for nc2 this year; its a tiny step but that may be a small step backwards efficiencywise, but with it done it opens the door to starting to use nc2 for some of the update system changes we’ve long dreamed of – which is segregating the update information between stateful and progressive.

The progressive information (like position) can then afford to be “lossy” – which can be a huge boon in congested situations (players with low bandwidth or during packet loss recovery). Right now any packets you drop are going to come back to haunt you and further choke your connection.