Parallel sort

My next self-challenge for parallelism with ZeroMQ is going to have to be putting together a parallel sort.

Since you’re here, Laccy

(From passiveaggressivenotes.com)

That can never happen.

How often have you been playing a game and come across something “weird” that clearly should never happen: like finding a spot where the AI can’t “see” you while you wail on them mercilessly, or in the process of button mashing you catch a trigger that kills the boss who had you down to 1 hp without losing any of his own…

You scratch your head and ask: Why didn’t they think of that?

Chances are … someone did :) That someone just had reason to figure it would never happen.

Someone once told me that programming is like writing out a series of steps for a robot: Go to kitchen, pick up coffee cup, go to coffee machine, pick up coffee pot, pour coffee into cup, take coffee to me.

Sadly, programming isn’t chirpy fun like that. Infact, you have to be a bit of a pessimist.

Instead…

I wound up distracting myself even more from my already sidelined weekend plan.

I keep harping on about ZeroMQ but I don’t think I’ve yet managed to really convey why it’s so worth programmers looking at…

The problem is, when you look at ZMQ for the first time, it just looks “meh, sockets”. But that is just the API. ZeroMQ isn’t primarily a networking library! It does a whole lot of stuff, across multiple languages, all through that one single API. So all you have to do to change the scope of your application is change the URL of the socket you plan to connect to…

StarCraft 2 :)

I’m not passing judgement on SC2 itself here, rather … I’m just wondering what your average WoW player is going to think when they buy SC2 after watching this

to find that they get this

Well, not quite.

My weekend plan fell apart fairly quickly :)

After updating SVN repositories for all 4 branches involved under Windows and my Ubuntu virtual machine, first task was to merge code. Don’t ask why, but I somehow stumbled upon a line of code that literally blew me away: for every incoming game connection, a 64Kb block of memory is allocated. The only other reference to this memory was in the disconnect code, which freed it.

Ok, well that’s 76Mb of RAM saved per server.

Now … wtf was I doing before this distracted me?

Plan for the weekend

I checked a rewrite of the cell-host grid system into SVN a few weeks back. I’m going to make a branch of the current baseline and re-integrate the new grid system code into that, instrument it up with unit tests, and see if I can’t have it fully working by tonight, then run it through it’s paces tomorrow.

Or … maybe I’ll spend a bunch of time today reviewing recent ideas about neighbor search systems first.

Part of the new grid system is slightly better optimized vis-list building on the host. More importantly, the new mechanism is encapsulated in an entirely thread-safe way, so I can potentially parallelize it, which means I can be a little more methodical inside the processing routines.

The cell hosts themselves are dual core-2 xeons (4 physical cpu cores), but we get very little work out of more than 1 core at a time :(

i am bed sorry

The peril of drunk gaming/foruming leads to a new meme :)

ZeroMQ and scalability

The elegance of ZeroMQ messaging is that it provides easy scalability. The API for 0MQ sockets is the same whether you are doing in-process (inter thread) communication, inter-process communication, peer-to-peer network communication or multicast communication.

As long as you are putting data and not pointers in your messages (*cough* Async::Worker *cough*), you convert your code from in-process communication to cross-network with a one line change:


// Create the socket: same code in either case.
zmq::socket_t outSocket(zmqContext, ZMQ_REP) ;

...

outSocket.connect("in-proc://my-connection") ;
// becomes
outSocket.connect("tcp://192.168.0.65:2342") ;

and voila – your application is networking instead of talking to itself.

They also provide three external utilities that make it a doddle to scale your application across multiple machines.

In Defense of Nettersheim, Finale

The finale to the Nettersheim series, hope you enjoy :)