Replay videos

Regarding the Bastogne “replay” videos, Madrebel asked:

ah so you’ll never likely allow access to the game DB. what about a slave sql db with a read socket open to something like wiretap?

We’re not likely going to provide external access to the data, but we have – of course – been throwing ideas around for how we might make it useful in-game. I think it would be really bad juju if I picked any particular idea as an example because right now it is just a debug tool.

One of the reasons for making the Bastogne videos was “to see what we can see”. It actually kicked off an interesting discussion point when I coaxed Rafter into putting it on the big screen at the close of an unrelated meeting last week.

What we all noticed was that … the videos weren’t very interesting. Having four completely independent views was 4x as dull.

Gnpatton was a good example – all it did was leave him with more questions (I might actually make ya a little video, gnp ;) Drave asked if Ramp got a kill and why I didn’t show the guy. We asked the same question :)

You also have to watch really intently to tell that the videos are actually related. There are a few points where you will see explosions going off in sync. If you watch the really-hi-res version, and pay really close attention to the tags you might be able to see further evidence of synchronization but … geez, man.

For something like this to produce interesting videos, it’s going to need a director, something that can identify points of interest ahead of time. Sounds to me like a heck of a fun challenge…  What sucks is its exactly the kind of thing I’m sure several of our community would love to be involved in :(

The way the replay system works is actually kinda simple and kinda neat. The “cell” hosts (the processes that deal with ‘avatar’ data, that is – your movement through the 3d world) have to share data between each other. They spit this data out on a network backbone and the remote cells use it to present your movements etc to other players so that the distribution is transparent.

For recording, I just stuff these packets into a database with some timestamp and other basic information.

Playback is a matter of reloading these and popping them back onto the network like we were the originator (in some cases, I have to modify the data — e.g. rename players or change some of their “session” data so as not to conflict with currently-active players, but that’s by the by).

At the moment, I have both the recording and playback code inside the standard “cell host” process, controlled by chat/irc commands. But I could just as easily relocate it to a separate process and run it on a dedicated box and perform on-the-fly analysis.

There’s a second alternative though. Part of this tool is a little external app, called “playback”, which extracts the raw data from the database and allows me to inspect it (uhm, feature creep, it also provides mechanisms for converting recordings between incompatible formats if the protocol changes).

Storing the data in a database is just ever-so-slightly stupid (oh, the indexes)! The recording tool can also dump data directly to a file (producing about a 20-40% reduction in storage requirements).

A more practical approach to leveraging these capabilities in any kind of user-friendly format is likely going to be continuous to-disk recording. I’m undecided right now as to whether I should have one process snarfing all the data off the wire and recording it directly to a single file, or if I should have each cell quietly use a background thread to write its data stream to a separate file and recombine the files later on.

If I rotate these files every 15 minutes or so, there only reason not to keep continually recording is … disk space. An average 6 week campaign is going to need around 1Tb of disk storage.

That’s a wee bit much. But, we can reduce it significantly if we come back to our “director” concept.

The recording contains everything everyone is doing. “dinker” flying out to the alps and “ouwd” gone AFK in his Daimler at Calais while the fight has moved to Luxembourg. Those guys just sitting there is contributing (albeit minutely) to the data stream.

There are currently three types of data record in the stream: “Intro”, “Update” and “Outro” (well, actually they’re “1stTime”, “Update” and “Removal” but you don’t want to know that do you? ;-p)

So I probably need to add a “POI” record which specifies some place someone wants observed. That someone could be me setting a 3d/visual “break point”, a GM wanting something looked at by a dev. Engage warm-fuzzy-future-ideas-mode: It could be a player-annotated event to allow us to provide players/squads with a “greatest moments” tool.

But it doesn’t have to be a human-generated item. It could easily be the system wanting to preserve a record of the activities of a player it kicked & banned.

Or … it could be injected by the post-processing tool…

That leaves the big question: How do I decide what’s worth watching. By ‘watching’ I mean ‘spectating’, say for a video – not as a GM or dev.

I think I’m gonna start with a heat map of some kind; starting with something like the “deathmaps” grid on WireTap: lots of deaths = start looking here.

From there it’s a matter of coming up with good weights for various kinds of activity and back-tracking the components of that action.

It’s probably going to make for a very long-term project. Already my mind is itching to go off thinking about things like “well, this vehicle takes tons and tons of hits – so he might be worth watching, but wait, he gets finished by a bomber, so we need to wind back and track the incoming bomber so that a smart filming-client knows to follow the bomb that hits him” etc.

I really wish there was a way I could open this up so that the small few of you who are salivating at the prospect of fiddling with something like this could participate…

18 Comments

From the players’ perspective, I think the most interesting things are generally kills. Yours, and your opponents. An interesting automated “cinema view” could be as simple as this:

– randomly pick a player who’s about to get a kill. Now either:

From the players’ perspective, I think the most interesting things are generally kills. Yours, and your opponents. An interesting automated “cinema view” could be as simple as this:

– randomly pick a player who’s about to get a kill. Now either:
– show a 3rd person over the shoulder of that player, looking in the direction of the player they’re about to kill
– or a 3rd person OTS of their victim, looking back at the killer (think of the typical “target-lock” view in single person flight sims)
– stay with that player until they (a) die or (b) nothing interesting has happened
– pick another player who’s about to get a kill

Notice how I’m advocating some kind of prediction of the future – pick a player who’s about to get a kill? WTF? But this system isn’t real time, it can search the event stream for a kill, identify the killer and victim, and then work backwards to an appropriate start time and display it. The search process can be asynchronous, finding “interesting stuff” and handing that as a queue to a separate (delayed) realtime display system which presents a coherent narrative. And it “feels” psychic while you’re watching it. :)

Exactly – the director process would be post-processing data. My thinking is that it would buffer several minutes of action (say 5-10 minutes) for which it would produce action heat maps to get a general sense of where is going to be interesting for general scrutiny (there’s very little point in analyzing the activities of “bob” out in the wilderness).

But you’re also going to want to keep track of events that make individual players a hot target.

– Dying (while moving) on the way into a capture building (1 point),
– Getting hit by multiple players (1 point),
– Entering an occupied enemy capture building (2 points),

etc

You’d want weights that would pick up a player getting a “lucky streak”, i.e. getting a rapid succession of kills. But you’d also want to adjust for weapons. Dude with a pistol getting 6 kills = lets see that. Two dudes shooting at each other with pistols until one dies = definitely see that.

Once I find something of interest above certain thresholds, retrofit in POI markers. Add in a buzzard-cam capability to receive POI markers and scoot to them (so they need to be early) and voila :)

Hmm.

Some of the most interesting things that happen out there are actually things that don’t happen. You get a nice build-up of tension as something decisive looks like it’ll develop, and then it develops otherwise into a non-event that spells a town being capped.

For interesting viewing, you need context. You need a sequence of things that tell a little story within the larger battle. And if you can get it, you need the larger view of how (if) that little story actually had an impact on the larger battle.

I’m not so interested in 2 dudes shooting one another with pistols until one dies, unless the guy who wins, say…then goes on to singlehandedly destroy the enemy line of ATGs who’ve been bottling up the town, and that guy he killed with the pistol was the one guy who could have stopped him.

Just being able to show one guy who turns out to be really good at circle-straffing as he zaps one guy after another in what’s ultimately a meaningless series of kills while the town’s being capped out from underneath “super soldier Bob”: not so entertaining.

Then again, maybe I’m jaded :P

I’ve got way more time in the buzzard than anyone else…and I’ve learned what makes for interesting viewing.

Even with that, it’s hard to find those moments. What you’re talking about…I’m not sure how you post-process that without a human to look at it and say “THERE’S a tale worth telling”.

Maybe your idea of what a tale like that is, is different than mine.

Maybe some post-event voting system…where interested players who want to find this stuff do a mini-aar…like…score thier most recent sortie on a scale of 1-10 for noteworthiness, and you take that into account in your picking scheme.

Maybe I’m in the minority here, but I’m opposed to any kind of replay. The “wtf just killed me” is an important aspect of the realism to me. Maybe that’s not so much of an issue for those who play inf who are in close-quarters combat, but for tankers and ATGs it would be a big deal to immediately reveal the position and identity of the killer. When you sneak into a nice flanking position undetected, the last thing you want is your position broadcast to the world with the first shot.

As a debug tool, and to investigate those cases when people claim to have been killed by someone who had x-ray vision, great. Save all the data for that, a terabyte is like $100 of disk. But please do NOT open these data up to everyone, unless it’s from the training server.

Love the ideas here. <3 if you make me a video.

Since were on the topic, I'll chime in with what the CoX system did. A player could type a console command and their client would start recording the packets they received in a file. To replay the video, you loaded the client in replay mode and fed it the file. There were commands to manage the playback (stop, play, rewind etc).

Most people who made videos ended up getting really into it. Often times they would modify the packet file to get the results they wanted. In a sense they were the director.

I could see that kind of system being very useful for anyone who wants to make 'movie movies' instead of AARs. Or it could even be a dramatized AAR. ;)

Back in the early days when the boxed game was available in Apple Stores, I was dismayed every time I was in my area’s store that, with the big displays (for the time) constantly showing demo content, there wasn’t a way to have WWIIOL on one of them…either real time, or a long looping highlights reel.

Mr. Jobs might not be much of a games supporter, but–at least in that one store–there was plenty of support at the local-staff level. They said they got a lot of inquiries about games, and of course they were highly interested in whatever software would get people to buy hardware.

I continue to think that a significant number of customer-candidates don’t realize that they’d enjoy a realistically immersive game like WWIIOL. They’d have to see it on a big screen to have that epiphany.

Lutorm, there is a simple solution to that issue, you just delay availablilty to it by a few hours. Problem solved.

Yeah, I’m not talking about a death-cam type view. For a start – it’s going to take a while for the data to become available (see discussion above).

As for making them available on the training server? Pfft. I think an instance would be way better :)

zomg, i got the full quote treatment, nice.

lutorm, from what i gather in all this is that these recordings may never (likely not) be ‘publicly accessible’. the reason i asked the question that kfs quoted was because it reminded me (his collection process) a bit of how old quake engine/source engine movie are recorded. in theory its the same thing the difference is those old engines aren’t as complex so when they dump their ‘cell’ data its basically dumping to and being replayed from your own private server so to speak.

what i was thinking was from an ‘after action’ point of view where say me and 3 wingmen fought at roughly X time in area Y and we had an amazing fight if i could access a DB of some sort and suck down that data then replay in my client in offline mode i could then capture with fraps and use thigns like chase cams and external views to make a really clean cool video.

what i think kfs is saying is that wouldnt be feasible in the short term (if ever). so my suggestion now to kfs and the rats is to look at using this tool for marketing. i can’t count the number of times i’ve read negative posts about BGE where “omg nobody even plays that shit game anymore” “omg that game is so boring nothing ever happens, you walk 10 miles and die instantly” … etc … etc.

so, make some really cool movies. like the vid kfs posted that start this discussion. make some videos showing huge battles. the bastogne video had to have 120+ people in that fight at least. so make some vids, show people what a big fight looks like.

my $.02

The problem with that approach, mad, is that it would be limited to what was in your vis range.

Now if, on the other hand, the host was saving off everything all the time in X minute chunks, and you had the option of being able to request a save of a particular section of data, the host could use that to determine which saves to maintain.

Then – why not kick back and review from your own private/squad “cine-instance”? :)

I’m sure Killer has some ideas to toss at you. Replay is one of those “always wanted to have it, never had anyone to make it work” things.

ok, i think i get what i was missing or more precisely why you say i would be locked to only what i was seeing. so there would need to be an ‘in cockpit ride along’ camera thing in order to view from actual player PoVs right? otherwise youre restricted to using buzzard cam for fre range of motion.

i think … it was idk gophur or maybe doc that spoke about wanting some sort of feature for training players and such. so, without this ability in these recordings you’re always limited to the buzzard view right? because that is the only thing with free range action? or if i were viewing i would be locked into what i am seeing, right?

if you had this ‘ride along’ camera and then allowed for in game trainers to idk flag themselves to allow ride along camera attachment or for offline where you could bounce around to any recorded PoV and then use chase or external views then you could really make this thing sing.

also, since this stuff is already being dumped to a DB isn’t it theoretically possible to query for idk “planes in grids E4-E8 thru F4-F8 at time 1200-1300 gmt on 4/7/2010” and then dump that data to a text file, compress it, and then decompress and ‘play’ it as cell data in an offline client? i’m guessing some tools would need to be written to allow an offline client to handle that data but it would be pretty damn cool :).

or do it online. create some VM cell hosts off the production cluster that present themselves in the launcher app as “viewing servers” after you pay idk 2-5 bucks for 1-2 hours. you could probably use a webapp that does the query work and gathers the data set and allows the paid client to login at a certain scheduled time or something.

just spit balling :)

It wouldn’t be an “in cockpit” view, I’m just saying that if you captured the stream at the client, you would be recording a you-centric view. The server variant means everything that was happening on the server at the time is recorded, so you are free to move around the world and see anything that was happening from any perspective you like during the recording :) It’s replay to the nth degree :)

The cell-to-cell data is significantly different to the stream we send down to the client, so you could pack it and send it down to the client, but given that we already have a mechanism for doing that (i.e. the cell host :) it makes more sense to stream it :)

So rather than recording everything that passes through the server and being limited to what you can make actually happen, why not build a script editor, and use a client that plays a scripted stream?

We know that movie making as done with live player-actors is difficult, because the in-game actors are working in a difficult environment with no visible marks, spatial distortion, etc.

So why not go to movie making as the animators are beginning to do it, with scripted characters?

a) That would only be useful as a video making tool,
b) You clearly haven’t thought that through.

Trackbacks and Pingbacks

Oh, gnpatton :) « kfsone's pittanceJuly 6, 2010 at 6:32 am

[…] Replay videos […]

Leave a Reply

Name and email address are required. Your email address will not be published.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

You may use these HTML tags and attributes:

<a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <pre> <q cite=""> <s> <strike> <strong> 

%d bloggers like this: