The Harbingers randomised HD video (Needs testing)

[BonsaiTreehouse]

Dang, you're really going the extra mile with all of this! AI upscaling, deepfake tracking, complex scene splicing, this is all seriously impressive work!

I'm guessing the ultimate intent with this is to make a kind of randomised SUPER-tape that encompasses almost the whole Atmosfear gatekeeper series? Maybe there could be room in your DVD project for an Express-style short game option too if possible (just while the fan-restoration train is chugging along)?

[ThatWidescreenGuy]

Dang, you're really going the extra mile with all of this! AI upscaling, deepfake tracking, complex scene splicing, this is all seriously impressive work!

Well thank you very much, I'm very passionate. I can't say it's the most impressive stuff I've seen in this community though. You guys really do take this seriously!

I'm guessing the ultimate intent with this is to make a kind of randomised SUPER-tape that encompasses almost the whole Atmosfear gatekeeper series? Maybe there could be room in your DVD project for an Express-style short game option too if possible (just while the fan-restoration train is chugging along)?

Honestly, I was only going to use Mr. Nosul's footage, to keep the Gatekeeper as consistent as possible while still using as much footage as I can. But if a SUPER-tape is something that would be of the community's interest, then sure, I could do that. For now I'm sticking to The Harbingers, with a couple of interventions from the original game as easter eggs. If the AI experiment goes well I think I could have a new version ready by the end of the week.

[ThatWidescreenGuy]

Alright, update on this enormous mistake of a project. Soooo the deepfake took a lot more manual work than I originally expected and, despite the big improvement, it's still not perfect. Here's the staring contest with the appropriate colour correction and the hood edited in:
https://youtu.be/zN8iRaCEMek
As you can see, there is a lot of flickering and the tracking of the face is not perfect. Those are both issues that could be solved manually, but that would require going in frame by frame and that takes a long time. So I'm going to try rendering it again, even though I don't expect it to improve too much since I've already used all the existing Gatekeeper footage and let the AI train for an inordinate amount of time.

Anyway, I've been staring at this man's face for weeks already, so I'd like some input in how it looks as of now. I haven't touched the audio yet, do you think he sounds close enough or should I try to alter his voice to be more faithful to the raspy, old tone he has in The Harbingers?

[BonsaiTreehouse]

Not bad so far! Very impressed that you’ve managed this much!

The gatekeeper’s face is pretty flickery to be sure, though I’d personally try a different face of his for the capture process if only because the current one as is seems not to smile or leer much when he should be smiling and leering in the original tape. I’m not too familiar with how the deep-faking pipeline works so there’s likely more to it than what I suggested, but that’s just my more immediate thoughts right now.

It sounds like a lot of work though so if you are committed to this, be sure to step back for a bit when needs must!

[ThatWidescreenGuy]

I'm not sure what you mean by trying a different face. Basically, what the AI does is take all the faces you input and learn how to make your actor's face in any expression, then detects the expression your destination video has in every frame and copies that, changing the facial structure. Ideally you'd have images of your actor in any possible lighting and with any possible expression, but in the tapes The Gatekeeper is pretty much lit the same way all the time. So basically the look you're seeing is a blend of all the looks he has through the three tapes (which are all very similar anyway). I think the lack of smiles is because it's copying the original performance exactly, and the original Gatekeeper doesn't make many extreme expressions (you can see his eyebrows are still all the way through, for example).

If you mean I should try to transform a different intervention, I actually did this with a bunch of them, I only edited the staring contest to post here since it's the one I'd like to put in the tape as easter egg. But I'd say same thing applies; the original Gatekeeper performance just doesn't have as much of a dynamic face as the Harbingers one.

The other option would be shooting an impersonator lip-syncing to the audio and trying to make the same kind of expressions we see the older Gatekeeper make, and then deepfake the face onto that. However, I'm not sure how ethical that would be, and it takes away the charm of it being the same actor all along. I guess it could make for a better result though, and the performance would still be there in the voice... I don't know.

[marr0w]

This might be completely out of left field but I feel like it might fix some of the problem to overlay something similar to the purple and pink clouds rolling in from the Harbingers tape. For one thing it'd help mask some of the flickering and if you can somehow make it match the timing of the clock (which I assume you plan on adding somehow, someway?) so that it becomes more of a storm as it goes on it'd give everything that extra layer of polish. There's a lot of free storm footage on the Internet Archive that might work. I'm not sure what your technical wheelhouse is though so I understand if it's not something doable. I'm just spitballing.

Personally I see nothing wrong with having an actor mime the lines and then overlay the Gatekeeper over the footage as long as it's done in a respectful way. The performance is largely about the audio, that's why we accepted his visual change between games. Plus it'd be way easier to randomly use lines from Third Dimension. Which, by the way, if you're still having problems finding that audio, attached is the files converted to mp3 and renamed to make it easier to find specific sound clips.

Since the base game is on the Internet Archive too, I don't think anyone's going to care anymore if we distribute these... That said I should consider uploading them to the wiki at some point. That might be a project for later.

[ThatWidescreenGuy]

Okay, first of all, thank you so much for the voice files. I have a copy of the CD but I know nothing about ripping and I wasn't able to find half of them. So many new possibilities! Although we'd have to come up with ways to stitch the lines together in order to make interventions that can be used in the tape. Personally, I'm not a fan of the minigames in the DVD game and I tend to skip them playing The Third dimension, so I wouldn't include those in the tape (I don't know what the general consensus on this is).

Second, I do have a clean plate of the clouds used in the tape + timer, it's actually the most important asset I made out of the videos imo, so there would be no problem with including the interventions anywhere in the game. I agree the clouds could help mask the flickering, especially since the staring contest is supposed to take place in the second half of it.

But yeah, with so many audio lines I may have to consider the whole reenacting thing. If someone here thinks they can nail the facial performance, I can supply the deepfake. But maybe before committing to that we could try shooting close-ups of the eyes (so you get the eyebrow movements but you can't really see the face) or replicating the facial look with prosthetic makeup. It would mean staying true to the character design without having to take another person's face. Thoughts?

[BonsaiTreehouse]

If it's just eyes you need, you can always use the first and last few seconds of the main Harbingers tape that's nothing but eyes (the parts where he's saying "on your marks" as well as the "good riddance" ending). None of the facial movements match up with what he's saying on that instance of the tape anyway, so you can probably get away with it here if there's a good quality transfer to be found online.

But if it's the full face you need, I got a little lip-sync "audition" video saved in my Google Drive here if it helps: https://drive.google.com/file/d/1HUEhis ... sp=sharing

[marr0w]

Ooooh, right. This all reminds me that there where video files hidden on the disk too. Some of them have unique audio. I've made gifs of two of the onscreen Gatekeeper images.

GKLOOP1.gif (336.39 KiB) Viewed 21667 times

One of them is just the eyes moving. There are actually three different "eye twitch" videos it plays with random audio playing over them to give the impression of him speaking directly to the player.

G22GATE.gif (904.93 KiB) Viewed 21667 times

"Welcome... to another state of... mind... Welcome to the OTHER SIDE!"

The others are him actually speaking but I don't recall these actually loading in the game itself. These and the audio files are hidden in the folders on the disk image itself inside the AV1-AV3 and W1-W11 folders (for those wanting to see what else there is). I'll attach the actual files I mentioned.

Make sure you take a listen to INTRO and ENDVID, these are actually the logos with audio overtop them. They might be good for an extra start or stop. I honestly don't know if I ever got around to ripping the audio for any of these files. If I've simply forgotten they should be in the pack I uploaded the other day. If not they probably need to be. I hope this helps somehow.

Side note, I'd love a copy of just the storm clouds if you've got that footage isolated somehow. Those would make an excellent screen saver or animated desktop wallpaper.

As for the question about the mini-games, I've never been a huge fan of them either so I'm all for leaving them out. There's nothing like everyone having to stop playing a game to do some arbitrary little distraction that just eats time for little reward and leaves most of the people playing sitting on their hands.

[ThatWidescreenGuy]

But if it's the full face you need, I got a little lip-sync "audition" video saved in my Google Drive here if it helps: https://drive.google.com/file/d/1HUEhis ... sp=sharing

Yeah, I think we could do something like that, I feel like it's better than only using the eyes, especially if I have to recycle the same clip over and over. If you give me a bunch of footage of yourself making the crazy faces I could use it to train the AI on your face and get the performance for The Man. But, as I said, I feel like shooting it with prosthetic makeup instead of using direct footage of his face may not only give better results, but also be a bit more ethical, since it's a different actor's performance.