<turbo-stream action="append" target="posts_list"><template><turbo-frame class="main-list__list-item" data-testid="Post" id="post_86799">
    <div class="post" access="public">
  <div class="post__inner">
      <div class="post__media">
        <div class="media-player media-player--video">
            <div
  class="embed-player"
  data-controller="youtube-player"
  data-youtube-player-watch-times-path-value="https://ohheyvoid.com/supporters/api/v1/media_catalog/posts/video_embeds/86799/watch_times"
  data-youtube-player-video-id-value="ZA9hWBh2-2k"
>
  <div class="media-player__cover" data-youtube-player-target="element">
    <img src="https://storage.googleapis.com/popshopprod-membership-assets-single-b5px4371/2cq6y166m1ii80rfnpmy6s88tnxs" class="media-player__cover-image media-player__cover-image--cover" loading="lazy" />
    <button type="button" class="media-player__cover-button" data-action="click->youtube-player#createPlayer" data-testid="YoutubePlayer.PlayButton">
      <svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" viewBox="0 0 32 32" fill="none" role="img"><path d="M28.422 14.211c1.474.737 1.474 2.84 0 3.578L2.894 30.553A2 2 0 0 1 0 28.763V3.237a2 2 0 0 1 2.894-1.789l25.528 12.764Z" fill="currentColor"></path></svg>

    </button>
  </div>
</div>

        </div>
      </div>

    <div class="post__main">
  <div class="post__content">
        <a data-turbo-frame="_top" class="post__meta" href="/supporters/video_embeds/86799">
          Jan 28, 2024
</a>

      <div>
          <a data-turbo-frame="_top" class="post__title" href="/supporters/video_embeds/86799">
            Rokoko Challenge! Top 100
</a>      </div>

      

        <div
          class="post__body"
            data-controller="trim"
            data-trim-class-value="rich-text--trimmed-short"
            data-trim-height-value="220"
        >
          <div class="rich-text" data-trim-target="content">
            <body>
<p><strong>SCENE 26</strong></p>
<p>This scene is the "Hive Bee" scene where the audience is introduced to a surveillance drone. It's right in the middle of our team being chased by a horde of Zombies. We wanted it to feel like a "we're pinched!" moment. </p>
<p></p>
<p><strong>YUKI &amp; THE HIVE BEE</strong></p>
<p>Yuki is feeling a lot of things in this scene. Moments before, she had taken out a zombie that landed on some trash cans. This noise alerts the horde the team was trying to avoid. So there is definitely some guilt. Now add the threat of being captured by Hive Bees?</p>
<p><em>The Terror! </em></p>
<p>This is the first scene in the pilot where we introduce the Hive Bees. They're The Sanctuary District's surveillance and recovery droids. They patrol an area of Nuvitta called Salt Town. And it's implied that they "clean out" areas of the city by removing survivors. </p>
<p>The initial idea was that these Bees were new or retired tech that our team of five hadn't encountered before the X80 outbreak. All they know is that these Mech Bees are picking up people and taking them to the walled-off area they call "The Hive."</p>
<p>A place where no one is ever seen again. <em>Dun-dun-duh!</em></p>
<p>We'll learn in later episodes that those outside of the Sanctuary District have created myth and legend around what's behind the 100-foot wall and the droids that kidnap survivors.</p>
<p>When we were naming the Hive Bees, it was important that it feel like something a pre-teen would call them. In our heads, "Bees!" is definitely a term the five survivors use amongst themselves whenever they are on prevision runs. </p>
<p><strong>ROKOKO CHALLENGE</strong></p>
<p>This is only a partial moment in the scene as we were submitting to the Rokoko Intergalactic Challenge. A 7-sec video moment meant to get people to stop channel surfing through an intergalactic TV-feed. </p>
<p>We were so excited when we placed in the <a href="https://youtu.be/Q1IYuisYt8A?si=n1FJYQGmZxi1yn9O&amp;t=3357" target="_blank" rel="noopener">Top 100 of submissions</a>. It was really cool seeing how the judges reacted to the scene, especially because it was our first test run with Rokoko and the UE5 workflow. </p>
<p><strong>ANIMATING FOR THE CHALLENGE</strong></p>
<p>When we decided to enter the Rokoko Intergalactic Challenge, we knew we wanted to do something with Zomborgs. But what scene to choose from? </p>
<p>Why... let’s pick one of the hardest scenes we could possibly choose for our first mocap-to-UE5 scene. That's a good idea, right? </p>
<p>In the end... yes. It was a brilliant idea.</p>
<p>Jayson had just finished his enrollment with Josh Toonen at <a href="https://www.unrealforvfx.com/" target="_blank" rel="noopener">UnrealForVFX.com</a>. And the knowledge he brought to the project was so very helpful. The community in the course was so helpful when it came to feedback on this one too.</p>
<p>By forcing ourselves into one of the hardest scenes, it pushed us to learn things fast. </p>
<p><strong>MOCAP</strong></p>
<p>This was our first go using Metahuman Animator and the Rokoko Suit, so we were very excited to explore bringing Yuki to life. Amber mo-capped Yuki's performance, and we were pleasantly surprised how simple the data capture was. Even the setup was pretty straightforward once we got it going.</p>
<p>Now, yes, there were some connectivity issues - but that's nothing new to us. Tech fails. Then it works. Gotta flow with it, ya know? The only gripe we had though was our glove had a malfunction where the middle finger just wiggled out of control. It required having to send the glove back to rokoko for repair. Not awesome. But we expect a return soon. </p>
<p>Despite the glove malfunction we were able to capture mocap for about 20 characters in the same day - which was... amazing. The cleanup was pretty minimal on this once we learned the power of control rigs and the use of cameras to hide the ugly stuff.</p>
<p><strong>CAMERA &amp; EDITING</strong></p>
<p>Speaking of cameras - the experience of being able to go in and fine-tune shots and angles long after the action is shot was... so very exciting. </p>
<p>We come from a background of traditional pre-pro, production, post. The animation workflow is similar, but man does it change the production process. </p>
<p>Coming from the post pipeline there have been sooo many projects we've edited where we said, "Man, I wish this shot was just a little to the left!" Or "Where's your 180 rule, people?" Being able to go in and adjust the shot was something we were both so excited about. </p>
<p>It's pretty awesome that production IS the editing process with this workflow. It feels like a superpower.</p>
<p><strong>CHARACTER DESIGN</strong></p>
<p>We used Metahumans for all of our Zomborgs characters and have really enjoyed the process of using Blender and UE5's Mesh to Metahuman workflow. It helped us and our limited character design skills get closer to what we had previs’d using Midjourney and Stable Diffusion. </p>
<p>In this scene, we really only had one character that had hair that wasn't "Metahuman Hair." Because let’s be honest, hair is one of the first giveaways when watching stuff. That and their teeth... but that's a challenge yet to be tackled.</p>
<p>Thankfully, Prompt Muse came to the rescue with her <a href="https://www.youtube.com/watch?v=l0PiYCDmjBk&amp;t=1086s&amp;pp=ygUacHJvbXB0IG11c2UgbWV0YWh1bWFuIGhhaXI%3D" target="_blank" rel="noopener">Hair Tutorial</a> the same day Amber went to buckle in and learn it. Who doesn't love a serendipitous moment on a journey?</p>
<p>It was our first attempt at custom hair, definitely not the best, but man, for this video it felt like magic. </p>
<p><strong>IT'S AI-FREE</strong></p>
<p>So for this one, we actually went the route of no AI. We used the UEforVFX post-processing workflow to get a pretty fun look. </p>
<p><strong>MOVING FORWARD</strong></p>
<p>Moving forward, we're going to be putting in the reps. We really want to get more practice in the engine, so we'll probably pick out a few scenes to tackle from the *Survivors'* screenplay.</p>
<p>At some point we'll tackle hair and wardrobe issues again. But right now we just want to get better at the scene building. </p>
<p>Let us know in the comments what scenes you want to see previs for! </p>
</body>
          </div>
          <button class="text-button text-button--pale post__action-button hidden" data-action="click-&gt;trim#expand" data-trim-target="button">
    ...Continue reading
</button>
        </div>

      

        <div class="post__section">
          <div class="post-actions">
            <form class="post-actions__item-form" data-turbo="false" action="/supporters/sign_up" accept-charset="UTF-8" method="get">
  <button class="text-button text-button--small text-button--pale" aria-label="Become a member">
    
    <div class="post-actions__item">
      <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="none" viewBox="0 0 16 16" role="img" class="post-actions__icon"><path fill="currentColor" fill-rule="evenodd" d="m2.662 7.721 5.14 5.918a.25.25 0 0 0 .378 0l5.142-5.92c1.856-2.21 1.25-4.386.03-5.37-.62-.5-1.407-.711-2.203-.513-.796.197-1.712.833-2.504 2.243a.75.75 0 0 1-1.308-.001c-.794-1.416-1.708-2.054-2.5-2.253-.79-.2-1.573.01-2.19.51-1.214.983-1.822 3.167.015 5.386Zm5.33-5.375C7.172 1.274 6.212.623 5.202.37c-1.292-.325-2.552.032-3.5.8-1.913 1.55-2.524 4.702-.19 7.515l.012.013 5.146 5.925a1.75 1.75 0 0 0 2.642 0l5.146-5.925.008-.009c2.362-2.805 1.75-5.956-.171-7.507-.95-.766-2.213-1.124-3.508-.802-1.01.25-1.974.898-2.795 1.966Z" clip-rule="evenodd"></path></svg>

          <span class="post-actions__item-number">
            3
          </span>
    </div>

</button></form>
              <form class="post-actions__item-form" data-turbo="false" action="/supporters/sign_up" accept-charset="UTF-8" method="get">
    <button class="text-button text-button--small text-button--pale" aria-label="Become a member">
    
      <div class="post-actions__item">
        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="none" viewBox="0 0 16 16" role="img" class="post-actions__icon"><path fill="currentColor" fill-rule="evenodd" d="M1.75 2.25a.25.25 0 0 0-.25.25v8.067c0 .139.112.25.25.25H3c.967 0 1.75.784 1.75 1.75v1.21c0 .216.255.33.416.187l3.053-2.706a1.75 1.75 0 0 1 1.16-.44h4.871a.25.25 0 0 0 .25-.25V2.5a.25.25 0 0 0-.25-.25H1.75ZM0 2.5C0 1.534.784.75 1.75.75h12.5c.966 0 1.75.784 1.75 1.75v8.067a1.75 1.75 0 0 1-1.75 1.75H9.38a.25.25 0 0 0-.166.063L6.16 15.087c-1.13 1-2.911.199-2.911-1.31v-1.21a.25.25 0 0 0-.25-.25H1.75A1.75 1.75 0 0 1 0 10.567V2.5Z" clip-rule="evenodd"></path></svg>

        <span class="post-actions__item-number"></span>
      </div>

</button></form>
            
<div class="dropdown" data-controller="dropdown link-share" data-dropdown-placement-value="bottom-start" data-action="link-share:unavailable-&gt;dropdown#toggle" data-link-share-url-value="https://ohheyvoid.com/supporters/video_embeds/86799?utm_medium=copy-share-link&amp;utm_source=share-link&amp;utm_campaign=post-share-supporter">
      <div class="comment__menu" data-dropdown-target="button" data-action="click->link-share#share">
      <div class="post-actions__item">
        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="none" viewBox="0 0 16 16" role="img" class="post-actions__icon"><path fill="currentColor" fill-rule="evenodd" d="M6.996.471a1.41 1.41 0 0 1 2.008 0l4.943 5.013-1.068 1.053L8.75 2.35v9.121h-1.5V2.35L3.12 6.537 2.054 5.484 6.996.471ZM1.5 11.108v3.143c0 .138.111.249.249.249H14.25c.138 0 .249-.11.249-.25v-3.142H16v3.143c0 .965-.781 1.749-1.749 1.749H1.75A1.748 1.748 0 0 1 0 14.25v-3.142h1.5Z" clip-rule="evenodd"></path></svg>

        <span class="post-actions__item-number hidden@sm">Share</span>
      </div>
    </div>


  <div class="dropdown__menu hidden" data-dropdown-target="items">
    <div class="dropdown__items">
        <div class="dropdown__title">Share this post</div>

      

  <button class="dropdown__item" data-action="click-&gt;dropdown#hide" data-controller="clipboard" data-clipboard-text="https://ohheyvoid.com/supporters/video_embeds/86799?utm_medium=copy-share-link&amp;utm_source=share-link&amp;utm_campaign=post-share-supporter" type="button">
    <div class="dropdown__item-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="none" viewBox="0 0 16 16" role="img"><path fill="currentColor" fill-rule="evenodd" d="M12.145 1.5a1.762 1.762 0 0 0-1.246.516L8.234 4.681l-1.06-1.06L9.837.955a3.264 3.264 0 0 1 4.615 0l.591.591a3.264 3.264 0 0 1 0 4.613l-3.849 3.85a3.262 3.262 0 0 1-4.614 0l-.593-.592 1.062-1.06.591.592a1.763 1.763 0 0 0 2.493 0l3.85-3.85a1.762 1.762 0 0 0 0-2.492l-.592-.591a1.764 1.764 0 0 0-1.247-.517ZM7.112 6.534c-.468 0-.916.186-1.247.516L2.016 10.9a1.762 1.762 0 0 0 0 2.492m0 0 .592.592a1.764 1.764 0 0 0 2.493 0l2.665-2.665 1.06 1.06-2.664 2.666a3.264 3.264 0 0 1-4.615 0l-.592-.592a3.263 3.263 0 0 1 0-4.614l3.85-3.85a3.264 3.264 0 0 1 4.614 0l.592.593-1.06 1.06-.592-.592c-.331-.33-.78-.516-1.247-.516" clip-rule="evenodd"></path></svg>

    </div>

  
    Copy link

</button>
  <a class="dropdown__item" data-action="click-&gt;dropdown#hide" href="https://twitter.com/intent/tweet?url=https%3A%2F%2Fohheyvoid.com%2Fsupporters%2Fvideo_embeds%2F86799%3Futm_medium%3Dcopy-share-link%26utm_source%3Dshare-link%26utm_campaign%3Dpost-share-supporter" target="_blank">
    <div class="dropdown__item-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 32 32" fill="none" role="img"><path d="M18.666 13.857 29.093 2h-2.47l-9.056 10.294L10.338 2H2l10.932 15.567L2 30h2.47l9.557-10.873L21.662 30H30M5.36 3.822h3.795L26.62 28.267h-3.794" fill="currentColor"></path></svg>

    </div>

  
    Share on X

</a>
  <a class="dropdown__item" data-action="click-&gt;dropdown#hide" href="https://facebook.com/sharer.php?u=https%3A%2F%2Fohheyvoid.com%2Fsupporters%2Fvideo_embeds%2F86799%3Futm_medium%3Dcopy-share-link%26utm_source%3Dshare-link%26utm_campaign%3Dpost-share-supporter" target="_blank">
    <div class="dropdown__item-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 14 14" fill="none" role="img"><path d="m5.27 14-.02-6.125H2.625V5.25H5.25V3.5C5.25 1.138 6.713 0 8.82 0c1.009 0 1.876.075 2.129.109v2.468H9.488c-1.146 0-1.368.545-1.368 1.344V5.25h3.255L10.5 7.875H8.12V14H5.27Z" fill="currentColor"></path></svg>

    </div>

  
    Share on Facebook

</a>
    </div>
  </div>
</div>
          </div>

        </div>

      </div>
</div>

  </div>
</div>

</turbo-frame><turbo-frame class="main-list__list-item" data-testid="Post" id="post_86797">
    <div class="post" access="public">
  <div class="post__inner">
      <div class="post__media">
        <div class="media-player media-player--video">
            <div
  class="embed-player"
  data-controller="youtube-player"
  data-youtube-player-watch-times-path-value="https://ohheyvoid.com/supporters/api/v1/media_catalog/posts/video_embeds/86797/watch_times"
  data-youtube-player-video-id-value="GhQD1ZptQ2Q"
>
  <div class="media-player__cover" data-youtube-player-target="element">
    <img src="https://storage.googleapis.com/popshopprod-membership-assets-single-b5px4371/gbwloc41wn9g0ify6250tk1gaj6w" class="media-player__cover-image media-player__cover-image--cover" loading="lazy" />
    <button type="button" class="media-player__cover-button" data-action="click->youtube-player#createPlayer" data-testid="YoutubePlayer.PlayButton">
      <svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" viewBox="0 0 32 32" fill="none" role="img"><path d="M28.422 14.211c1.474.737 1.474 2.84 0 3.578L2.894 30.553A2 2 0 0 1 0 28.763V3.237a2 2 0 0 1 2.894-1.789l25.528 12.764Z" fill="currentColor"></path></svg>

    </button>
  </div>
</div>

        </div>
      </div>

    <div class="post__main">
  <div class="post__content">
        <a data-turbo-frame="_top" class="post__meta" href="/supporters/video_embeds/86797">
          Sep 1, 2023
</a>

      <div>
          <a data-turbo-frame="_top" class="post__title" href="/supporters/video_embeds/86797">
            Scene Test | Scene 32
</a>      </div>

      

        <div
          class="post__body"
            data-controller="trim"
            data-trim-class-value="rich-text--trimmed-short"
            data-trim-height-value="220"
        >
          <div class="rich-text" data-trim-target="content">
            <body>
<p><span>Hey Y'all!</span></p>
<p><span>We're thrilled to unveil the Zomborgs Teaser Trailer, a project that reflects months of learning and experimentation with UE5 &amp; AI technologies. It's been about 2 months since our <a href="http://news.ohheyvoid.com/2023/08/getting-ai-actors-to-talk-using.html" target="_blank" rel="noopener"><b>AI Test with Metahumans &amp; Stable Diffusion</b></a>, and we're excited to showcase what we've achieved by building upon that foundation.</span><span></span></p>
<p><span></span></p>
<p><span>Our journey has been one of trials, errors, and a ton of learning. It's taught us new skills and reminded us that even with AI's remarkable ability to elevate our work, it takes a combination of expertise in script writing, performance, 3D design, animation, and sheer determination to create something worthy of sharing.</span></p>
<p><span></span></p>
<p><span>For those of you interested in the process here are some of the details of what it took to make this thing happen. </span></p>
<p><span> </span></p>
<h2><span><b>🛠️ TOOLS &amp; SETTINGS USED 🛠️</b></span></h2>
<p></p>
<h3><span>+ <b><a href="https://www.unrealengine.com/" target="_blank" rel="noopener">Unreal Engine 5</a></b> (on a GTX 1070) </span></h3>
<p><span>Unreal Engine 5 became our mainstay for heavy lifting. Learning this software has been an ongoing journey, but the real-time animation workflows, asset libraries, and lighting capabilities have proved invaluable for elevating our storytelling.</span></p>
<p><span> </span></p>
<h3><span>+ <b><a href="https://www.unrealengine.com/en-US/metahuman" target="_blank" rel="noopener">Metahuman &amp; Metahuman Animator</a> </b>w/ iPhone 12-mini </span></h3>
<p><span>We continued our exploration of facial performance capture using Metahuman tools. Its accuracy and ease of use saved us significant time, making it an indispensable part of our toolkit. </span></p>
<p><span>Pro tip 1 - Subscribing to iCloud is worth every penny for larger projects. </span></p>
<p><span>Pro Tipe 2 - Matching your intended AI actor's head shape/features with a Metahuman character improves AI performance, and diverse head shapes help AI produce desired effects more easily.</span></p>
<p><span> </span></p>
<h3><span>+ <a href="https://quixel.com/mixer" target="_blank" rel="noopener"><b>Quixel Mixer</b> </a>(For custom Zombie) </span></h3>
<p><span>While Substance Painter is a popular choice, Quixel Mixer shouldn't be underestimated. Our budget constraints led us to Mixer, which offers quality UE5 and Megascans assets. It's a smart alternative for those looking to maximize resources.</span></p>
<p><span> </span></p>
<h3><span>+ <b><a href="http://move.ai/" target="_blank" rel="noopener">Move.AI</a> </b>(Experimental 4 Camera MoCap)</span></h3>
<p><span>Move.AI's multi-camera mocap trial served us well, even with minor hiccups. The credit system, while occasionally unpredictable, proved useful in the end. We used their experimental camera set up with various cameras (1 iPhone 12, 1 GalaxyNote9, 1 GalaxyNote10, and 2 gopros) and got the body movement for both characters in the piece.</span><span></span></p>
<p><span>We're also currently exploring options like Cascadeur, especially its new <a href="https://cascadeur.com/help/category/203" target="_blank" rel="noopener">MoCap Alpha</a>. </span><span></span></p>
<p><span>Other Ai-Video-Mocap options we're keeping an eye on include; Rokoko Vision just released a <a href="https://www.youtube.com/watch?utm_source=Youtube&amp;utm_medium=Main-launch&amp;utm_campaign=rokoko-vision&amp;utm_content=videodescription&amp;v=X30-2zcFI8M&amp;feature=youtu.be" target="_blank" rel="noopener">dual cam version</a>. There are also projects like <a href="http://freemocap.com/" target="_blank" rel="noopener">FreeMoCap.com</a> and <a href="http://remocap.com/" target="_blank" rel="noopener">ReMoCap.com</a> that we're keeping an eye on and hope to test in the future as well. </span></p>
<p><span> </span></p>
<p><span><b>*UPDATE*</b> We are happy to announce that this piece won their discord #show-off competition and got us 600 move.ai credits, which we're pretty pumped about! **</span></p>
<p><span> </span></p>
<h2><span><b>+ <a href="https://stability.ai/" target="_blank" rel="noopener">Stable Diffusion 1.5</a> w/ <a href="https://github.com/AUTOMATIC1111/stable-diffusion-webui" target="_blank" rel="noopener">Automatic 1111</a></b></span></h2>
<p><span>I know, I know. SDXL is out! What the heck are we doing still on 1.5? Look, okay? We run a GTX 1070 with low vram. While ComfyUI is cool - it makes my brain melt - plus batching and controlnt wasn't an option when we needed it. So while Automatic1111 fixes the bugs - we did this one on 1.5 and it did pretty nicely. </span></p>
<p><span> </span></p>
<p><strong>Basic Settings Used: </strong></p>
<p><span>Steps: 12 | DPM++ 2M Karras | CFG 25 | Denoise .3</span></p>
<p><span>ControlNet (Lineart-Realistic | Normal Bae | Softedge-Hed)</span></p>
<p><span>Models: <a href="https://civitai.com/models/16804?modelVersionId=2968" target="_blank" rel="noopener">LifeLikeDiffusion Model</a> and the <a href="https://civitai.com/models/58390/detail-tweaker-lora-lora" target="_blank" rel="noopener">Detail Tweaker Lora</a> (1.0 setting)</span><span></span></p>
<p><span>This round we followed <a href="https://www.youtube.com/watch?v=Z6pR7IB9in0&amp;t=0s" target="_blank" rel="noopener">a tutorial</a> by <a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/channel/UCZoiQ87mrg0zvVJsScRy1rA" target="_blank" rel="noopener">HilarionOfficial</a> that had some great tips for temporal consistency. This involved a higher CFG setting which allowed us to use a higher Denoise in our img2img process. </span></p>
<p><span>The LifeLikeDiffusion Model was really great to work with and does a great job with our diverse cast. The Detail Tweaker Lora was used for Yuki's shots as it added some nice skin texture and hair strands in some of the shots. </span><span></span><span></span></p>
<p><span>Edit: When we shared this process with some folks I realized they didn't really understand "WHY" we had to include an AI PASS at all. For reference here are comparison shots of the "Metahuman" and what the AI Pass offered as an alternative. </span><span></span></p>
<p><span></span></p>
<div><span>Metahuman Frame on the Top VS AI Pass Frame on Bottom</span></div>
<p><span> </span></p>
<div class="separator"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFT5wHRXZEc21HcFu8SC9IHqlioV_2sUZ8Cp0EosGAahR9wZ6mbN84VhnPY6vbmCRoTxn4cDZIFmVTrAVMCeNMlhM3_m9sm3mBUwYBTpzfjc2qGFHp8ecI0731cordyeOF6fOUlx8FrTzYeJ1JcJMlm4tiIjF5JEQo0l5lE9P3DTZbiBi5mM6u-p6lm3c/s2042/Meta-to-AI2.png" target="_blank" rel="noopener"><span><img height="345" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFT5wHRXZEc21HcFu8SC9IHqlioV_2sUZ8Cp0EosGAahR9wZ6mbN84VhnPY6vbmCRoTxn4cDZIFmVTrAVMCeNMlhM3_m9sm3mBUwYBTpzfjc2qGFHp8ecI0731cordyeOF6fOUlx8FrTzYeJ1JcJMlm4tiIjF5JEQo0l5lE9P3DTZbiBi5mM6u-p6lm3c/w613-h345/Meta-to-AI2.png" width="613" onerror="this.style.display='none'"></span></a></div>
<div class="separator"></div>
<div class="separator"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRKUPZWH4PZ3AZypXaVTPVYGDlcpAr9HbFub5YEjEbcTuHgfdKOob-xZlNIsg2yWBFIYqe6lXDr_ApgqOMAeTTznz6J8NPAuoC13M386eje9Uemb18vSs9r1M4yaD5tdYaWwDrdr51e_-dNlHMv2uMhdQCA2MBEYzYoSJ3rTH6Tu87bBhBiu4ZdnEKozI/s2042/Meta-to-AI.png" target="_blank" rel="noopener"><span><img height="350" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRKUPZWH4PZ3AZypXaVTPVYGDlcpAr9HbFub5YEjEbcTuHgfdKOob-xZlNIsg2yWBFIYqe6lXDr_ApgqOMAeTTznz6J8NPAuoC13M386eje9Uemb18vSs9r1M4yaD5tdYaWwDrdr51e_-dNlHMv2uMhdQCA2MBEYzYoSJ3rTH6Tu87bBhBiu4ZdnEKozI/w621-h350/Meta-to-AI.png" width="621" onerror="this.style.display='none'"></span></a></div>
<div class="separator"><span></span></div>
<div class="separator">To us  the AI Pass looks more "realistic" than the Metahuman frames. It removes that Metahuman uncanny valley thing and puts a layer of "Human Looking" on top. Thus upgrading our quality significantly, considering we've only just started learning the inner workings of 3D design, 3D Animation, and Character Design. </div>
<p></p>
<h3><span>+ <b><a href="https://www.midjourney.com/" target="_blank" rel="noopener">Midjourney</a> + <a href="https://github.com/lllyasviel/ControlNet" target="_blank" rel="noopener">ControlNet</a> + </b><b><a href="https://github.com/bmaltais/kohya_ss" target="_blank" rel="noopener">KohyaSS Lora Training </a></b></span></h3>
<p><b><span> </span></b></p>
<p><span>For this project we did train a new Lora for Yuki. Her original Lora was trained on Midjourney version 2 &amp; 3 character concepts which were prompted using Metahuman source images so they had a bit of a cgi/concept art feel to the training data. </span></p>
<p><span> </span></p>
<p><span>By using our original Lora, LifeLikeDiffusion, and ControlNet we generated new dataset images that were more reallistic and less "cartoony." This helped us get a little more consistent head turns and realistic skin textures on the final video AI pass. </span></p>
<p><span> </span></p>
<p><span>In addition to a new Yuki Lora we had to build a custom Metahuman to build our Zomborg. As mentioned above we used Quixel Mixer to customize the metahuman textures. We then rendered out some stills and prepped them for a general Lora dataset trained on Kohya. </span></p>
<p><span> </span></p>
<p><span>Because the LifeLikeDiffusion model was having trouble with zombie-fying our Zomborg we needed to train a general zombie style lora. We used midjourney to create this Zombie dataset using basic text-to-image prompting and our metahuman zombie as image reference to get a variety of different zombies for a general Zombie dataset. </span><span></span></p>
<p><span></span></p>
<p><span>We then trained a style Lora for this general Zombie dataset. This made it possible for us to mix both the Zomborg Lora and the Zombie Lora in a single prompt to get more intense "zombie features." </span><span></span></p>
<h3><span>+ <b>Audio Clips from: <a href="http://freesound.org/" target="_blank" rel="noopener">Freesound.org</a> | <a href="http://mixkit.co/" target="_blank" rel="noopener">Mixkit.co</a> | <a href="http://pexels.com/" target="_blank" rel="noopener">Pexels.com</a></b> (Mixed by Amber)</span></h3>
<p><span>was pivotal, and we used free resources from Freesound.org, Mixkit.co, and Pexels.com. I then mixed them in  <a href="https://www.adobe.com/products/audition.html" target="_blank" rel="noopener">Adobe Audition</a>. </span></p>
<p><span></span></p>
<div>
<h2><b><span>👨🏽‍🏫 TUTORIALS THAT GUIDED US 👨🏽‍🏫</span></b></h2>
This is a collection of tutorials used for this screen test.</div>
<div><b></b></div>
<div>
<b>Getting Started with Metahuman Animator</b> - <a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/channel/UCBobmJyzsJ6Ll7UbfhI4iwQ" target="_blank" rel="noopener">@UnrealEngine</a> A great straight forward way to set up and use Metahuman Animator for face performance. This one came in handy and its from the folks who made the app. Huzzah! (<a href="https://youtu.be/WWLF-a68-CE)" target="_blank" rel="noopener">https://youtu.be/WWLF-a68-CE)</a>
</div>
<div><b></b></div>
<div>
<b>Move AI |  </b><b>Complete Guide to Getting Set Up</b> - <a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/channel/UCmzWP6o2cw73moEF7LO_KvA" target="_blank" rel="noopener">@JonathanWinbush</a> He goes over the I-phone set up. His approach was helpful even though we used to the experimental workflow. (<a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/watch?v=MY7c6-_GXmg&amp;t=0s" target="_blank" rel="noopener">https://www.youtube.com/watch?v=MY7c6...</a> )</div>
<div><span><b></b></span></div>
<div><span><b>Move AI - Set Up Playlist</b> - <a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/channel/UCeaN10kbdPhqSPErlDp0IYw" target="_blank" rel="noopener">@moveai</a> This is a great playlist for the Move.Ai motion capture process. (<a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/watch?v=ZpvjDNFwKZQ&amp;list=PLs4kkUwQCTkUhjQ5LIk5qEsz3i1wM5sqF&amp;index=1&amp;t=0s" target="_blank" rel="noopener">https://www.youtube.com/watch?v=ZpvjD...</a> ) </span></div>
<div><span><b></b></span></div>
<div><span><b>3D Animation w/ AI Pass </b>- <a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/channel/UCP2RXiW0zYz7N2CnR9ANRCQ" target="_blank" rel="noopener">@promptmuse</a> This was the general workflow we approached this with but instead of reallusion products we used UE5 &amp; Metahumans. We did not use the multiscreen script for this one - but hope to in the future. (<a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/watch?v=T2nw9BThSvo&amp;t=0s" target="_blank" rel="noopener">https://www.youtube.com/watch?v=T2nw9...</a> )</span></div>
<div><span><b></b></span></div>
<div>
<span><b>Animation w/ ControlNet Only</b> - <a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/channel/UCZoiQ87mrg0zvVJsScRy1rA" target="_blank" rel="noopener">@HilarionOfficial</a> This was a creat tutorial that explained concepts and settings very well. It's what allowed us to really force our custom Lora onto the characters. (<a class="yt-core-attributed-string__link yt-core-attributed-string__link--display-type yt-core-attributed-string__link--call-to-action-color" href="https://www.youtube.com/watch?v=Z6pR7IB9in0&amp;t=0s" target="_blank" rel="noopener">https://www.youtube.com/watch?v=Z6pR7...</a>)<br><br><br></span>
<p>CONCLUSION</p>
<p><span>Our Zomborgs Teaser Trailer was a project-based learning experience. We harnessed disruptive technologies, pushed our creative boundaries, and used our limited budget wisely enough that we're looking forward to making another project very soon. We hope this peek behind the curtain offers insights into the journey. </span></p>
<p><span>If you haven't already don't forget to download Zomborgs.... now titled <a href="https://ohheyvoid.com/products/survivors-dont-die-episode-1-ebook" target="_blank" rel="noopener">Survivors Dont Die!</a></span></p>
<p></p>
<div><span>Until the next adventure,</span></div>
<div><span>J-Wall &amp; Amber</span></div>
<div><span>Oh Hey Void</span></div>
<p></p>
<span></span>
</div>
</body>
          </div>
          <button class="text-button text-button--pale post__action-button hidden" data-action="click-&gt;trim#expand" data-trim-target="button">
    ...Continue reading
</button>
        </div>

      

        <div class="post__section">
          <div class="post-actions">
            <form class="post-actions__item-form" data-turbo="false" action="/supporters/sign_up" accept-charset="UTF-8" method="get">
  <button class="text-button text-button--small text-button--pale" aria-label="Become a member">
    
    <div class="post-actions__item">
      <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="none" viewBox="0 0 16 16" role="img" class="post-actions__icon"><path fill="currentColor" fill-rule="evenodd" d="m2.662 7.721 5.14 5.918a.25.25 0 0 0 .378 0l5.142-5.92c1.856-2.21 1.25-4.386.03-5.37-.62-.5-1.407-.711-2.203-.513-.796.197-1.712.833-2.504 2.243a.75.75 0 0 1-1.308-.001c-.794-1.416-1.708-2.054-2.5-2.253-.79-.2-1.573.01-2.19.51-1.214.983-1.822 3.167.015 5.386Zm5.33-5.375C7.172 1.274 6.212.623 5.202.37c-1.292-.325-2.552.032-3.5.8-1.913 1.55-2.524 4.702-.19 7.515l.012.013 5.146 5.925a1.75 1.75 0 0 0 2.642 0l5.146-5.925.008-.009c2.362-2.805 1.75-5.956-.171-7.507-.95-.766-2.213-1.124-3.508-.802-1.01.25-1.974.898-2.795 1.966Z" clip-rule="evenodd"></path></svg>

          <span class="post-actions__item-number">
            3
          </span>
    </div>

</button></form>
              <form class="post-actions__item-form" data-turbo="false" action="/supporters/sign_up" accept-charset="UTF-8" method="get">
    <button class="text-button text-button--small text-button--pale" aria-label="Become a member">
    
      <div class="post-actions__item">
        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="none" viewBox="0 0 16 16" role="img" class="post-actions__icon"><path fill="currentColor" fill-rule="evenodd" d="M1.75 2.25a.25.25 0 0 0-.25.25v8.067c0 .139.112.25.25.25H3c.967 0 1.75.784 1.75 1.75v1.21c0 .216.255.33.416.187l3.053-2.706a1.75 1.75 0 0 1 1.16-.44h4.871a.25.25 0 0 0 .25-.25V2.5a.25.25 0 0 0-.25-.25H1.75ZM0 2.5C0 1.534.784.75 1.75.75h12.5c.966 0 1.75.784 1.75 1.75v8.067a1.75 1.75 0 0 1-1.75 1.75H9.38a.25.25 0 0 0-.166.063L6.16 15.087c-1.13 1-2.911.199-2.911-1.31v-1.21a.25.25 0 0 0-.25-.25H1.75A1.75 1.75 0 0 1 0 10.567V2.5Z" clip-rule="evenodd"></path></svg>

        <span class="post-actions__item-number"></span>
      </div>

</button></form>
            
<div class="dropdown" data-controller="dropdown link-share" data-dropdown-placement-value="bottom-start" data-action="link-share:unavailable-&gt;dropdown#toggle" data-link-share-url-value="https://ohheyvoid.com/supporters/video_embeds/86797?utm_medium=copy-share-link&amp;utm_source=share-link&amp;utm_campaign=post-share-supporter">
      <div class="comment__menu" data-dropdown-target="button" data-action="click->link-share#share">
      <div class="post-actions__item">
        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="none" viewBox="0 0 16 16" role="img" class="post-actions__icon"><path fill="currentColor" fill-rule="evenodd" d="M6.996.471a1.41 1.41 0 0 1 2.008 0l4.943 5.013-1.068 1.053L8.75 2.35v9.121h-1.5V2.35L3.12 6.537 2.054 5.484 6.996.471ZM1.5 11.108v3.143c0 .138.111.249.249.249H14.25c.138 0 .249-.11.249-.25v-3.142H16v3.143c0 .965-.781 1.749-1.749 1.749H1.75A1.748 1.748 0 0 1 0 14.25v-3.142h1.5Z" clip-rule="evenodd"></path></svg>

        <span class="post-actions__item-number hidden@sm">Share</span>
      </div>
    </div>


  <div class="dropdown__menu hidden" data-dropdown-target="items">
    <div class="dropdown__items">
        <div class="dropdown__title">Share this post</div>

      

  <button class="dropdown__item" data-action="click-&gt;dropdown#hide" data-controller="clipboard" data-clipboard-text="https://ohheyvoid.com/supporters/video_embeds/86797?utm_medium=copy-share-link&amp;utm_source=share-link&amp;utm_campaign=post-share-supporter" type="button">
    <div class="dropdown__item-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="none" viewBox="0 0 16 16" role="img"><path fill="currentColor" fill-rule="evenodd" d="M12.145 1.5a1.762 1.762 0 0 0-1.246.516L8.234 4.681l-1.06-1.06L9.837.955a3.264 3.264 0 0 1 4.615 0l.591.591a3.264 3.264 0 0 1 0 4.613l-3.849 3.85a3.262 3.262 0 0 1-4.614 0l-.593-.592 1.062-1.06.591.592a1.763 1.763 0 0 0 2.493 0l3.85-3.85a1.762 1.762 0 0 0 0-2.492l-.592-.591a1.764 1.764 0 0 0-1.247-.517ZM7.112 6.534c-.468 0-.916.186-1.247.516L2.016 10.9a1.762 1.762 0 0 0 0 2.492m0 0 .592.592a1.764 1.764 0 0 0 2.493 0l2.665-2.665 1.06 1.06-2.664 2.666a3.264 3.264 0 0 1-4.615 0l-.592-.592a3.263 3.263 0 0 1 0-4.614l3.85-3.85a3.264 3.264 0 0 1 4.614 0l.592.593-1.06 1.06-.592-.592c-.331-.33-.78-.516-1.247-.516" clip-rule="evenodd"></path></svg>

    </div>

  
    Copy link

</button>
  <a class="dropdown__item" data-action="click-&gt;dropdown#hide" href="https://twitter.com/intent/tweet?url=https%3A%2F%2Fohheyvoid.com%2Fsupporters%2Fvideo_embeds%2F86797%3Futm_medium%3Dcopy-share-link%26utm_source%3Dshare-link%26utm_campaign%3Dpost-share-supporter" target="_blank">
    <div class="dropdown__item-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 32 32" fill="none" role="img"><path d="M18.666 13.857 29.093 2h-2.47l-9.056 10.294L10.338 2H2l10.932 15.567L2 30h2.47l9.557-10.873L21.662 30H30M5.36 3.822h3.795L26.62 28.267h-3.794" fill="currentColor"></path></svg>

    </div>

  
    Share on X

</a>
  <a class="dropdown__item" data-action="click-&gt;dropdown#hide" href="https://facebook.com/sharer.php?u=https%3A%2F%2Fohheyvoid.com%2Fsupporters%2Fvideo_embeds%2F86797%3Futm_medium%3Dcopy-share-link%26utm_source%3Dshare-link%26utm_campaign%3Dpost-share-supporter" target="_blank">
    <div class="dropdown__item-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 14 14" fill="none" role="img"><path d="m5.27 14-.02-6.125H2.625V5.25H5.25V3.5C5.25 1.138 6.713 0 8.82 0c1.009 0 1.876.075 2.129.109v2.468H9.488c-1.146 0-1.368.545-1.368 1.344V5.25h3.255L10.5 7.875H8.12V14H5.27Z" fill="currentColor"></path></svg>

    </div>

  
    Share on Facebook

</a>
    </div>
  </div>
</div>
          </div>

        </div>

      </div>
</div>

  </div>
</div>

</turbo-frame><turbo-frame class="main-list__list-item" data-testid="Post" id="post_86798">
    <div class="post" access="public">
  <div class="post__inner">
      <div class="post__media">
        <div class="media-player media-player--video">
            <div
  class="embed-player"
  data-controller="youtube-player"
  data-youtube-player-watch-times-path-value="https://ohheyvoid.com/supporters/api/v1/media_catalog/posts/video_embeds/86798/watch_times"
  data-youtube-player-video-id-value="_yf-pjU31GM"
>
  <div class="media-player__cover" data-youtube-player-target="element">
    <img src="https://storage.googleapis.com/popshopprod-membership-assets-single-b5px4371/wj9d7rkb5qhoguwt18uq388ye1vs" class="media-player__cover-image media-player__cover-image--cover" loading="lazy" />
    <button type="button" class="media-player__cover-button" data-action="click->youtube-player#createPlayer" data-testid="YoutubePlayer.PlayButton">
      <svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" viewBox="0 0 32 32" fill="none" role="img"><path d="M28.422 14.211c1.474.737 1.474 2.84 0 3.578L2.894 30.553A2 2 0 0 1 0 28.763V3.237a2 2 0 0 1 2.894-1.789l25.528 12.764Z" fill="currentColor"></path></svg>

    </button>
  </div>
</div>

        </div>
      </div>

    <div class="post__main">
  <div class="post__content">
        <a data-turbo-frame="_top" class="post__meta" href="/supporters/video_embeds/86798">
          Jun 29, 2023
</a>

      <div>
          <a data-turbo-frame="_top" class="post__title" href="/supporters/video_embeds/86798">
            Getting AI Actors to Talk using Metahumans and Stable Diffusion
</a>      </div>

      

        <div
          class="post__body"
            data-controller="trim"
            data-trim-class-value="rich-text--trimmed-short"
            data-trim-height-value="220"
        >
          <div class="rich-text" data-trim-target="content">
            <body>
<h4>The journey has begun! </h4>
<p></p>
<p>We recently "soft" launched our digital download of<span> </span><a href="https://zomborgs.ohheyvoid.com/" target="_blank" rel="noopener">ZOMBORGS Episode 1</a>. A sci-fi horror series about a girl, an android, a corporate cult, and lots of zombies with a bit of extra "holy-eff!" </p>
<p></p>
<p>We're determined to see how far UE5, Blender, and Synthography can take us to make a high-end trailer for the script book currently available for download (<a href="https://zomborgs.ohheyvoid.com/" target="_blank" rel="noopener">here</a>). The goal isn't to "just get it done." We want to create Hollywood-quality visuals for a series that could be an epic tale of human identity and social conformity. </p>
<p></p>
<p>Sure, we've worked in video production for all of our adult lives, but in order to do the high-end visuals we want we'd need a massive budget, epic gear, and a vfx powerhouse propelling us forward to a big fat Hollywood "maybe." None of which we have at the moment. </p>
<p></p>
<p>What we do have, however, is self-taught 3D design skills, a general understanding of animation in UE5 &amp; blender, comfort in stable diffusion, and an iPhone. </p>
<p></p>
<p>That should get us where we gotta go, right? </p>
<p></p>
<p>Maybe. </p>
<p></p>
<h4>The Test</h4>
<p>The test we ran this week explored what the workflow would look like if we face-captured with UE5's new Metahuman Animator, ported that into a metahuman, and then took those frames into stable diffusion to get our custom character talking using a voice2voice trained model. </p>
<p></p>
<h3>The Models</h3>
<p>Thankfully, we already had our LORA's trained from our script-book project. And we already knew that we liked the<span> </span><a href="https://civitai.com/models/16804/life-like-diffusion" target="_blank" rel="noopener">LifeLikeDiffusion model</a><span> </span>found on CivitAI. It does a great job of capturing our ethnically diverse cast and life-like proportions &amp; expressions. </p>
<p></p>
<p>We also already had our<span> </span><a href="https://www.unrealengine.com/en-US/metahuman" target="_blank" rel="noopener">MetaHuman</a><span> </span>base designed, as we used these metahumans to influence our synthetic actors' looks with<span> </span><a href="https://www.midjourney.com/" target="_blank" rel="noopener">Midjourney</a><span> </span>img2img and blend workflows. </p>
<p></p>
<h4>The Face MoCap</h4>
<p>With the release of<span> </span><a href="https://www.unrealengine.com/en-US/blog/new-metahuman-animator-feature-set-to-bring-easy-high-fidelity-performance-capture-to-metahumans" target="_blank" rel="noopener">MetaHuman Animator</a><span> </span>earlier this month we splurged on a refurbished iPhone 12-mini and quickly got to work on testing the setup process. It took a minute, but Jwall is a genius and power hit the fine performance demonstrated in the test footage. Needless to say, we were impressed with the mocap capabilities and very pleased with the first test. </p>
<p></p>
<h4>The Voice2Voice</h4>
<p>Next up, we wanted to be able to use AI voices - but we didn't like any of the text-to-voice options. They sounded sterile, disturbing, and inhumane. Meaning we had to get acquainted with the voice2voice AI options out there. Unfortunately, I was unable to get the local option I wanted to try installed properly. So, for the sake of time, we went with the social media popular,<span> </span><a href="https://voice.ai/r/ift8Y" target="_blank" rel="noopener">Voice.ai</a> (referral link). </p>
<p></p>
<p>A few years back I (Amber) had tried my hand at narrating audiobooks. It was one of the most grueling, fun, and underpaid gigs I'd done in my entire career. I narrated two books and ended my VO career quickly after. The bright side of having done it was I now have hours of audio that I could use to train my voice with. And so I did. Within an hour I had an ai voice model of myself. I ran Jwall's voice performance through it and BAM! We got us a girly voice! </p>
<p></p>
<p>It was really important to us that the voice be able to capture performance in tonality and expression. Voice.ai did a pretty good job. I just wish it wasn't so distorted. I ended up having to do A LOT of post-cleanup and it still sounds a bit off. But... we had a voice that was closer to that of a 13-year-old girl - so we moved on. </p>
<p></p>
<p><b>Stable Diffusion</b></p>
<p>The final step was putting our synthetic actress' face onto the performance. Using<span> </span><a href="http://runpod.io/" target="_blank" rel="noopener">Runpod.io</a><span> </span>(referral link) we ran a RTX 3090 and hoped for the best. Using our previously trained Lora, img2img, controlNet, and the Roop extensions we hobbled our way through it. </p>
<p></p>
<p>The biggest issue I ran into was anything with a denoise over .15-.2 failed to capture the mouth movement. After generating 600 frames of failed small batch testing I was definitely grateful for the batch image option within Stable Diffusion. </p>
<p></p>
<p>Prompt Muse's "<a href="https://www.youtube.com/watch?v=HUPcr5njxkM" target="_blank" rel="noopener">Taking AI Animation to the NEXT Level</a>" tutorial was helpful in suggesting the scribbleHed option. </p>
<p>However, I wasn't really digging the fact that I couldn't get the LORA to manipulate the output more. Maybe I was doing something wrong... </p>
<p>But I eventually stumbled across Nerdy Rodent's "<a href="https://www.youtube.com/watch?v=M-Vd0l6W9fg&amp;" target="_blank" rel="noopener">Easy Deepfakes in AUtomatic1111 with Roop</a>" and everything began sorting itself out. </p>
<p></p>
<p>Here are the SD settings I ended up using as I recall them. </p>
<div class="separator"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgn7qM2cen4YnAHrhd_aLUbAhuZ4nGV0Vf8DM5EKHQtfH_BIEs3YR2PG4XQqpiECM6HoX1PSyV9wjM0OOXNNvwxMCwRF3F9Ok9msTGNoHUNjSXQ1oC96M-U9ctofkUP1VcoXn0YSBF9uedVkPkpVBakQpLsKdxwcrrvTZvVSNL_eneR2XhPWXcEH5DZi_s/s1110/29.06.2023_08.03.50_REC.png" target="_blank" rel="noopener"><span><img height="394" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgn7qM2cen4YnAHrhd_aLUbAhuZ4nGV0Vf8DM5EKHQtfH_BIEs3YR2PG4XQqpiECM6HoX1PSyV9wjM0OOXNNvwxMCwRF3F9Ok9msTGNoHUNjSXQ1oC96M-U9ctofkUP1VcoXn0YSBF9uedVkPkpVBakQpLsKdxwcrrvTZvVSNL_eneR2XhPWXcEH5DZi_s/w646-h394/29.06.2023_08.03.50_REC.png" width="646" onerror="this.style.display='none'"></span></a></div>
<p></p>
<p>Sure, I probably could have got away with using the Roop extension only, but I wanted to run the lora as high as I could get it then apply roop to make sure the face looked as consistent and accurate as possible. Because, remember... I needed the girl to talk. </p>
<p>In the end... the results were a satisfying start. </p>
<p>For our next adventure we're planning to do more than just a single shot, maybe even throw in some body movement too. So this test definitely proved that it's possible to get the images to chatter and look semi-lifelike. We still have some work to do on the voice2voice ai, the flickering, and the additional facial movements. But we have faith that it's possible. </p>
<p>If you have any ideas on how to improve our workflow, DO SHARE! Either comment to this blog or the video above. Whichever you're more comfortable with.</p>
</body>
          </div>
          <button class="text-button text-button--pale post__action-button hidden" data-action="click-&gt;trim#expand" data-trim-target="button">
    ...Continue reading
</button>
        </div>

      

        <div class="post__section">
          <div class="post-actions">
            <form class="post-actions__item-form" data-turbo="false" action="/supporters/sign_up" accept-charset="UTF-8" method="get">
  <button class="text-button text-button--small text-button--pale" aria-label="Become a member">
    
    <div class="post-actions__item">
      <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="none" viewBox="0 0 16 16" role="img" class="post-actions__icon"><path fill="currentColor" fill-rule="evenodd" d="m2.662 7.721 5.14 5.918a.25.25 0 0 0 .378 0l5.142-5.92c1.856-2.21 1.25-4.386.03-5.37-.62-.5-1.407-.711-2.203-.513-.796.197-1.712.833-2.504 2.243a.75.75 0 0 1-1.308-.001c-.794-1.416-1.708-2.054-2.5-2.253-.79-.2-1.573.01-2.19.51-1.214.983-1.822 3.167.015 5.386Zm5.33-5.375C7.172 1.274 6.212.623 5.202.37c-1.292-.325-2.552.032-3.5.8-1.913 1.55-2.524 4.702-.19 7.515l.012.013 5.146 5.925a1.75 1.75 0 0 0 2.642 0l5.146-5.925.008-.009c2.362-2.805 1.75-5.956-.171-7.507-.95-.766-2.213-1.124-3.508-.802-1.01.25-1.974.898-2.795 1.966Z" clip-rule="evenodd"></path></svg>

          <span class="post-actions__item-number">
            3
          </span>
    </div>

</button></form>
              <form class="post-actions__item-form" data-turbo="false" action="/supporters/sign_up" accept-charset="UTF-8" method="get">
    <button class="text-button text-button--small text-button--pale" aria-label="Become a member">
    
      <div class="post-actions__item">
        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="none" viewBox="0 0 16 16" role="img" class="post-actions__icon"><path fill="currentColor" fill-rule="evenodd" d="M1.75 2.25a.25.25 0 0 0-.25.25v8.067c0 .139.112.25.25.25H3c.967 0 1.75.784 1.75 1.75v1.21c0 .216.255.33.416.187l3.053-2.706a1.75 1.75 0 0 1 1.16-.44h4.871a.25.25 0 0 0 .25-.25V2.5a.25.25 0 0 0-.25-.25H1.75ZM0 2.5C0 1.534.784.75 1.75.75h12.5c.966 0 1.75.784 1.75 1.75v8.067a1.75 1.75 0 0 1-1.75 1.75H9.38a.25.25 0 0 0-.166.063L6.16 15.087c-1.13 1-2.911.199-2.911-1.31v-1.21a.25.25 0 0 0-.25-.25H1.75A1.75 1.75 0 0 1 0 10.567V2.5Z" clip-rule="evenodd"></path></svg>

        <span class="post-actions__item-number"></span>
      </div>

</button></form>
            
<div class="dropdown" data-controller="dropdown link-share" data-dropdown-placement-value="bottom-start" data-action="link-share:unavailable-&gt;dropdown#toggle" data-link-share-url-value="https://ohheyvoid.com/supporters/video_embeds/86798?utm_medium=copy-share-link&amp;utm_source=share-link&amp;utm_campaign=post-share-supporter">
      <div class="comment__menu" data-dropdown-target="button" data-action="click->link-share#share">
      <div class="post-actions__item">
        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="none" viewBox="0 0 16 16" role="img" class="post-actions__icon"><path fill="currentColor" fill-rule="evenodd" d="M6.996.471a1.41 1.41 0 0 1 2.008 0l4.943 5.013-1.068 1.053L8.75 2.35v9.121h-1.5V2.35L3.12 6.537 2.054 5.484 6.996.471ZM1.5 11.108v3.143c0 .138.111.249.249.249H14.25c.138 0 .249-.11.249-.25v-3.142H16v3.143c0 .965-.781 1.749-1.749 1.749H1.75A1.748 1.748 0 0 1 0 14.25v-3.142h1.5Z" clip-rule="evenodd"></path></svg>

        <span class="post-actions__item-number hidden@sm">Share</span>
      </div>
    </div>


  <div class="dropdown__menu hidden" data-dropdown-target="items">
    <div class="dropdown__items">
        <div class="dropdown__title">Share this post</div>

      

  <button class="dropdown__item" data-action="click-&gt;dropdown#hide" data-controller="clipboard" data-clipboard-text="https://ohheyvoid.com/supporters/video_embeds/86798?utm_medium=copy-share-link&amp;utm_source=share-link&amp;utm_campaign=post-share-supporter" type="button">
    <div class="dropdown__item-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="none" viewBox="0 0 16 16" role="img"><path fill="currentColor" fill-rule="evenodd" d="M12.145 1.5a1.762 1.762 0 0 0-1.246.516L8.234 4.681l-1.06-1.06L9.837.955a3.264 3.264 0 0 1 4.615 0l.591.591a3.264 3.264 0 0 1 0 4.613l-3.849 3.85a3.262 3.262 0 0 1-4.614 0l-.593-.592 1.062-1.06.591.592a1.763 1.763 0 0 0 2.493 0l3.85-3.85a1.762 1.762 0 0 0 0-2.492l-.592-.591a1.764 1.764 0 0 0-1.247-.517ZM7.112 6.534c-.468 0-.916.186-1.247.516L2.016 10.9a1.762 1.762 0 0 0 0 2.492m0 0 .592.592a1.764 1.764 0 0 0 2.493 0l2.665-2.665 1.06 1.06-2.664 2.666a3.264 3.264 0 0 1-4.615 0l-.592-.592a3.263 3.263 0 0 1 0-4.614l3.85-3.85a3.264 3.264 0 0 1 4.614 0l.592.593-1.06 1.06-.592-.592c-.331-.33-.78-.516-1.247-.516" clip-rule="evenodd"></path></svg>

    </div>

  
    Copy link

</button>
  <a class="dropdown__item" data-action="click-&gt;dropdown#hide" href="https://twitter.com/intent/tweet?url=https%3A%2F%2Fohheyvoid.com%2Fsupporters%2Fvideo_embeds%2F86798%3Futm_medium%3Dcopy-share-link%26utm_source%3Dshare-link%26utm_campaign%3Dpost-share-supporter" target="_blank">
    <div class="dropdown__item-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 32 32" fill="none" role="img"><path d="M18.666 13.857 29.093 2h-2.47l-9.056 10.294L10.338 2H2l10.932 15.567L2 30h2.47l9.557-10.873L21.662 30H30M5.36 3.822h3.795L26.62 28.267h-3.794" fill="currentColor"></path></svg>

    </div>

  
    Share on X

</a>
  <a class="dropdown__item" data-action="click-&gt;dropdown#hide" href="https://facebook.com/sharer.php?u=https%3A%2F%2Fohheyvoid.com%2Fsupporters%2Fvideo_embeds%2F86798%3Futm_medium%3Dcopy-share-link%26utm_source%3Dshare-link%26utm_campaign%3Dpost-share-supporter" target="_blank">
    <div class="dropdown__item-icon">
      <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 14 14" fill="none" role="img"><path d="m5.27 14-.02-6.125H2.625V5.25H5.25V3.5C5.25 1.138 6.713 0 8.82 0c1.009 0 1.876.075 2.129.109v2.468H9.488c-1.146 0-1.368.545-1.368 1.344V5.25h3.255L10.5 7.875H8.12V14H5.27Z" fill="currentColor"></path></svg>

    </div>

  
    Share on Facebook

</a>
    </div>
  </div>
</div>
          </div>

        </div>

      </div>
</div>

  </div>
</div>

</turbo-frame></template></turbo-stream>

<turbo-stream action="remove" target="posts_load_more"></turbo-stream>

