Gadgetory

why the heck do you guys need a whole roomful of servers just to make youtube videos we get asked this actually a lot of other questions about our data management and our workflow all the time so for the last year actually ever since we got our red cameras I've been meaning to do an update for you guys on how we handle such heavy footage how we share resources with such a large team and how we do all of that while maintaining our rigorous daily release schedule so when G Winn reached out to us asking us to do a feature on their smooth for camera gimbal I thought well hey there's an opportunity why don't we grab this thing and take it with us while I show you guys the way that data flows through linus media group to become a video on your screen so while I let you all appreciate the irony of a video about our overly complicated workflow shot on a cell phone with a simple handheld gimbal I'll tell you guys a little bit more about the zoom four so it's got a highly integrated control panel down here with apps for both iOS and Android that allow you to control your camera it's got focus pull and zoom capabilities it's got the ability to do vertigo shots with one button through the app or manually it's got what they call phone go mode for instant scene transitions and it's got both a quick standby mode and a one-click mode switch everything that we shot outside of the intro and the souter right now was filmed on this it's got a no date on it right now but it was shot on an iPhone 10 and I'll let you guys be the judge of the stabilized footage that comes off of it so to demonstrate some of the challenges with working with red footage we're actually gonna take that intro that we just shot just now and we're gonna ingest the whole thing to show you guys how we do it now red cameras are not great for recording audio in fact even with an external preamp it is really difficult to get anything resembling usable audio out of the camera directly so recently we finally gave in and resorted to an external audio recorder it really sucks to synchronize audio especially if you're stuck using the scratch audio that gets recorded directly to the camera and then the SD card out of your external recorder and aligning it manually but we recently got a little doodad called a tentacle sync you get two pieces here a master that sits on our mix pre-six and a slave that sits on our camera you synchronize them at the beginning of the day and what they do is they inject what's called time code into the mag and into the SD card so that you can easily synchronize them later on down the line but that is far from the only problem so the thing about the weapon 8k so that's our camera recently renamed the dsm see to helium or something like that is that it records footage they're both the same camera at up to eight K 60 frames per second and with the 480 or 960 gig Meg's that's up to hundred megabytes per second so to put that in perspective if we were filming a simple tech linked episode even though we will usually use like a twenty to one compression ratio so that's about four times the compression that they would probably use on something like a feature film that means that a 10 gig clip could still end up being in the neighborhood of about - a 10 gig clip that means that a 10 minute clip could still end up being in the neighborhood of around 50 gigs and while it's less of an issue for something like Linus tech tips or tech quickie when you're recording dailies and you're expected to release the video on the same day that you film it like with something like a new show size of files ends up becoming a potential problem so that's why our to ingest stations which you might recognize from one of our tech showdown episodes are both equipped with 10 gigabit network cards so one of them has this Asus one which is basically the last generation version of this action 1 they're both using a quantities that allow them to be somehow under $100 for 10 gig networking there are a number of different ways that we ingest footage depending on the type of camera we're using so for red you can see each of our video clips is actually a folder made up of smaller broken up video clips so we're gonna go ahead and grab that and copy the whole thing into our project folder for this video so that goes in a roll one what we're looking at right here is probably some kind of like weird buffering thing or something but this is about what we'd expect to see in terms of our ingest speeds it's not the full 10 gigabit because that would be in the neighborhood of one to one point one gigabytes per second but the reality of it is that you're going to run into bottlenecks elsewhere and in fact these red Meg's even though we're using an eSATA interface for them are just not that fast and they only read it about 230 minutes a second then what we got to do is grab the audio clip off of our SD card that goes into our audio folder right here and that's pretty much the whole process for red footage but for that we wouldn't need powerful machines and there is a reason that each of these is running 64 gigs of ram and a 10 core Extreme Edition processor and that's because for our other cameras like the a7 s for example we actually use Adobe prelude which is an imperfect piece of software but has its uses to transcode the footage when we're bringing it in so what we do is we take whatever project it is and this is generally fast as possible we still shoot that on the a7 s we create a subfolder and we set it to transcode to Cinna form 4k this improves our timeline performance when we're scrubbing through the clips in Adobe Premiere the other thing that we do is set a second destination for the original files that we're not transcoding just in case something goes wrong with the transcode which does happen from time to time and we need to go back and grab the original files now just to give you some idea even if we're not using the full potential of this network connection all the time of what it's capable of let's go ahead and just grab a file off the desktop and show you just what this puppy can do so that is saturating the read speeds of the raid 1 SSDs that each of these machines are booted off of so that was a 10 gigabyte file let's go have a look on the other end of this Ethernet cable at how exactly that whole thing comes together now I've shown you guys our server room fair number of times but I haven't given you guys an update in quite a while on how exactly it's working in here so the main server that everyone is editing off of at the same time is this one right here this is one Xserve err and it's running 24 plus 1/2 four or five six seven so it's running 31 Intel 750 series one point one terabyte nvme SSDs and this guy is an absolute monster so if we fire up performance monitor you can see that even though we've got four of our editors in office right now and we're seeing each of them doing a hundred and fifty one hundred and twenty fifty megabytes a second of reads this thing is barely suffering and our there we go our Z disk you death is only about 0.5 that's the benefit of nvme is that nice fast responsiveness and in fact when we were still using our SATA SSD server for every one we were starting to run into issues once we had the room fully staffed over there where Adobe Premiere would crash but not just trash everyone's would crash at the same time and we narrowed that down to slow response times on our nas so this guy's connected at 40 gigabits per second this one right here has 24 SATA SSDs and this one only gets used this is called QQ server this one only gets used for large projects that only one person needs to work on at a time it's still SSD based so we still don't run into any crashes or any other weird issues like that but it doesn't have quite the snap that an nvme machine does now something that we learned a really valuable lesson about I guess it must have been about two years ago was real-time data replication and off-site backup and that is where this guy comes in so I want to show you guys a little trick here I don't know if I've ever actually done this demo in a video before I've shown people that have come into our office so I'm going to use my test folder here and I'm going to create a document called test for video right here this is a text file on our main drive then I'm going to jump over to this hard drive based server under it so this is running 8 8 terabyte drives for a total of 64 terabytes and I'm just gonna pull up the Wanek sink folder get that test folder open test iridium put back that hole just like that so within about five to ten seconds whether it's a text file or whether it's a large video file the synchronization software that we're running here will automatically dump it over to here so if this crashes we can actually continue editing videos within about 10 to 15 minutes of switching over everyone's map drives and another benefit that it gives us is that normally on a network drive when you delete a file it's just gone but instead check this out if I delete test for video I'm actually going to have to do it from a separate server but if I delete test for video Boop you're gone you're done and I go into here that file is going to disappear what what will happen is we can go into the deletions folder and we can rescue it this has actually saved our butts more times than I would like to admit because stuff does get accidentally deleted now all of this is only for active projects that we are working on right now once we're done with something it goes on to petabyte project and we're actually running both phases of it right now when we originally deployed it we were only using one of them so we could just save power on ours for all the drives that were in the other one not the case anymore so petabyte project is now up to seven hundred and seventy-seven terabytes of total space of which we've consumed four hundred and thirty-two this holds all of our archived projects so that in the event that we want to make a video that refers back to one of our other videos we can grab the original quality files rather than downloading off YouTube like a lot of other youtubers do bring us into this room one of the biggest challenges is not just working with red footage because the files are so heavy one of the biggest problems that we have is that we've got multiple people working off of that same shared nas all at the same time so this is where the 10 gigabit connections that our editors are also using comes in we're more scrubbing through footage we can actually see data rates in excess of two-and-a-half gigabit per second so we peaked at just shy of four gigabit per second that my friends is why we've got the bangin mass with the high-speed networking so once the edits done Taron or one of our other editors will export the entire project as an MOV but those are extremely inefficient files so they're they're great quality but they're absolutely massive so he's gonna copy that those in the transcode flow plane yeah and then you pasted here now what's gonna happen there is that water-cooled server that I built a while ago the one that's at the very bottom of the rack is going to see that that file got transferred and then it's going to spit out the correct formats for all the different platforms that we upload to whether it be YouTube or flow plane or Facebook or whatever the case may be that way the editors machine doesn't get tied up with this so this is cool once that file is done copying it does take a second and media encoder is not always perfect before it'll pick it up but we use VNC in order to remote into that machine so everyone can go in at the same time unlike remote desktop connection and make sure that it has actually picked up the project and that it's transcoding to the correct format okay cool the system works so thanks for watching guys if you disliked this video you can hit that button but if you liked it hit like get subscribed or maybe consider checking out where to buy the stuff we featured at the link in the video description also down there is our merch store which has cool shirts like this one and our community forum which you should totally join