Gadgetory


All Cool Mind-blowing Gadgets You Love in One Place

36 Cores, 72 Threads, and Two Titan Xs - Our New INSANE Rendering Machine Part 3

2015-06-22
so this is exciting we are finally going to be validating if our idea for the rendering machine over there is going to work but first we need a long Uther net cable so FreshBooks is the online invoicing solution that lets you get organized save time and get paid faster click on this spot right here for a free trial now alright so one of the few pieces of the puzzle still MIT well there's a few pieces of the puzzle still missing ok one of the pieces that the puzzle is still missing is getting a 10 Gigabit Ethernet connection to that machine so we've swapped out to the Supermicro board that doesn't have it on board but the good news is oh I just dropped one ok well the good news is my X 540 s and other 10 gigabit cards are here now so we're going to go ahead and get that puppy linked up then we have to actually build the machine I've just had it in a test bench up till now so huh this this was the thinking behind the carabiners in the ceiling okay so now in theory we are 10-gigabit sick all right we're good so let's let's actually let's actually build this machine finally finally okay so I thought that this RPC 4224 was actually going to be one for us to grow into but it looks like we need it now so this is the last of my 4u cases from Norco that they graciously provided for our new server room and we are going to be using this one for the rendering machine so there's all of our mounting hardware is a there's a nice soft foam that my cat would eat if he could get at it and he would try to swallow it and he would then get it stuck in his stomach which he did again costing me another $1,200 he does it again I'm gonna kill him that's twice in four months he's eating foam and he did surgery oh so there is it yeah it would have been cheaper to get two new cats the first time now I could have had three new cats and like a pretty cool lizard but he's my baby and I love him that's what I tell myself like gee why doesn't Linus have a Lambo with all that YouTube money because he has the world's most expensive cat that's why okay I'm gonna need a minute to clear some space here I think and machine gun power supply now I could leave the 860 I that I have in there but then if I ever want to actually put graphics cards in because I'm going to take those graphics cards out if I ever wanted to put proper graphics cards in then I would have to replace it with something else which is kind of a pain but so mmm you know what no I'm gonna stick with the 860 I the odds of throwing graphics cards in here are pretty slim do you know how long it's been since I've seen a motherboard where I actually have to pop the covers on the aisle plate off myself it's like are you for real Yolo man oh that's really tight okay I need pliers okay just doing a quick test fit here finding out where the devil all the holes are so that one over there and this one right here what okay so if that one there's that one there's that one where's this one you need all of them well that's optimal Eve it - I think that's an idea um this show is all about the right way to do things so it's at least three I guess we'll need a power supply at some point okay oh my bad defender is over at the frontier it's perfect it's like Christmas kind of a weak Christmas a hand header your expectations are so high how how did you know that I wasn't going to do fan headers for Christmas bonuses this year now my feelings are hurt they should be you know how that my feelings are hurt maybe there'll be no Christmas bonus do you ever think of that but you'd rather have a fan header than nothing yeah I think I'd rather nothing than fan headers just because it's kind of insulting duly noted sorry Christmas everyone is poop oh you don't like poop oh you want Putin nothing fish I think everyone in the comments will agree with me that fan headers are better than poop that doesn't even make sense at least you've used poop or what leaving on your boss's desk forgiving a fan ever since azuz give you I actually ordered a game with little turds and velcro helmets that you wear that they stick to on Amazon that I was thinking we'd play on until super flood so it occurs to me now I need a graphics card I didn't really want to put one in here but this motherboard has no onboard graphics and you're sure you can't use a Titan so I'm thinking as I install the power supply this is the one piece of hardware in this machine that I would really like to change not because the 860 watt isn't enough for for what we're going to do and that may be the case in the future if we find some kind of application that can take advantage of GPU which is why I wanted a motherboard in the first place that would support multiple GPUs but the reason I would like to have a different power supply in here is actually more to do with redundancy but finding a the mini redundance that's what they're called a mini redundant power supply one in this form factor of sufficient wattage for this system with a couple of high-power GPUs actually not not easy there was a 600 watt there may be something higher than that feel free to leave a message in the comments if you guys can can find something better than that but yeah I'm pretty much stuck with a single non redundant power supply the good news is this is a very reliable power supply and this machine won't actually be storing anything mission-critical data goes on to another server gets ripped off of it and then transcoded over to another server by this one it never actually actually touches the data so it should be fine it's just I would rather I would rather it was actually redundant and it'll be on a UPS as well so the power supply would have to fail to take it down even temporarily I guess it's alright alright in theory this boots now anytime hey alright ok so here we are here's our 72 CPU threads only half of which are actual course weaksauce I know and this my friends is the reason for some of the confusion around this particular system had a lot of people asking things like why do you need so many CPU cores to which I replied nothing because I wasn't really ready to talk about the solution was yet the reason we need so many CPU cores is we're going to be using it for transcoding and then doing the final outputs of our videos to make our videos easier to work on on our workstations without lag as we're scrubbing through the timeline and to make our exports more seamless of the finished videos and hopefully faster question number 2 - why aren't you running Linux on a server it's a server to which my reply is well gee I guess if you may be asked what software we were using I might have been able to reply that the software we're using doesn't run on Linux and in fact there isn't a Linux version so we're going to be using Sorenson squeezed server and how this works is pretty much it's this is a server of our server we are going to be using Sorenson squeezed server and the way that works is we pretty much have a watch folder that we drop files into and it transcodes them for us so we had to do a fair bit of experimentation by we I mean Edie who's behind the camera had to do a fair bit of experimentation to find out exactly how much we could get out of our 36 CPU cores and if we did would it even do anything positive for our workflow so the exact workflow is not a hundred percent finalized yet but what we do know is that if we use in code and watch folders and we set ourselves up a watch folder and a destination when we ingest footage off of our cards which will be over Thunderbolts will actually be able to ingest very quickly even off multiple cards automatically our watch folder is going to take those and apply a particular preset to them so we can take a big whack of footage which we've actually got on the local disk here and then we are going to copy here and well this will do two things so number one it will take the footage off of our SD cards which we would cut paste instead of copy pasting and then number two now that it's safely on our storage array it will take these and transcode them to a different easier to work with format and then boot them out somewhere else so in this case we are using another local folder which loops I just pause that which may end up being a bottleneck but we'll we'll do a quick experiment before this video is over to find out if our ten gigabit network is going to benefit us here what we're waiting for now as the files copy is we've set up our watch folder to monitor itself everyone minute and then start processing so we can have a look at our current jobs nothing has happened yet but we'll know soon enough because our CPU is going to start going bananas here oh here we go here we go I don't something's happening 5% 7% CPU let's see if the job is showing up yeah there we go so you can see it's taking all these clips and importing them then transcoding them then it does something else and there's a lot of there's a lot of waiting involved but our CPU usage should start to slowly ramp up as more of them get past the importing stage and make it to the encoding stage excuse me transcoding which I guess is also encoding 76 oh it's going up to 85% oh now it's working well what the sim heck I guess it just takes a while to get all fired up or something look at it go we're actually getting a fairly reasonable turbo frequency out of this whole process like 2.7 plus gigahertz right now so there's actually a couple of different views we can get of this it looks like CPU zero is getting most of the love but we can actually change it to look at the two different Numa nodes and you can see that CPU zeros utilization is higher but it's not that much how well it is quite a bit higher so it is significantly higher than then node 1 and then we can also change the graph to overall utilization so you can see as it climbs really high CPU one kind of catches up to zero but it is it is significantly lower none of these jobs are finished they're all at the importing or transcoding stage still he's very interesting ok so we're only seeing like max highest peak 70% CPU utilization and we realized that the last time we validated any of this it was actually in this server with one of the processors plugged into the motherboard so maybe the advantage that this one has over the one we're using now is much much faster storage so if we can connect to this one over the network then we should be able to leverage that storage in theory so let's fire up the 24 SSD server and see if using that as a target helps one gigabyte per second over the network and what's interesting about being able to see network utilization here is that unlike local disk utilization we know where the hard cap is and it gives us a really clear idea of how much CPU utilization corresponds to how much network utilization you can see network is actually not even changing in spite of the fact that CPU is gone from 13 to 45 percent but what's remains to be seen is whether our CPU usage will go higher than it was able to before with less of a storage bottleneck so like so many of our videos this one ends up actually raising more questions than it answers but hopefully we were at least able to answer the question of why we wanted such a high powered server it's to enable a smoother workflow when multiple people are hitting the files at the same time and in order to have an easy quick way to replicate our data as soon as we ingest it onto our server so we can avoid the current workflow which is just leave it on the SD card just in case because there's only one copy on the server this way we'll actually have a copy on one server and a copy on another one as soon as we've ingested the footage and it has gone through this transcoding process okay so we're back at the yeah the current office for now but that doesn't mean I'm not going to take an opportunity to tell you guys about our sponsor today here and that is a tre not har in spite of the fact that these things go in your ears is a pretty unique little campaign that's up on Kickstarter right now now to be clear these are not earbuds they're not a bluetooth handset they're not any of those things what they are is they're a real-world real-time sort of audio Effects applicator so you can adjust the volume of the real world you can change EQ settings of the real world you can go ahead and you know like say for example you're at a concert and you're like you know what this could use some reverb you whip up the app on your phone and if all goes according to plan you will be able to add like a concert hall sound to that dude in the cubicle next to you if you so desire you can do more useful things with it for example like tuning out a baby on an airplane or even the engine noise on an airplane simply by making adjustments through the app there quoting a six-hour battery life and it comes with a charging case that holds an additional two charges so over the course of your day I'm sure you'll come up with a lot of interesting effects application things to do to it now you might be asking yourself okay well hold on a second what about the latency they're quoting 30 microseconds between the microphone on the outside and the speaker on the inside taking noises changing them and then blasting them into your doesn't have to be blasting because if there's a volume control and then blasting them into your ears for you to hear so if you're curious to learn more you want to support them check out the Kickstarter page which is linked in the video description thanks for coming along for the ride you guys if you disliked the video I think you know what to do but if you liked it then go ahead and press that like button helps us out a lot and get subscribed and all that good stuff if you really liked it then you can go ahead and check out the links in the video description where you can buy a t-shirt give us a monthly contribution or change your amazon bookmark to one with our affiliate code helps us out a lot and I guess that's pretty much it thanks again for watching and if you are looking for something else to watch after having done all that stuff then go ahead and click the little I in the top right corner and check out the video on channel super fun that we did where mm-hmm a certain Terran van he Mert and an electrical vehicle and some thug life involved I don't think that's how you hold it you should probably have closed yeah they ever lend you
We are a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for us to earn fees by linking to Amazon.com and affiliated sites.