Gadgetory

as many of you probably know the last few years have been a heck of a ride for data enthusiasts with high-speed storage becoming increasingly accessible I mean something like Samsung's 960 Pro SSD it's tiny it costs about a third of what I paid per gigabyte for my first large boot SSD and it's over an order of a magnitude faster when it comes to real-world performance but what some of you might not know is that networking has been keeping pace so these bad boys right here are Mellanox connect decks for cards and even though they're already two generations old they can reach transfer speeds of 100 gigabit or about 12 gigabytes per second that is fast enough to download fortnight's install files yes we went there in literally one second holy shit so let's check them out but before that check out the silent base 601 from be quiet it delivers a great combination of maximum silence and usability check it out at the link below before we get to the new crazy stuff let's do a quick refresher on traditional networking hardware so I've brought along a couple of examples here these two cards run it gigabit and ten gigabit speeds respectively so this guy right here is about ten times faster than this one but other than that they've got a lot in common so they've both got Intel controllers on board they both plug into a PCI Express 2.0 slot and they're both Ethernet so thanks to their use of inter compatible communication standards and the ubiquitous rj45 connector they can talk to each other directly or through a network switch like this one I'll be it only at the speed of the slowest link in the chain be it on this one gig card or through this one gig network switch and honestly either of these especially this one should be more than enough for the average person for quite some time but this isn't average person land which sounds like the world's most boring amusement park so we've decided to go totally overkill and take it to the next step to greater than ten gigabit speeds which brings us then back to our connect X fours so the first difference the ports this beefy looking thing right here is what's called a queue SFP+ connector and as you've probably figured out on your own you can't just plug a standard network cable into this port and even if you could well I guess that brings us to the next difference the fact that out of the box these cards are designed to run not on Ethernet networks but on InfiniBand networks so even if you could plug it into your network switch it wouldn't be able to communicate with it without some configuration and then finally this one's actually pretty interesting these cards yes my friends these network cards use a full fat piece see I Express gen3 times 16 connection that is the same as your graphics card and they actually need it probably more so than your GPU if all you're doing is gaming so then with that in mind we will not be using just your average gaming machine for our testing so on one side of our link we've got Intel's flagship 18 core processor with a rampage 6 extreme motherboard a hundred and twenty eight gigs of ram yeah and then in the other corner we have to slum it a little with a 16 core 79 60 X and an Asus x2 99 deluxe but then with the same amount of RAM and of course its RGB across the board now the reason that we're using the X 2 99 platform with core I nine processors is that we need to make sure that we have enough PCI Express Lanes coming directly off the CPU so 44 should give us enough for 16 lanes for networking 16 lanes for our quad nvme storage devices and then you know some leftovers for the graphics card so then for our nvme storage we scraped together for samsung 960 pros for our first one and then for corsair MP 500s for the other with both of them running with quad SSDs in raid 0 today's video is about how to go fast not about how to put on your seat belt yeah this is like hilarious to me I never thought I would see the day when the GPU in a gaming rig is the lowest priority like tier PCI Express device sitting at the bottom of the board anyway for our OS we had to go with Windows Server 2016 because as much as we wanted to try out windows 10 pro 4 workstation which is supposed to support our DMA the tech that allows for these super high speed transfers it just didn't seem to be working for us for some reason alright now at this point before we go further I want to give a big shout out to the guys over at Mellanox for hooking us up with these hundred gigabit cards as well as a pair of 100 gigabit capable passive copper direct-attached cables wow that just hit the bench oh so if you want to try this out at home these cards are actually available on eBay for like two three hundred bucks a pop and then you'll pay about sixty dollars for a three meter cable like this one it is worth mentioning though that if you're planning on running anything further than five meters you have to use an active fiber cable which could cost upwards of twenty-five hundred dollars new so no all right moving on configuration then is our last step so what Jake does that behind me here thank you Jake let's talk about some of the technology behind this networking magic so these cards are designed for use with two different network fabrics InfiniBand and Ethernet and what makes InfiniBand special is that compared to even the sub-millisecond latency of a typical Ethernet network InfiniBand networks can have less than 25 percent as much making them suitable for use cases like over network storage access and combining the processing power of multiple servers just like in a data center or supercomputer and this is cool when you configure InfiniBand correctly it also forms what's called a lossless Network meaning that packet loss should basically never happen for compatibility though we're going to be using them in Ethernet mode alongside a technology called our DMA or remote direct memory access when you put these together the set up is called our OCE or our DMA over converged Ethernet now regular Ethernet implementations require a lot of hoop jumping in order to transfer data as any information sent must be first moved through the transport protocols driver then through sockets before it can reach the applications memory eating up CPU cycles and increasing latency in the process however with our DMA the network adapters are able to access data directly from application memory offloading much of the processing from your CPU onto the actual processor that sits right on your network adapter so these are known as copy transfers and they allow for incredibly fast transfers that are no longer limited by CPU processing power pretty dang snazzy so are we ready to go we should be each card has an IP and a 50 gigabyte RAM disk so we should be able to do some pretty quick windows transfers so are we going directly from RAM disk around just right now we've gotta see make sure it's working first right that would be good I mean that's supposed to be your job fingers trust me never the second you try to do a networking demo like we got our 40 gigabyte txt file that's 36 or 37 K quit why there's why even it's just a big file okay oh that's not bad that's not 10 gig or 800 gig so we've got just over 2 gigabytes per second so we're reading from that systems nvme array okay and then we're dumping to the RAM disk of the system so that's right doing the other way so now we can go Ram disk to Ram disk it's alright no that's the same thing but to say other way around whoa-ohh just shy of 4 gigabytes per second I mean let's put what just happened there in context that is a 40 we'll just shy a 40 gigabyte file so like okay doom-doom on PC is like 60 gigs and it takes you know probably for your internet connection at home like what an hour to download and install that just happened in real time okay what's next Ram disk Aram disk are you gonna check I think whatever Ram this isn't working ok so check this out we've got the RAM disks working but for now this is just another quick benchmark so when we aren't limited by the overhead of Windows File Explorer in Windows File transfers if we're just using a straight disc performance benchmark I want to show you the kinds of numbers we're looking at so anyone who's familiar with a dough is gonna already know that this is freaking nuts at two kilobyte sizes we are already seeing speeds in excess of 100 megabytes per second because we are actually reading and writing off of the for Corsair SSD array that's on the other machine this is stupid it's pretty cool actually it's really cool and it's getting stupider as time goes on we are already hitting 600 megabytes a second writes that's weak at 16 kilobyte stove just wait that's that's the key and it just keeps getting crazier we're at 1 gigabyte per second already 3 doubled 3 gigabytes a second writes oh wow oh man we just cracked 4 gigs a second we just cried 5 gigs a second and this is on a remote machine this is not a local array this is over the network this is a slow test yeah it's a really fast slow test yeah it takes a little slow fast test in the billions of bytes billions of bytes it's like not even readable anymore at that point no just like what is going on how do my maths Oh meanwhile I'll be there just cracked it 10 gigabytes a second okay so what's our next test to ramdisk yes sure let's do it so what's interesting here is that we actually hit our peak speed earlier but then we level off to the ramdisk yeah but our rights are like right so a better and if we look at CPU usage it will show in here processor time was almost zero of the whole time crazy it's like if you're using traditional Ethernet without our DMA that would be like pretty high pegged up there so why don't we do as fast a windows transfer as we can we'll go straight to the RAM disk sure doesn't take long term forty gigabyte files are like this even on like a relatively slow the relatively slow oh two gigabytes a second how will we ever manage painful yeah so about the same yeah so the windows thing pretty much what's going on here is that we have reached because we've seen faster transfer speeds in a dough so we have reached pretty much the limit of what Windows can handle for the time being and I don't think it's gonna be in Microsoft's list of high priorities anytime soon to figure out how people can use Windows Explorer to copy files it faster than about four gigabytes a second but that's okay because this was a lot of fun and hopefully you guys enjoyed it if you want to see what this kind of tech gets used for in the real world because this is just not not what it's really for check out our kind of unboxing of SF use cedar data center we'll have that linked in the description because that's this kind of technology on a whole other level with with that set I mean even for someone like us actually it could be useful maybe we could try to rig up like a crazy way to improve the responsiveness of scrubbing in Adobe Premiere for the editors or something 100 and dig a bit for all the editors yeah 100 gate well no but we could have like a hundred gigabit trunk yeah and then they can gig it's it's bonded ten gates Oh like so you know what tell you what let us know in the comments if you'd like to see a video on what we can figure out to do with these in an actual deployment so thanks for watching guys if you enjoy our videos on networking you should probably check out our sponsor private Internet access VPN private Internet access hides your true IP address and allows you to bypass your restrictions and censorship by making you appear as though you are connecting from somewhere else and you can use it on 5 devices at once it also helps prevent attacks by blocking unwanted connections it Auto blocks all traffic if the VPN disconnects and keeps your data out of the hands of advertisers and other activity tracking snoops PII supports multiple VPN protocols and has apps for Windows Mac Android iOS Linux and a Chrome extension and they've got over 3,000 servers in 28 countries and they don't log user activities so what are you waiting for check them out today at the link below so thanks again for watching guys if you dislike this video hit that button but if you liked it hit like get subscribed or maybe consider checking out where to buy the stuff we featured at the link below probably eBay links I guess while you're down there you can check out our merch store which has cool shirts like I don't know how long this one's gonna last but it's gonna be up there as of the time of shooting this pretty soon and I think it's funny and also we have a link to our community forum which you should totally join that shirt I know right