Gadgetory


All Cool Mind-blowing Gadgets You Love in One Place

100TB at over 1GB/s - The "Storinator" is back!

2015-12-16
way back before the big move I built a 100-plus terabyte storage server to replace the awful store data on disconnected drives on a shelf in the bathroom system that we were rocking before thank Seagate for the awesome drives and thanks 45 drives comm for the rock and personalized tornado server but some of you may have noticed that I never followed up on the performance testing that I promised to do on that machine I was supposed to be showing off one gigabyte per second transfers with the 10 gigabit network set up what gives well today we finally get the whole story the mastercase 5 by Coolermaster gives you the freedom to truly make your mid-tower pc case your own with a variety of modular parts and accessories check out the link in the video description to learn more so the short version is this in spite of 45 drives telling me that they had customers with similar config saturating a 10 gigabit link or more I couldn't even get half of that and it made no sense really I did a lot of tinkering with this box before eventually deploying it different network cards different drive configurations and finally got to the point where it was whether freenas hardware or petcock I had to roll it out because we needed to put our data somewhere and I was just gonna have to live with the results that I got I mean I know I know poor Linus only has 300 to 350 megabyte per second speeds to is over 100 terabytes of safe storage boohoo but this disrupted my plans for our storage infrastructure in a bigger way than you might think in addition to archiving old stuff to the server my intention was to have our daily use Ness the SSD one that you probably remember from this video doing nightly syncs or even hourly checkpoints if we could get away with it so we'd have two full copies of all of our mission-critical data so I wanted the magnetic Nazz to be fast enough to handle that and any random data that our editors needed to read from it from old projects which we were not able to do so while I've had four months to diagnose this and ponder what could be wrong because it's had up to 60 terabytes of important data on it with nowhere else to offload that I've had no choice but to just limp along at 300 to 350 megabytes per second until today seagate sent us 35 of their new 8 terabyte enterprise capacity drives and no these are not the shingled platter archival ones these are rock and ass capable of well in excess of 200 megabytes per second transfer speeds rated at 2 million hours mean time between failure and with a five-year warranty to back it up proper enterprise capacity drives so I immediately tore them out of their packaging and began building pyramids that no just kidding ok I did build pyramids but but when I actually built with them after the pyramids was 2 additional servers each to hold a copy of our 60 terabytes of data while I worked on the vault so one of those machines is actually eventually going to be an AZ unit at my house and the other one is going to be an off-site backup server for this puppy but each of those will get their own videos later so with the data safely stored on a hardware raid 6 and on a software butter FS raid 5 each of those transfers took over a day by the way I wiped the freeness and began trying things so first I tried 6 drive v des since that's a more optimal number for ZFS - nope still shoddy transfer speeds next I tried 10 Drive V dads no difference again finally in desperation I tried a 27 drive raid 0 an experimental class configuration that no one should trust to hold any data no matter how amazing the drives are and same thing which after talking to the folks at 45 drives about my findings revealed that the issue is probably a software one because they've seen NFS shares just fly in a similar configuration to mine which doesn't do me any good because this is a Windows environment and we need SMB shares and so I had to keep investigating because if I'm going to be running around saying this nas unit in these drives are capable with over a gigabyte per second of transfer speed we use them here at Linus Media Group I mean I'm basically endorsing the things it's not good enough to me 445 drives to see it in their lab I need to see it so I've been chatting a lot with the unread guys ever since they helped us do the to gamers one CPU project which you should definitely check out if you haven't already and they offered to spend some time configuring an experimental raid five butter FS array in on raid and tuning both the network settings as well as the SMB share settings so while we're initial tests on a vanilla on raid server was frankly pretty ho-hum actually fairly similar there's that poor SMB optimization outside of Windows platforms rearing its ugly head again some four kilobyte packet and jumbo frame tuning to the network card tuning of on raids networking configuration and boom that my friends is the cleanest ten gigabit transfer that I've actually ever seen now a lot of lime Tech's customers are running ten Giggy but from their perspective I guess it's just valuable R&D for down the road when that gear becomes more common but I mean even then this is not the kind of config that most people will encounter even on unrated I actually don't intend to continue to run it like this but RFS raid 5 and raid 6 are both in the experimental stage but the good news here is that what I realized after running the slow FreeNAS configuration for so long was that generally speaking I don't need more than the 200 to 220 megabyte per second transfer speeds that my individual drives are capable of in a normal unread array and that the only thing that needs to be lightning-fast performance wise is the new footage and projects that we offload to it relatively little of which is created on a daily basis so we devised a new plan and to help us realize the new plan Kingston stepped up and offered to send us eight of their a 50 enterprise-grade 480 gig SSDs with power loss protection these drives will act as a 2 terabyte raid 10 right cache that will be capable of the full 10 gigabit transfer rate for fast up dates throughout the day and that then flushes nightly to the hard drives when no one is using them all of this can be completely transparent to the user so the only time we'll ever see sub one gigabyte per second transfers is when we're accessing cold data we're when doing a massive dump of over two terabytes at a time another cool side note is that this might turn out to be a better way to leverage the extra horsepower that this OPC server is leaving on the table anyway because she never touches more than about 20% CPU usage so I could take a couple of cores and turn them into a network rendering box or game server or something else and on the subject then of our server being OPI I guess that brings us to the conclusion it turns out that the hardware is but SMB shares on non Windows platforms takes some tuning and optimization that if you're willing to endure the dense documentation and condescending attitude of the FreeNAS community you could probably achieve there but instead I ended up working directly with lime tech to have baked into an upcoming release of unmade six and I'm super happy with the new config if you're building a mobile app and searching for a simple payment solution you might want to check out Braintree with the Braintree V zero SDK which is just one small snippet of code you can be all set up and ready in less than 10 minutes to take online payments and if you're having any trouble they have support staff ready to walk you through the process over the phone if you need them their code supports Android iOS and JavaScript clients and they have SDKs in seven programming languages and it makes it easy to offer multiple mobile payment types including PayPal Apple pay Bitcoin venmo cards and more all with a simple integration they've got quick knowledgeable developer support if you have any questions and to learn more all you got to do is go over to Braintree payments comm slash Linus and if you use that link you can also get your first $50,000 in transactions with no fees whatsoever so check it out today at the link in the video description so thanks for watching guys if this video sucked you know what to do but if it was awesome get subscribed hit the like button like button hit the like button or even consider supporting us directly by using our affiliate code to shop at Amazon instructions for which were up here buying a cool shirt like this one or with a direct monthly contribution through our form it gets you a little contributor tag now that you're done doing all that stuff you're probably wondering what to watch next so click that little button in the top right corner to check out the ultimate showdown between AMD's eight core cpu and intel's 8 core server anyway it's complicated you guys should check it out the video
We are a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for us to earn fees by linking to Amazon.com and affiliated sites.