What I've got: Show
1) Is it best to use the four USB ports or find some kind of adapter for the 40-pin GPIO? 2) What do I need to make these five drives into a NAS with the Raspberry Pi?
asked Jun 18, 2015 at 23:30
 3 I realize this is an old thread but it's still high on the Google ratings so an update. I've just built a NAS using a Raspberry Pi 4+ 4GB and four external 1-TB backup drives (Seagate Basic Portable STX1000400). The Pi's power over USB is insufficient to run four spindles, so it requires a powered USB hub, but this also has the advantage of spinning all four drives on USB 3.0 ports. For setup I used RicMedia's excellent guide here: https://www.ricmedia.com/build-raspberry-pi3-raid-nas-server/. I then added WebMin for easier administration, and smartmontools (https://www.smartmontools.org/) to monitor the SMART status of each drive (note, watch the hardware compatibility on smartmontools, as not all USB drives play nice with smartctl over UAS). The array won't break any speed records, but I have copied a single 1GB file to and from a Windows 10Pro PC over ethernet in about 9 seconds each way. When backing up a huge folder the backup went at about 1GB every 30 seconds. And if I ever want to expand, there's still another free USB3 port on the Pi, so another USB hub and set of drives should be hot-mountable (haven't tried this). answered Nov 20, 2019 at 13:26
DanMDanM 1761 silver badge1 bronze badge 3 1) The GPIO is useless for this purpose. 2) Your best shot is using 5 USB to SATA converters or one that provides 5 SATA ports on the same USB port. However, both of these solutions would be pretty slow, since the Raspberry PI's USB ports are all coming from a single USB port on the SoC. This means that you'll be limited by the 480Mbps of this single USB port. To make it worse, the Ethernet port is also actually coming from this single USB port, so, if you're downloading from the PI, in theory, you'll be sharing a bandwidth of 480-100=380Mbps over the 5 drives. If all you want from a RAID setup is increased storage space and redundancy, go ahead. If speed is all you want, anything that doesn't already come wiht 5 SATA ports will be a bad idea in general for this setup. answered Dec 16, 2017 at 3:51
NunoNuno 412 bronze badges 1 Ideas for a Pi Based NAS with RAID 1Hi, At our local Model Town we have been using an old D-Link DNS-320L NAS Box to host a database and custom Python code to allow status and reading data from the various sensors and valves in the River System. Getting it to work wasn't easy and one of the volunteers spent many (happy?) hours rebuilding the D-Link code to raise the kernel version so that the chosen database worked. That was several years ago and we can foresee the time when we will be unable to maintain this box any more. What we would like to do is build a NAS Box from scratch including the hardware, so that we have full control over everything. The hardware would be my pigeon and the software would be done by the volunteer who built the original. The preference is to power this thing with a Raspberry Pi. The question is therefore; what suggestions do people have for building this item? Relating to that there are associated questions:
I have seen the post at viewtopic.php?t=327444, but that doesn't do RAID. We feel that we need RAID 1 so that we can continue to store readings in the event of a disc failure. All ideas welcome. epoch2970Posts: 8095 Joined: Thu May 05, 2016 9:33 amLocation: France Re: Ideas for a Pi Based NAS with RAID 1Sat Jul 23, 2022 10:01 am Can you quantify the data bandwidth requirement? I see hundreds of bytes per second at most, here. If bandwidth is that low, perhaps regular storage by multiple hosts over the network could answer the availability requirement? "S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel Re: Ideas for a Pi Based NAS with RAID 1Sat Jul 23, 2022 10:33 am
TBH I don't think that we too sure about that, but you are probably right. Each device generates data for anything from 1 to 6 sensors and writes the numbers to the database every 30 seconds. This might typically be the depth of water in a butt (0 to 1000 mm in 10 mm steps), the position of a Gate Valve (0 to 100%) or True or False for a pump status.
That is true, but we still want to host the database somewhere and doing it in a single NAS Box makes it more straightforward. Back in the days before we got the NAS, we used to write out the data as strings to local files and then back those up to another host. The trouble with that is that it is difficult to manage whereas having a database in a remote device makes it easy to drill down into the data and call for graphs and reports. In addition, the existing NAS Box has now been fully operational for nearly 2 years and the code is now quite mature. If we did away with the NAS Box we would have to redevelop the code. Re: Ideas for a Pi Based NAS with RAID 1Sat Jul 23, 2022 11:24 am TL;DR: for a low bandwidth application, a 3B with software RAID1 is likey to be up to the job but you will need USB to SATA adapters and external power to the drives. Whether a 3B is upto the task depends on whether USB 2 provides enough bandwidth for your data bearing in mind that it will be going over the USB bus at least twice: once from ethernet to SoC and once from SoC to drive. THree times if using software RAID1. The only sensible way to add bulk storage to any PI except the CM4 is via SATA to USB adapters. If you have to purcahse those I'd recommend spending a bit more and buying a self powered two (or more) bay enclosure that has internal support for RAID1. Power availability on Pi is such that in most cases only a single 2.5" drive can be powered directly, Additional 2.5" drives will require either a self powered USB hub or a self powered enclosure. And there are potential boot issues with self powered USB 3 devices on a 4B - they can prevent boot. 3.5" drives will require their own PSU as the Pi cannot provide the 12v they require (CM4 with CM4IO excepted). With Linux software RAID, you cannot boot a Pi from the RAID array as the bootloader doesn't support it. It should be possible to put the root partition on the RAID array but the extra configuration needed (basically an initrd) adds complexity and, if you are not careful, breaks when the kernel is updated. A CM4 gives you the possibility to add a PCIe SATA card but that comes at a monetary cost and at the cost of not having USB 3 (when compared to a 4B). FWIW, my current NAS is:
TBH, given the availability issues you'd probably be better off using an old PC with Debian linux installed. Less need for external power and adapters too. Lastly, and appologies if you are aware of this, RAID1 is not a substitute for backups. It'll keep things running in the event of a drive failure but doesn't help with other causes of data loss. Knowledge, skills, & experience have value. If you expect to profit from someone's you should expect to pay for them. All advice given is based on my experience. it worked for me, it may not work for you. Re: Ideas for a Pi Based NAS with RAID 1Sat Jul 23, 2022 11:40 am
A good point.
Do you mean USB to SATA? There is no SATA on a Pi and so I was thinking of going that way.
I understand that. I'm used to producing 5 V DC from 12 V for the other Pi projects on this site, so I would expect to use the 12 V (after filtering for HDDs (if that's what we end up using).
I understand that too. We would still use the SD Card for the OS and Database software. The discs would simply be for data,
We have been looking at that, but largely dismissing it because we think it is way over the top for what we need and for our budget.
We have considered that too, but it would be nice to do it with an RPi.
Yes. We do do proper backups. The RAID is there so that we don't lose data. Re: Ideas for a Pi Based NAS with RAID 1Sat Jul 23, 2022 12:04 pm
Yep. And that's as good a reason as any. Like I said and if your budget allows, I'd go for a USB HDD enclosure that does the RAID1 internally. The Pi will see a single drive and the enclsoure takes care of the mirroring so no extra transfers on the USB bus. A further though occurs: if you have or can get hold of a 4B this may be worth investigating. Not a recommendation as I've no experience of it or their other products. Knowledge, skills, & experience have value. If you expect to profit from someone's you should expect to pay for them. All advice given is based on my experience. it worked for me, it may not work for you. Re: Ideas for a Pi Based NAS with RAID 1Sat Jul 23, 2022 1:26 pm I have a Pi that is a NAS, has a number of databases - one of them storing data from a number of temperature/humidity/pressure sensors and system temperatures of 12 Pis - readings taken every 5 minutes. It also has some local websites and other applications. Until recently that was running on a Pi 2B with a 2TB USB HDD which handled the load easily. Then I reorganised things and split the databases and most of the websites off to a Pi 4B 4GB and upgraded the original to a Pi 4B 2GB. Unreadable squiggle ejolsonPosts: 10047Joined: Tue Mar 18, 2014 11:47 am Re: Ideas for a Pi Based NAS with RAID 1Sat Jul 23, 2022 3:38 pm
The SOC used in your present NAS is based on an ARMv5TE core with two PCIe (x1) channels, two GbE (I think only one is available on the DNS-320L) and a USB2.0 port. While the RAM is much less than a Pi, it's not until the Raspberry Pi 4B that Ethernet at gigabit speeds was available and there is still no Pi with two PCIe. In practice this means anything older than a Pi 4B is likely to be slower than the NAS you have. Given the amount of third-party firmware available for the DNS-320L, it's also not clear that an older Pi would be more maintainable. My recommendation is not to replace the existing NAS until you can get at least a 4GB Pi 4B as the replacement. Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 6:27 am
Well. Yes, No, Maybe. Our network does not support Gigabit Ethernet, so older Pis would suffice there. However, I hadn't appreciated that the DNS-320L had PCIe channels, mainly because we have to use SATA Drives in the Bays. Presumably that limits the performance of the drives, but not as much as USB would. I also assume that the DNS-320L has a hardware RAID Controller, so that would also be an issue if a Pi was used. On the plus side; the RPi 3+ has a quad core processor, which presumably would help with the custom code that we currently run on the DNS-320L. Either way, the biggest problem would seem to be that the only way to access RPis earlier than the 4, is via USB, which would limit drive performance significantly as noted by thagrol above. I wasn't aware that third-party firmware existed for the DNS-320L. That might be a way forward, depending on its performance. Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 9:00 am
You said your network doesn't support Gigabit. If so, that is likely to be the bottleneck. A Pi 3B+ should be capable of pretty much handling close to 100Mb/s throughput from any USB attached drives to the ethernet port. USB2 is 480Mb/s - shared between the four USB ports and the GBit Ethernet. Unreadable squiggle Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 10:43 am
I see your logic. However, I don't think that the network will be the bottleneck in our case, since the data throughput is actually so low. More likely it will be the RAID transfers, as pointed out by by thagrol above. Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 11:25 am
The bottleneck is going to be the lowest of the rate of incoming data, network speed, transfer/write speed to drives. If you're not saturating the network bandwidth the drives will likely be able to keep up even in a software RAID. Even if you are, you should still have roughly 150Mbps bandwith to each drive over USB 2. So, yes, there is a performance hit with software RAID1 but with your network and the incoming data rates you describe you probably won't notice it. And regarding the 4B, without major surgery*, the only way to connect drives is still over USB. The advantage of the 4B is that it has USB 3, gigabit ethernet (as does the 3B+, kinda), and that the ethernet is not connected via USB. *: removing the USB controller and hand soldering on wires to a PCIe connector or bridging to one of the USB 3 ports and using a riser card. Knowledge, skills, & experience have value. If you expect to profit from someone's you should expect to pay for them. All advice given is based on my experience. it worked for me, it may not work for you. Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 11:35 am
So. To be clear, this could be done with a Pi3 without too much of a data hit?
We don't have either a 3B+ or a 4 to hand so we would have to buy one. If we went down that route, I would plump for the 4. (Strictly speaking we have a Pi 3B+, but it belongs to my wife and has already been co-opted to provide a reference system for testing the VPN Server.)
Probably a bit too techy for me. I'll stick to using an RPi 4. Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 11:54 am
Given my understanding of what you have described, yes.
And for me. Not to mention my eyesight and dexterity.. That's why I'm using a CM4. If you have a 3B available, I'd start with that and use the latest OS and USB 3 adapters/enclosure (avoid Jmicron chipsets thoguh). That way you can get started now and upgrade to a 4B if necessary when the current availability problems have been resolved. Knowledge, skills, & experience have value. If you expect to profit from someone's you should expect to pay for them. All advice given is based on my experience. it worked for me, it may not work for you. Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 11:59 am
It's a thought. That would involve the VPN Reference hardware 'lending' the 3B for the NAS Box. ejolsonPosts: 10047Joined: Tue Mar 18, 2014 11:47 am Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 1:23 pm
Any model Pi before the 4B is not ideal for Linux software RAID because the total bandwidth between the network and two disks is shared through a single USB port. NAS software, such as Open Media Vault will detect that you have such a Pi and refused to create the array. You can do it by hand, but the problem is the resilver will take a long time. Here is my calculation: There is a total of 480 megabits per second bandwidth with a single USB2 port. Not all of this can be used. Maybe about 40 MB/sec total, for example, is possible. In a resilver the data from one disk is read and copied to the other. Thus, resilvering a two disk 2TB array involves 4TB of input and output, or in other words 4000000 MB. Since, 4000000/40=400000 seconds optimistically it will take about 4.6 days to resilver the array. Simultaneously logging data to the array in degraded mode might extend the resilver time to weeks. Has anyone measured NAS performance of a Pi 3B during a RAID resilver? While performance on a Pi 4B will also suffer when synchronising a RAID, the 4B is much faster to begin with. Another advantage of the 4B is the newer model is currently manufactured in greater numbers and likely to be supported for longer into the future. Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 3:52 pm I confess that was not something I had taken into account in my advice. Oops. Sorry. This is the other reason I'd go with a drive enclosure that does the RAID1 internally. A rebuild will still take time but shoudln't involve sending the data across the USB bus twice. Knowledge, skills, & experience have value. If you expect to profit from someone's you should expect to pay for them. All advice given is based on my experience. it worked for me, it may not work for you. blsPosts: 2642Joined: Mon Oct 22, 2018 11:25 pmLocation: Seattle, WA Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 5:01 pm
Based on the 1-star reviews, the answer to your wondering is: NO. But, there are a LOT more choices available than there were a couple of years ago when I bought my 4-disk JBOD array, so seems likely that there's one that's reasonably priced with acceptable performance and operational characteristics (and lots of 5-star reviews!). Pi tools: ejolsonPosts: 10047Joined: Tue Mar 18, 2014 11:47 am Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 5:40 pm
I was also not sure what to make from the reviews but encouraged by
My impression was the main complaint people had was the noisy fan. This is actually the only under 100 $US dual-bay RAID enclosure I could find. All the others were simple JBOD enclosures or much more expensive. Note the goal here is hardware RAID. blsPosts: 2642Joined: Mon Oct 22, 2018 11:25 pmLocation: Seattle, WA Re: Ideas for a Pi Based NAS with RAID 1Sun Jul 24, 2022 5:45 pm
I only got as far as the percentage of 1-star reviews Great that it uses ASMedia instead of the "J guys", and fans...yep, always a risk. If it's good enough* it could be a reasonable choice. Half the reviews were quite positive. * Highly subjective call plus a dash of "are you feeling lucky?" Pi tools: Re: Ideas for a Pi Based NAS with RAID 1Mon Jul 25, 2022 5:34 am Thanks for the insights. This is all food for thought and we'll be discussing this with a team meeting shortly. We may just wait until the chip situation improves and/or a suitable enclosure falls into our lap (via a donation from someone who has an old one for example). Return to “Networking and servers” |
라즈베리 파이 NAS RAID - lajeubeli pai NAS RAID
저작권 © 2024 toptenid.com Inc.
