Computer Storage for 3D files

[Unruly]

43 minutes ago, Gadgetman! said:

 

How old is 'old'?

 

A HDD that's over 5years old is more suspicious than a Goblin whistling innocently while standing near a paint rack. 

And anything over 3years old is on the 'not enough lifetime left to bother with' list. 

It doesn't matter if it has been left unpowered, on a shelf. The lubricants for the bearings degrades. 

 

 

Well, it's currently got two absolutely ancient(more than 20 years old) Maxtor 40gb IDE drives in it because they were my test beds when I first tried messing with it back in 2010. They still spun up when I powered it up just now, but they were audibly rough. They won't be staying in there. Most likely I'll toss a couple 10 year old HDDs into RAID 1 and see how long they last.

 

Hard drives aren't really my worry for this, though. Drives are easily sourced, and this is solely for local storage with immediate access. Anything I really want to make sure I keep goes onto my Google Drive as well as a couple other locals that are less immediate. Because nothing, not even a brand new drive, is 100%.

 

My main worry is that the capacitors on the motherboard have started to go bad. This is a board from 2005. If it goes, there's likely no replacing it. And I don't really have the money to build anything new unless I go the Raspberry Pi route. And that would be a whole new learning curve. Though, honestly, a Pi would probably spank this thing since it's quad core at just 300mhz slower per core, and it would draw a fraction of the power. Wouldn't have good RAID support, but it also wouldn't cost me a couple hundred bucks like a proper NAS would.

[KruleBear]

Got my two Seagate hard drives in, but the Synology NAS chassis is back ordered. In the meantime....

 

Started  playing with my old 3TB Seagate GoFlex  remote storage drive. I finally got the newer laptop to see it and open files from it via WiFi, but I can’t write to it. Loaded the latest GoFlex software and it crashed. Followed their online recommendation to adjust it through their GoFlex site. Yeah my old password worked. Boo, GoFlex site only works with Adobe Flash. Their link to Adobe Flash software says the discontinued Flash 12/31/20 and you should remove it from your computers for security purposes....

 

arrrrrrrggg   Well I guess I will connect to it with USB next weekend and see if I can get it to work that way. 

[TheAuldGrump]

5 hours ago, KruleBear said:

Got my two Seagate hard drives in, but the Synology NAS chassis is back ordered. In the meantime....

 

Started  playing with my old 3TB Seagate GoFlex  remote storage drive. I finally got the newer laptop to see it and open files from it via WiFi, but I can’t write to it. Loaded the latest GoFlex software and it crashed. Followed their online recommendation to adjust it through their GoFlex site. Yeah my old password worked. Boo, GoFlex site only works with Adobe Flash. Their link to Adobe Flash software says the discontinued Flash 12/31/20 and you should remove it from your computers for security purposes....

 

arrrrrrrggg   Well I guess I will connect to it with USB next weekend and see if I can get it to work that way. 

There is a Flash alternative being worked on - Ruffle.

 

Don't know how how far along the project is.

 

The Auld Grump

[TaleSpinner]

I bought into MS OneDrive.  

[Ratmaster2000]

On 1/13/2021 at 5:10 AM, TaleSpinner said:

I bought into MS OneDrive.  

how much is that providing size wise? and how much

geez, i haven't even received my 3 Kickstarters yet, just one patreon and somehow im already at 12GB of files. though to be fair, the zip files are in there as well as the unpacked files:)

[Clearman]

5GB Free

100GB = 1.99/month

 

1TB = 69.99/month.  Included Office 365

6TB = 99.99/month.  Includes Office 365 and can be shares with family (6 people)

[Gadgetman!]

100GB?   

Sounds like about half a year of .stls if you pick two or three of the more prolific Patreons....

 

 

[KruleBear]

Ended up getting a Synology 4 bay NAS DiskStation DS920+ 4 bay. Installed two Seagate 8 TB drives. I assume the best configuration to start with is mirroring?  I plan on plugging it in and setting up tomorrow.

 

I will let you know how it goes. 

[Gadgetman!]

Yeah, Mirroring is good to start with.  It's faster for write speed than the other RAID levels, too, which is practical when you first get it and starts filling it up. 

(I expect there's a fair few GB of .stls to be copied over to it as soon as it is set up. Later on, when it's the monthlies, it's not that critical.)   

[KruleBear]

2 hours ago, Gadgetman! said:

 

(I expect there's a fair few GB of .stls to be copied over to it as soon as it is set up.)   

That is an understatement !LOL 

[KruleBear]

22 hours ago, KruleBear said:

Ended up getting a Synology 4 bay NAS DiskStation DS920+ 4 bay. Installed two Seagate 8 TB drives. I assume the best configuration to start with is mirroring?  I plan on plugging it in and setting up tomorrow.

 

I will let you know how it goes. 

Set up the unit. Not too difficult, but not as easy as it could have been. Thank goodness for YouTube. Transferred a bunch of files off the PC and figured out how to down load photos off the PC. 
 

Tomorrow will start downloading my remaining stl files, which is pretty exciting. 
 

I assume that once I configure it for mirroring that it does it automatically when I load files to it. 
 

Next I need to set it up to back up all the devices in the house. 

[JoeGKushner]

I have two 8 gb hard drives, one the main, the other a back up I hit every month.

 

I agree that Patron's using Drop box or what have you is less than ideal and prefer patrons that use My Mini Factory for their delivery. Mind you some have their own stores so I'm baffled why they don't just use those platforms. Heck, Duncan uses Gumroad and you can get a year or two subscription there and it's always on Gumroad.

 

Patreon is great for it's popularity but sucks for actual delivery. All up to the individual creators. 

[Gadgetman!]

10 hours ago, KruleBear said:

 

I assume that once I configure it for mirroring that it does it automatically when I load files to it. 
 

 

The Synology should have automatically set itself up to use Mirroring. 

This all happens in the background on the NAS. 

 

Unless you started it up with jut one HDD installed, of course. 

 

Mirroring means that pairs of HDDs are set up to be exact replicas of each other. 

There's a slight penalty to write operations because of this, but not much.   

One is the 'master' and if that one fails, the other takes over without any noticeable delay. 

 

RAID 5 which is the other common method of storing data is 'striped with CRC'   

(Cyclic Redundancy Check. )

 

It'll look something like this

3 Disk setup
HDD1    HDD2    HDD3
D1      D2      CRC1
D3      CRC2    D4
CRC3    D5      D6

4 Disk setup
HDD1    HDD2    HDD3   HDD4
D1      D2      D3     CRC1
D4      D5      CRC2   D6
D7      CRC3    D8     D9
CRC4    D10     D11    D12

Each checksum block is the same size as the data blocks because it needs to contain enough information to completely rebuild a missing data block with just the remaining datablocks and the checksum.

And the reason that the checksums are spread all over instead of on one HDD is because of one of the other drives fails, it will almost always have to reconstruct missing data. Spreading it out like that means that in at least 1 out of 4 reads, it won't have to do the reconstruction.

 

There's a RAID6 also, but that just adds another HDD worth of checksums, so that you can have 2 failed HDDs at the same time and still read out your data. You don't use that on small units like this.   

 

And then there's mirrored versions of RAID5 and RAID6. This is for extremely critical always up servers. 

 

And then there's the other end of the scala...

RAID0.is just 'striped sets' 

Take a RAID5 setup and just skip the checksum blocks. 

With no calculations, and a GOOD controller, it can receive massive amounts of data quickly because blocks can be written out as qui8ckly as the individual HDDs can receive it, often all at the same time. 

It's used as a buffer for radio telescopes, as scratch and temp areas for film editing, and any other operation that produces masses of data where data recovery is less of an issue than the write speed.  

 

Synology will use mirroring if only 2 HDDs are installed, but will use RAID5(they call it something else) if 3 or more drives are present. 

If you start with 2, then later install more drives, it will automatically convert the mirror and expand it to take advantage of the additional disk space.  

UNLESS the new disk(s) are smaller than the original disks. 

(I believe the Synology will set up a new mirror if you install a pasir of smaller HDDs in sots 3 and 4), but I'm not certain how it makes it available. Don't see the reason to try it anyway... 

 

Different size disks. 

It will only use the same amount of space off of every disk, so if you start with a mismatched set, the smallest HDD will determine the amount of space you have available.  If you replace the smallest disk with a large disk, it will expand the usable array based on whichever other HDD is the smallest.  

I started with 3 x 4TB disks(=8TB usable space) 

One of the disks was failing, so I replaced it with a 12TB disk. I still have 8TB usable space. Wasting 8TB...

If I found a massive 20TB disk and placed that in my 4th slot, I would have 12TB usable space... (8 + 4) and wasted 8 + 16TB...   

I hope to replace the other two 4TB drives this summer, and as long as the HDDs I get are at least 12TB each, I'll end up with 24TB space. 

And still have room for another 12TB disk...

 

So if you have a set of small drives(maybe you ripped a pair of 1TBs out of desktop PCs or USB disks) you can still expand with larger drives and get large storage space eventually as long as you always remember to replace the smallest disk first as and when you get hold of larger drives. 

 

Final warning; if you reuse old drives; they will be reformatted and all current data deleted.  

If you want to transfer the data, it may be faster to connect it to one of the USB ports, then log into the management webpage and start a copy there.  

 

[KruleBear]

@Gadgetman! great and clear info.  Based on your prior comments I bought two new 8 TB Seagate drives. Set them up using SHR (Synology Hybrid Raid). Sounds like it is tweaked RIAD 1 that was the default, so I left it. 

 

Once I forget how much I have put into this I will get one or two more drives to increase the RAID level. 

 

I assume I could set it up to Make one drive a back up that could be removed and kept in a fire proof safe. 

 

Learning quite a bit on this. Also looks like Synology has some cool AI software that will help organize the thousands of photos my wife takes. The challenge will be if she actually uses it once I get her phone synced today. 

Edited January 17, 2021 by KruleBear

[Gadgetman!]

Oh, there's lots of cool SW for it...   

I even used it as a server when I started my home automation project.    

(I've changed to a different system now, though)