WD 3TB EFRX longevity: replacement log

[OneCD]

I've just replaced the first of the 5 WD 3TB drives in my old TS-569 Pro running Debian, and thought I'd document some numbers for posterity. I'll add to this topic as the remaining WD drives are replaced. This might be of interest to other EFRX owners.

I bought these drives about 10 years ago, and they've run almost continuously the whole time (I never went down the "HDD standby" rabbithole).

Today's drive is a WD30EFRX-68AX9N0 model, manufactured 27th November 2012 in Malaysia. Total power-on hours at the time of replacement were 85,723.

The reason for the replacement? I got an email from the NAS (running Debian) via 'smartd' with "CurrentPendingSector" in the title yesterday, which occurred during the monthly RAID 5 scrub. Yesterday there were 2 unreadable (pending) sectors. This morning, there are 4. Time for a new drive.

I've now replaced the old drive with one of the newer version EFRX drives (the ones that make more noise ) that I bought to use while developing software in my TS-230. There were no problems during the changeover, and the rebuild is happening now:

Code: Select all

$ cat /proc/mdstat 
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10] 
md0 : active raid5 sdd1[6] sda1[3] sdb1[5] sdc1[4] sde1[1]
      11720536064 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [UU_UU]
      [>....................]  recovery =  1.1% (34434804/2930134016) finish=2311.3min speed=20879K/sec
      bitmap: 17/22 pages [68KB], 65536KB chunk

unused devices: <none>

Then connected the old drive to my desktop PC and captured the SMART info below:

Code: Select all

$ sudo smartctl --all /dev/sda
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.10.0-20-amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68AX9N0
Serial Number:    WD-WMC1T1500475
LU WWN Device Id: 5 0014ee 6ada798d9
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jan  4 12:02:29 2023 AEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (38880) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 390) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x70bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       807
  3 Spin_Up_Time            0x0027   171   170   021    Pre-fail  Always       -       6425
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       397
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   001   001   000    Old_age   Always       -       85723
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       366
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       74
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       322
194 Temperature_Celsius     0x0022   122   101   000    Old_age   Always       -       28
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       4
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   100   253   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     20109         -
# 2  Short offline       Completed without error       00%     19941         -
# 3  Short offline       Completed without error       00%     19773         -
# 4  Short offline       Completed without error       00%     19605         -
# 5  Short offline       Completed without error       00%     19437         -
# 6  Short offline       Completed without error       00%     19270         -
# 7  Short offline       Completed without error       00%     19102         -
# 8  Short offline       Completed without error       00%     18934         -
# 9  Short offline       Completed without error       00%     18766         -
#10  Short offline       Completed without error       00%     18598         -
#11  Short offline       Completed without error       00%     18431         -
#12  Short offline       Completed without error       00%     18263         -
#13  Short offline       Completed without error       00%     18095         -
#14  Short offline       Completed without error       00%     17927         -
#15  Short offline       Completed without error       00%     17759         -
#16  Short offline       Completed without error       00%     17591         -
#17  Short offline       Completed without error       00%     17424         -
#18  Short offline       Completed without error       00%     17256         -
#19  Short offline       Completed without error       00%     17088         -
#20  Short offline       Completed without error       00%     16920         -
#21  Short offline       Completed without error       00%     16752         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[OneCD]

Just replaced the 2nd of 5 drives.

Today's drive is a WD30EFRX-68AX9N0 model, manufactured 27th November 2012 in Malaysia. Total power-on hours at the time of replacement were 87,660.

First notification was an email from Debian on the NAS:

Code: Select all

This is an automatically generated mail message from mdadm
running on talia

A Fail event had been detected on md device /dev/md/0.

It could be related to component device /dev/sde1.

Faithfully yours, etc.

P.S. The /proc/mdstat file currently contains the following:

Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
md0 : active raid5 sdd1[4] sdc1[6] sde1[1](F) sda1[3] sdb1[5]
      11720536064 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [U_UUU]
      bitmap: 4/22 pages [16KB], 65536KB chunk

unused devices: <none>

I was unable to check any SMART info while in the NAS as 'smartctl' couldn't see it.

Pulled it from NAS and connected it to a PC. SMART shows all OK - no errors. Weird.

Code: Select all

$ sudo smartctl --all /dev/sda
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.10.0-20-amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68AX9N0
Serial Number:    WD-WMC1T1381695
LU WWN Device Id: 5 0014ee 602fcfe99
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sun Mar 26 06:02:07 2023 AEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (40320) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 404) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x70bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   196   173   021    Pre-fail  Always       -       5175
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       402
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   001   001   000    Old_age   Always       -       87660
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       371
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       94
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       307
194 Temperature_Celsius     0x0022   118   103   000    Old_age   Always       -       32
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   100   253   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     21955         -
# 2  Short offline       Completed without error       00%     21788         -
# 3  Short offline       Completed without error       00%     21620         -
# 4  Short offline       Completed without error       00%     21452         -
# 5  Short offline       Completed without error       00%     21284         -
# 6  Short offline       Completed without error       00%     21116         -
# 7  Short offline       Completed without error       00%     20948         -
# 8  Short offline       Completed without error       00%     20781         -
# 9  Short offline       Completed without error       00%     20613         -
#10  Short offline       Completed without error       00%     20445         -
#11  Short offline       Completed without error       00%     20277         -
#12  Short offline       Completed without error       00%     20109         -
#13  Short offline       Completed without error       00%     19942         -
#14  Short offline       Completed without error       00%     19774         -
#15  Short offline       Completed without error       00%     19606         -
#16  Short offline       Completed without error       00%     19438         -
#17  Short offline       Completed without error       00%     19270         -
#18  Short offline       Completed without error       00%     19102         -
#19  Short offline       Completed without error       00%     18935         -
#20  Short offline       Completed without error       00%     18767         -
#21  Short offline       Completed without error       00%     18599         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

As this is the first blip with this drive, and it's at the end of its life, time for a new one. I'll keep it for testing and development, but its production days are over.

I'm unable to buy another 3TB WD EFRX locally (it seems they may not be available anymore), so I purchased a 4TB WD EFPX instead. Works fine so-far, nicely quiet, and as it turns-out has lower power specs than the 3TB drive it replaced.

3TB:

• 5VDC : 0.60A
• 12VDC : 0.45A

4TB:

• 5VDC : 0.38A
• 12VDC : 0.28A

Rebuild time for RAID 5 is estimated to be another 40 hours (which is about right for this NAS).

[Toxic17]

Thanks for the heads up, running the same drives on my 673A.

[OneCD]

Thanks for the heads up, running the same drives on my 673A.

No worries Simon. You’re welcome to add end-of-life details for your drives (if available) if you want-to, as-is anyone else.

[Toxic17]

Thanks for the heads up, running the same drives on my 673A.

No worries Simon. You’re welcome to add end-of-life details for your drives (if available) if you want-to, as-is anyone else.

so I scan SMART tests rapid (weekly) and complete (monthly) on my drives, is there a way of getting all the scan results in one file? there must be a file somewhere that the schedules are written too?

[P3R]

Adding my history to the community log.

Bought 2*WD30EFRX 2012-08-10 (manufactured 2012-07-01).
One of them had pending sectors after about 2.5 years and luckily had it replaced on warranty (manufactured 2014-11-04).
Bought one more 2015-02-12 (manufactured 2015-01-05).

Mine haven't been running as many hours as they have been moved around between systems and spent several years with spin-down enabled, in a stripped down system intended for backups only and running an old QTS so spin-down actually works. The power on hours are between 68644-51527 hours for the three drives and currently only the oldest one is in use with the other two waiting to replace even older drives as they fail.

I'm running daily rapid SMART tests, complete weekly tests and scrub the RAID monthly.

[OneCD]

so I scan SMART tests rapid (weekly) and complete (monthly) on my drives, is there a way of getting all the scan results in one file? there must be a file somewhere that the schedules are written too?

Sorry, I don't know.

[OneCD]

I'm running daily rapid SMART tests, complete weekly tests and scrub the RAID monthly.

I should have thought to include the SMART testing schedule in my previous posts.

I run short SMART tests just before the weekly backup executes, and have never run long tests.

[P3R]

I run short SMART tests just before the weekly backup executes, and have never run long tests.

I have had two disks that passed the rapid test but failed the complete. So if I have to choose, I'd say that the long test is more useful and important to run.

[OneCD]

I have had two disks that passed the rapid test but failed the complete. So if I have to choose, I'd say that the long test is more useful and important to run.

I must confess: I'm uncertain as-to what is checked during these tests.

Does running a long (or short) test cause any wear on the drive?

[dolbyman]

This is what wikipedia says

https://en.wikipedia.org/wiki/Self-Moni ... Self-tests

Short
Checks the electrical and mechanical performance as well as the read performance of the disk. Electrical tests might include a test of buffer RAM, a read/write circuitry test, or a test of the read/write head elements. Mechanical test includes seeking and servo on data tracks. Scans small parts of the drive's surface (area is vendor-specific and there is a time limit on the test). Checks the list of pending sectors that may have read errors, and it usually takes under two minutes.
Long/extended
A longer and more thorough version of the short self-test, scanning the entire disk surface with no time limit. This test usually takes several hours, depending on the read/write speed of the drive and its size.
[...]

I think the longer test might not 'really' hurt the disk but the surface test might discover issues that were undiscovered before (so one could think it 'hurts' the disk)

[OneCD]

I think the longer test might not 'really' hurt the disk but the surface test might discover issues that were undiscovered before (so one could think it 'hurts' the disk)

Thanks @dm.

That surface scan may be the reason I never ran a long test. It seems like one of those operations that would let you test-and-test a drive into a "permanent failure mode".

[P3R]

It seems like one of those operations that would let you test-and-test a drive into a "permanent failure mode".

The long test won't cause more wear than reading from the disk would. It's ancedotal of course but I've been running this same SMART test schedule for 15 years or so and healthy drives don't fail prematurely from it. My currently longest running drive have 100225 power on hours and several have around 80k.

In my world it's always better to know about problems than not look for them and hope they don't exist. Therefore it's in my opinion best to use all the tools we have in the tool box. Especially if all disks are the same model, the same age and have been having the same life. Multiple concurrent disk failures are naturally more probable then than with an assortment of different history disks used together in a RAID array.

A pending sector (ID 197) only tell us that the sector is questionable but we don't know if that is a false positive or if it actually is faulty, we don't know if the sector is possible to remap (then it become an ID 5) or uncorrectable (ID 198).

Low ID 5 numbers isn't really a problem for an active administrator like you that keep track of pending and remapped. ID 198 above zero on the other hand mean you're running on garbage.

[OneCD]

Some good info there, thanks @P3R.

[Gaudi]

I have recently sold 4 units of this legendary model that had between 57 and 73k hours, these drives replaced the infamous ST3000DM001 3Gb drives that were removed from the compatibility list at the time (viewtopic.php?t=117039).

My WD drives had very few Power On Counts (between 80-132), had no bad or realocated sectors and SMART shows good as new.



I still have two new spares that I bought at the time to hot swap in case of an error, but never had to use them. These are too small for nowadays backups, so I will mostly probable sell them as well.


Regards