smartctl is complaining about /dev/sdf on wiwa

[https://id.mayfirst.org/jamie]

Email:

This message was generated by the smartd daemon running on:

   host name:  wiwa
   DNS domain: mayfirst.org

The following warning/error was logged by the smartd daemon:

Device: /dev/sdf [SAT], 1 Currently unreadable (pending) sectors

Device info:
ST3000NM0033-9ZM178, S/N:Z1Z96JTK, WWN:5-000c50-086ca530c, FW:SN04, 3.00 TB

For details see host's SYSLOG.

You can also use the smartctl utility for further investigation.
The original message about this issue was sent at Mon Jan 28 00:22:32 2019 EST
Another message will be sent in 24 hours if the problem persists.

The RAID array is still intact.

[https://id.mayfirst.org/jaimev]

What is usually next step for following up or monitoring this jamie?

[https://id.mayfirst.org/jamie]

Replacement. I was curious and found ​this thread on fixing an unreadable sector - it looks like there are a lot of ideas, but ultimately, a bad sector could be a sign that the drive is failing so best just to replace it.

I plan to go tomorrow morning before our meeting.

[https://id.mayfirst.org/jamie]

I've started by failing and removing sdf:

0 wiwa:~# cat /proc/mdstat 
Personalities : [raid1] [raid10] 
md2 : active raid1 sda1[0] sdb1[1]
      468748992 blocks super 1.0 [2/2] [UU]
      bitmap: 2/4 pages [8KB], 65536KB chunk

md1 : active raid10 sdc3[0] sdf3[3] sde3[2] sdd3[1]
      5859319808 blocks super 1.0 512K chunks 2 near-copies [4/4] [UUUU]
      
md0 : active raid1 sdc2[0] sdf2[3] sde2[2] sdd2[1]
      499648 blocks super 1.0 [4/4] [UUUU]
      
unused devices: <none>
0 wiwa:~# mdadm /dev/md0 --fail /dev/sdf2
mdadm: set /dev/sdf2 faulty in /dev/md0
0 wiwa:~# mdadm /dev/md0 --remove /dev/sdf2
mdadm: hot removed /dev/sdf2 from /dev/md0
0 wiwa:~# mdadm /dev/md1 --fail /dev/sdf3
mdadm: set /dev/sdf3 faulty in /dev/md1
0 wiwa:~# mdadm /dev/md1 --remove /dev/sdf3
mdadm: hot removed /dev/sdf3 from /dev/md1
0 wiwa:~# cat /proc/mdstat 
Personalities : [raid1] [raid10] 
md2 : active raid1 sda1[0] sdb1[1]
      468748992 blocks super 1.0 [2/2] [UU]
      bitmap: 2/4 pages [8KB], 65536KB chunk

md1 : active raid10 sdc3[0] sde3[2] sdd3[1]
      5859319808 blocks super 1.0 512K chunks 2 near-copies [4/3] [UUU_]
      
md0 : active raid1 sdc2[0] sde2[2] sdd2[1]
      499648 blocks super 1.0 [4/3] [UUU_]
      
unused devices: <none>
0 wiwa:~#

[https://id.mayfirst.org/jamie]

Disk is replaced, running smartctl short and long tests.

I'm to wait to sync until i can monitor more closely since sync'ng sometimes causing disk i/o problems.