- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
MySQLDB will not start code 22002010000
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
MySQLDB will not start code 22002010000
Good morning everyone
ReadyNAS 314
Firmare 6.10.5 Hotfix 1
Installed apps:
MySQLDB 1.0.5 from ReadyNAS Xtras
Nextcloud 20.0.1-rnx1 from ReadyNAS Xtras
PHP7.4 from ReadyNAS Xtras
PHPMyAdmin 4 from ReadyNAS Xtras
This morning I woke up to this annoying error. MySQLDB turned itself off and will no longer start. "The app failed to start, Please update or contact developer. Code 22002010000"
I see in this thread https://community.netgear.com/t5/ReadyNAS-Storage-Apps-Current/MYSQL/m-p/1113575 that the answer is to "bind address to my IP but after changing it back to 127.0.0.1 it is now back working."
Unfortunately, I've no idea how to do that.
I've e-mailed ReadyNAS Xtras for help but they did not respond yet.
If anyone could help I'd be greatful.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
@BuckoA512 wrote:
I see in this thread https://community.netgear.com/t5/ReadyNAS-Storage-Apps-Current/MYSQL/m-p/1113575 that the answer is to "bind address to my IP but after changing it back to 127.0.0.1 it is now back working."
Not quite. The original poster was saying that he had changed the binding to his IP address, but that changing it back to 127.0.0.1 resolved his problem. It's a 5 year-old thread, and it is possible that your problem is caused by something different. In any event, that setting should be in /etc/mysql/my.cnf It's conceivable it's in a different folder - if so, mysqld --help --verbose should find it. Note you need to use ssh.
Did this happen immediately after you updated to 6.10.5?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
Hi StephenB thank you for replying.
I've never had to ssh since my university days so I'm a little rusty on that to say the least.
No, it didn't happen right after I updated. Everything was working fine yesterday, then today this happens and MySQLDB won't start.
ReadyNAS Xtras has replied to me by e-mail too so I'll update if he can figure anything out.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
It looks like the failure is down to a bad disk in the array, or so it seems from the logs. I didn't realise that downloading the logs gave you more information than you get on the web page.
I thought if a bad disk was detected the ReadyNAS would alert me via notiifcation, rather surprised that it didn't.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
@BuckoA512 wrote:
It looks like the failure is down to a bad disk in the array, or so it seems from the logs. I didn't realise that downloading the logs gave you more information than you get on the web page.
Aug 31 10:13:48 ReadyNAS mysqld[6853]: 2021-08-31 10:13:48 7fd324f26780 InnoDB: Error: Write to file ./nextcloud/oc_filecache.ibd failed at offset 25690112.Aug 31 10:13:48 ReadyNAS mysqld[6853]: InnoDB: 8192 bytes should have been written, only 0 were written.
Not convinced it's a failed disk, though is it clearly a failed write.
How full is your OS partition (root)? You can see this by looking at volume.log - scroll down to the df -h section, and look at the /dev/md0 line.
=== df -h === Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 524M 3.2G 15% /
Similarly, how full is your data volume?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
=== df -h === Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 2.1G 1.6G 58% /
ReadyNAS data has 4.40tb free space. It has 4 x 3tb drives in Xraid.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
@BuckoA512 wrote:
=== df -h === Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 2.1G 1.6G 58% /ReadyNAS data has 4.40tb free space. It has 4 x 3tb drives in Xraid.
Fuller than usual, but not dangerously full.
Are you seeing any concerning smart stats for your disks in volume.log?
Also, are you seeing any other I/O errors in system.log and kernel.log
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
Nothing in volume log but I'm getting read/write errors in backup logs and stuff.
Also see this in kernel.log
Aug 30 16:58:20 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 6, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:17:43 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 7, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:18:42 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 8, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:18:45 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 9, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:18:46 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 10, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:22:24 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 11, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:22:32 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 12, rd 359, flush 0, corrupt 0, gen 0
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
@BuckoA512 wrote:
Aug 30 16:58:20 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 6, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:17:43 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 7, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:18:42 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 8, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:18:45 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 9, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:18:46 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 10, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:22:24 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 11, rd 359, flush 0, corrupt 0, gen 0
Aug 30 17:22:32 ReadyNAS kernel: BTRFS error (device md126): bdev /dev/md127 errs: wr 12, rd 359, flush 0, corrupt 0, gen 0
Ok. Try running the disk test from the volume settings wheel next. It's a long test (can be a few days if you have large disks).
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
Ran the check no errors detected, but backup still failed with read errors from my data so obviously something is wrong.
At this point I no longer trust the disks. I'm going to swap the disks one by one and rebuild the array. Hopefully that completes without further errors.
Have to say quite disappointed with this, you never know how reliable your systems are until something goes wrong like this and the ReadyNAS has not handled this gracefully at all. All I can hope now is there's no bit rot on data I don't use/access very often that doesn't turn up until years later.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
@BuckoA512 wrote:
Ran the check no errors detected
You mean you ran it earlier? There's no way it would have completed so quickly (with a pass) if you'd run it when I suggested it a few minutes ago. It runs in the background, and will eventually give you an indication of completion in the logs.
@BuckoA512 wrote:
At this point I no longer trust the disks. I'm going to swap the disks one by one and rebuild the array.
If you have a disk problem, your process will almost certainly result in a lost volume.
You could alternatively do a BTRFS scrub from the volume settings wheel, or test them in a Windows PC with vendor tools (with the NAS powered down).
FWIW, I am thinking file system corruption is more likely than a disk failure (though I can't rule out the disk failure from what you've posted so far). Normally I'd expect to see another error telling me what disk couldn't be read or written, and we are not seeing that here. You might check kernel.log and look for disk errors as well as btrfs errors.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
I started the disk test yesterday and it completed today.
"If you have a disk problem, your process will almost certainly result in a lost volume."
Well that's great, I thought the whole point of RAID was to protect against single disk failure?
What's even the point of this thing if I will almost certainly get a lost volume as soon as an error like this comes up?
Lucky I have a full backup I suppose!
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
@BuckoA512 wrote:
Well that's great, I thought the whole point of RAID was to protect against single disk failure?
Single redundancy RAID does protect against a routine disk failure. But your process requires dual redundancy.
When you re-insert a disk, it is rebuilt from scratch from the remaining disks. During that process you have no redundancy at all. If one of the other disks is the problem, then the rebuild will fail and you lose everything. This is normal RAID stuff, not related to the ReadyNAS specifically.
A scrub from the volume settings wheel is a better option.
@BuckoA512 wrote:
Lucky I have a full backup I suppose!
Hopefully not lucky - RAID isn't enough to keep your data safe, so you do need to back up the ReadyNAS if it is used for primary storage.
FWIW, if the disks turn out to be ok, then I suggest doing a factory default, reconfiguring the NAS, and restoring the data from the backup. Make sure you have all the app deb files, as you will need to reinstall them.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
I have started a data scrub as per your advice.
Yeah I understand RAID isn't a backup in its own right. You're suggesting two or more disks may be faulty here, that's always a possibility yet the ReadyNAS can't seem to decide/detect where the problem may lie.
Doing a factory default and reconfiguring everything from scratch is quite time consuming and the kind of situation I hoped haivng a RAID array would avoid.
Guess we'll see what the scrub comes back with.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
@BuckoA512 wrote:
You're suggesting two or more disks may be faulty here, that's always a possibility yet the ReadyNAS can't seem to decide/detect where the problem may lie.
Actually I haven't suggested that. I said that you have no RAID redundancy at all when you are rebuilding a disk. If we assume that one disk is bad, then the volume will fail if you rebuild one that isn't bad. The odds are that the first disk you try to rebuild will be one of the healthy ones.
And I have said a couple of times that I suspect the problem is not the disks at all, but might instead be file system corruption (which has other possible causes). But it is important to rule the disks out. I agree that the OS 6 ReadyNAS could do a better job in reporting/finding disk and file system problems.
FWIW, after you get past this, you might want to schedule the disk test and the scrub. Personally I run all four of the maintenance tasks on schedule (one per month). The scrub doubles as a reasonable disk test, as it does require every sector in the data volume to be either read or written. So scheduling those two every now and then is a good way to give you earlier warning on disk issues.
@BuckoA512 wrote:
Doing a factory default and reconfiguring everything from scratch is quite time consuming and the kind of situation I hoped haivng a RAID array would avoid.
Totally get that, and generally it isn't needed. But sometimes it's the best path forward.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
"Actually I haven't suggested that. I said that you have no RAID redundancy at all when you are rebuilding a disk. If we assume that one disk is bad, then the volume will fail if you rebuild one that isn't bad. The odds are that the first disk you try to rebuild will be one of the healthy ones."
Ah yes of course since we can't tell which disk (if any) is bad, now I think about it that makes total sense.
"FWIW, after you get past this, you might want to schedule the disk test and the scrub. Personally I run all four of the maintenance tasks on schedule (one per month). The scrub doubles as a reasonable disk test, as it does require every sector in the data volume to be either read or written. So scheduling those two every now and then is a good way to give you earlier warning on disk issues."
Good advice, thanks.
"Totally get that, and generally it isn't needed. But sometimes it's the best path forward."
Do you happen to know, If I back up the configuration, factory reset, and then re-load it, does that restore everything to how it was, users, passwords etc, just with all the shares empty and the apps uninstalled?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
Actually, you need to reinstall the apps before you restore the saved configuration because some of the files in the configuration .zip may have been modified by an app and putting them in without the app installed can result in problems.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: MySQLDB will not start code 22002010000
Makes sense, though I think I'll start from scratch with Nextcloud anyway if it comes to that.