Unable to access AFP - No inodes left?

[a_carneiro]

Hello!

I have gradually expanded my Readynas Pro 6 to include 6x6TB disks. This is using OS 6.2.4

I have been unable to access any shares via AFP lately, and upon looking at the AFP logs I see the error "cnid_dbd[3191]: Error opening lockfile: No space left on device"

The volume has nearly 4TB of free space so I'm gessing I ran out of inodes. Thing is, I have no idea what to do to fix that or indeed if that is the actual problem

Can anyone suggest how I can diagnose the issue and, should it be that I have no inodes left, what can be done about it?

Thank you very much! 😄

[StephenB]

OS 6 uses BTRFS for the data volume, and that format doesn't have inodes. The OS partition uses ext, so it does have inodes

I'd enable ssh, and look at the OS partition fullness next.  That would be done by

cd //

df . -h

df . -i

The -h gives you free space info (in user-friendly format), the -i gives you inode info.

Are you seeing any other mis-behavior?

[mdgm-ntgr]

Could be a misbalanced data volume perhaps.

# btrfs fi show
# btrfs fi df /data

[a_carneiro]

Thank you for your kind reply

It seems to be working OK over SMB (I can connect and read files. Any attempt to write files will give me a "disk full" kind of error).

I have also been unable to create a test share.

In reply to mdgm, I did rebalance teh volume today  but that doesn't seem to have fixed anything...

Looking at the logs, I have the following entries which may be of relevance for someone who understands this thing (ie. not me)

=== df -h ===
Filesystem Size Used Avail Use% Mounted on
rootfs 4.0G 744M 2.9G 21% /
tmpfs 10M 4.0K 10M 1% /dev
/dev/md0 4.0G 744M 2.9G 21% /
tmpfs 2.0G 44K 2.0G 1% /dev/shm
tmpfs 2.0G 4.5M 2.0G 1% /run
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
tmpfs 2.0G 0 2.0G 0% /media
/dev/md127 28T 24T 3.9T 86% /data
/dev/md127 28T 24T 3.9T 86% /home
/dev/md127 28T 24T 3.9T 86% /apps
/dev/md127 28T 24T 3.9T 86% /run/nfs4/home
=== df -i ===
Filesystem Inodes IUsed IFree IUse% Mounted on
rootfs 0 0 0 - /
tmpfs 505843 527 505316 1% /dev
/dev/md0 0 0 0 - /
tmpfs 505843 2 505841 1% /dev/shm
tmpfs 505843 698 505145 1% /run
tmpfs 505843 4 505839 1% /sys/fs/cgroup
tmpfs 505843 1 505842 1% /media
/dev/md127 0 0 0 - /data
/dev/md127 0 0 0 - /home
/dev/md127 0 0 0 - /apps
/dev/md127 0 0 0 - /run/nfs4/home

So inodes are maybe not the source of trouble.... 😕

[StephenB]

---

@a_carneiro wrote:

So inodes are maybe not the source of trouble.... 😕

---

No, your OS partition information looks fine.

SMB access is working normally?

[a_carneiro]

Other than the inability to write into it (says the disk is full), seems to be working just fine...

[StephenB]

So you can't copy new files onto the NAS with SMB and AFP can't write the lockfile either?

Does the system say the data volume is read-only?

If not, perhaps try a scrub next.

[a_carneiro]

Doesn't say it's read-only, it says it's full
I'll try a scrub but has its only recently had a new HDD replaced, it would have scrubbed the volume wouldn't it?

[StephenB]

Replacing the drive would have rebuilt the data volume, but that isn't the same as a btrfs scrub.

[a_carneiro]

I see
Well, it's scrubbing now. It'll take s while so I'll post an update when it's done.
Thank you for your help 🙂

[mdgm-ntgr]

Ah, so you downloaded the logs zip file (please send it in, see the Sending Logs link in my sig).

[a_carneiro]

Logs emailed Thank you so much for looking into this 🙂

André

[mdgm-ntgr]

You may wish to remove some files to free up some space then try running a balance and see if things work better once that has completed.

We would recommend keeping your usage down to 80% of the volume capacity and scheduling weekly defrags and balances.

[a_carneiro]

Yes, I've removed a few terabytes of data and scrubbing again. I'll rebalance and defrag and let you know how it goes.

Thank you 🙂