Greyhole should default to MySQL due to performance issues - was Re: Greyhole needs more aggressive priority
Greyholes queue gets filled up quickly and it can’t move data off the LZ as quickly as its written. This causes network transfers to stall on the client side with warnings about lack of space.
As the LZ is on the System partition by default this can cause other more serious issues for the system as a whole.
The only reasonable solution I can think of is to tweak greyhole to have a more aggressive approach to priority until it moves data off the LZ marginally quicker than its being written to the LZ when under heavy load.
#1 Updated by gboudreau almost 10 years ago
Can you help find the correct priority values ?
ionice -n 7 -p `cat /var/run/greyhole.pid`
Execute that, but replace 7 with a lower number. (Higher priority is 0.)
Let us know what value worked to fix the issue you're having.
(Maybe try with 0 or 1, then work your way up until the issue re-appears.)
You can monitor how fast greyhole file transfers are going by using iotop:
iotop -u root -o
Look for greyhole or rsync (spawned by greyhole).
The PRIO column should show "be/#", where # is what you used in the ionice command above.
#2 Updated by gboudreau almost 10 years ago
- Assignee set to cpg
- Priority changed from High to Urgent
After doing some tests, I confirmed that changing the priority of GH I/O doesn't change much the speed of file transfers.
But switching from SQLite to MySQL do make a difference:
$ time sudo ./greyhole-test real 1m29.367s
$ time sudo ./greyhole-test real 0m2.955s
The above test takes an old /var/log/messages file, and inserts 1000 tasks in the DB.
It takes 3 seconds to insert those 1k rows using MySQL, and 1m30s using SQLite.
cpg: Unless you find a way to optimize SQLite somehow, I suggest switching to MySQL.
This problem creates problems for most new users trying to migrate a lot of data.
#3 Updated by Zeiva almost 10 years ago
I can backup gboudreau on the priority change. It made no noticeably difference and checking with iotop before and after confirms this. The problem is not the speed of moving data but the speed of handling tasks from the looks of it.
I will be moving to MySQL as soon as my queue is done after which I'll report back with my findings.
I can also note that I've now seen several issues arise from the LZ getting full (as it’s on the OS drive) during the day when monitoring it from work. Ranging from VNC stopping to work correctly to the Amahi webgui giving me database errors (both solved by a reboot from SSH).
#4 Updated by cpg almost 10 years ago
Were you able to (carefully - not on the fly!) switch to MySQL for the DB engine and try to hose it with data to see how well it responds to load?
We'll switch the default, write a small howto and release ... but it would be good to have positive confirmation ...
#5 Updated by Zeiva almost 10 years ago
I'm still letting Greyhole work through the massive queue I've built up during this issue. Once I'm done I have some pretty nice folders to throw at it to test. What would you like me to start with, a 350GB~ folder with 250k of files (tons of pictures etc) or a 2TB folder with tons of large video files? I also have other more mixed folers which will also make for some nice testing I reckon.
I will be using the instructions listed here to switch to MySQL: http://forums.amahi.org/viewtopic.php?f=39&t=1372
Also available in: Atom