Baloo: tags missing

[Richard Z]

On Thu, Aug 21, 2014 at 05:10:58PM +0100, Piotr Gbyliczek wrote:
> On Thursday 21 August 2014 16:50:18 Piotr Gbyliczek wrote:
> > > so stop it (just kill it, there's no other way) and run the migrator -
> > > this
> > > time it could work. Then start baloo_file again.
> > 
> > A bit better, no lock errors, but still crashing :
> [...]
> > Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseA: Too many
> > open files
> [...]
> > Any ideas ?
> 
> In fact, upping the file limit from 1024 with 
> 
> $ulimit -n 4096
> 
> and going through the procedure again helped. 
> 
> Rc file updated with migrated=true and tags are present.


I did recently go again through this after migrating one of my machines
to F21.

Automatic migration did not happen or crashed without any notice. After
some googling, trial and error I did 

vim /etc/security/limits.conf # very big number of file descriptors

balooctl stop; ulimit -S -n 100000; rm ~/.kde/share/config/nepomukserverrc; nepomukctl start; sleep 600; nepomukbaloomigrator

I have repeated the commands several times because nepomukserver died
or crashed or the migrator could not get some locks or whatever. Also, 
the migrator does not give any success or progress messages so I was not 
really confident if it finished succesfully.

The file descriptor limit definitely must be high enough so the migrator
does not complain, otherwise the tags of some files simply won't be
migrated. Afaics the number must be 3x the number of files with tags
or comments so I was lucky that it still worked for me.

I did open a bug report upstreams about this:
  https://bugs.kde.org/show_bug.cgi?id=342832

Looking at the file descriptor leak I see something like this:

lr-x------. 1 rz rz 64 Jan 14 11:14 929 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 93 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 930 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 931 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 932 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 933 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 934 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 935 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 936 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 937 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 938 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 939 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 94 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 940 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 941 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 942 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 943 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 944 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 945 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 946 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 947 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 948 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 949 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 95 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 950 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 951 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 952 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 953 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 954 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 955 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 956 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 957 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 958 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 959 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 96 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 960 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 961 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 962 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 963 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 964 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 965 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 966 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 967 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 968 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 969 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 97 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 970 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 971 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 972 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 973 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 974 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 975 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 976 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 977 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 978 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 979 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 98 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 980 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 981 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 982 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 983 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 984 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 985 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 986 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 987 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 988 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 989 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 99 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 990 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 991 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 992 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 993 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 994 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 995 -> /home/rz/.local/share/baloo/file/position.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 996 -> /home/rz/.local/share/baloo/file/postlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 997 -> /home/rz/.local/share/baloo/file/record.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 998 -> /home/rz/.local/share/baloo/file/termlist.DB
lr-x------. 1 rz rz 64 Jan 14 11:14 999 -> /home/rz/.local/share/baloo/file/position.DB

Richard

---
Name and OpenPGP keys available from pgp key servers