Openmoko Bug #2131: "opkg upgrade" slow, duplicate entries in *.list files

Openmoko Public Trac bugs at docs.openmoko.org
Sat Nov 22 11:36:16 CET 2008


#2131: "opkg upgrade" slow, duplicate entries in *.list files
----------------------+-----------------------------------------------------
 Reporter:  h.koenig  |          Owner:  openmoko-devel
     Type:  defect    |         Status:  new           
 Priority:  normal    |      Milestone:                
Component:  unknown   |        Version:                
 Severity:  normal    |       Keywords:                
 Haspatch:  0         |      Blockedby:                
Estimated:            |    Patchreview:                
 Blocking:            |   Reproducible:                
----------------------+-----------------------------------------------------
 root at om-gta02:~# opkg list_installed opkg
 opkg - 0.1.4+svnr4640-r0.1 -

 root at om-gta02:~# time opkg upgrade
 Package ncurses-terminfo is already installed in root.

 real    1m9.038s
 user    0m56.890s
 sys     0m7.390s

 (this was much worse (2.5 to 3 minutes) before removing ncurses-
 terminfo.list -- see below!)


 questions/bugs:

 - why do *.list files grow with many duplicate entries ?

 - why does this cause reading gtk+-fastscaling.list many times ?

 - what's the problem with ncurses-terminfo ?


 BTW, this was an OM2008.8 installation with all updates, now updated to
 testing.



 with strace I noticed that one single pkg file list got opened 12294
 times.  once this was //usr/lib/opkg/info/ncurses-terminfo.list  which
 then I moved away and opkg upgrade was fast again.  right now it's
 //usr/lib/opkg/info/gtk+.list -- from strace output:

 root at om-gta02:~# sort opkd.strace.open1 | uniq -c | sort -n | tail -2
       1 syscall_983045(0x40305a70, 0x40305a70, 0x68c, 0x40306148,
 0x40024060, 0, 0x60, 0xf0005, 0xffff566c, 0x608, 0x40024000, 0xbedcbaec,
 0, 0xbedcb7c0, 0x40001fa8, 0x40001fc0, 0x20000010, 0x40305a70, 0, 0, 0,
 0xd708, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = 0
   12294 open("//usr/lib/opkg/info/gtk+.list", O_RDONLY) = 4

 why does opkg (sometimes) open a pkg list so many times ?   I kept this
 files in case someone wants to inspect them.


 unfortuneately this time (re)moving /usr/lib/opkg/info/gtk+.list does not
 help to get opkg upgrade working fast again :-(

 right now opkg upgrade runs 20 secs after opening
 //usr/lib/opkg/info/gtk+-fastscaling.list and another 20 secs after
 //usr/lib/opkg/info/xtscal.list (from strace -rtt -e open ...)


 xtscal.list is the last list being read, so that's the "real" opkg work.
 (re)moving gtk+-fastscaling.list saves 20 secs of runtime!

 checking gtk+-fastscaling.list  I noticed that it's very long and some
 times are duplicates _many_ times:

          sort gtk+-fastscaling.list  | uniq -c  | grep -v " 1 "
             1024 /usr/bin/gdk-pixbuf-query-loaders
            11264 /usr/lib/gtk-2.0/2.10.0/engines/libpixmap.so
               11 /usr/lib/libgdk_pixbuf_xlib-2.0.so.0

 so I "cleaned up" all *.list files with "sort -u"  reducing from 29816
 lines in all *.list files to 10371 lines!!

 now ncurses-terminfo.list still got read 2424 times, and gtk+.list 8 times
 , running for ~2.5 minutes!   without ncurses-terminfo.list  it runs only
 28 secs, gtk+.list does not matter in run time.


 so removing the duplicate file names i *.list files reduced runtime at
 least from 60+ to 30- seconds.


 I'd guess that the message

     Package ncurses-terminfo is already installed in root.

 is somehow related with the fact that ncurses-terminfo.list still is read
 2k times taking ~2 minutes. the reason for reading gtk+-fastscaling.list
 many times seemed to be fixed by cleaning up *.list files...

-- 
Ticket URL: <https://docs.openmoko.org/trac/ticket/2131>
docs.openmoko.org <http://docs.openmoko.org/trac/>
openmoko trac


More information about the devel mailing list