could it be you had bluez-gnome installed? since you have an 8gb card and xfce uses gtk the dependencies should not worry you -- the only other graphical passkey agent i know in debian is in kdebluetooth whcih you probaly would remember if you installed it.