The Halacha requires us to only hold those items needed for a mitzvah purpose such as a siddur and lulav (at the right time) during prayer so that we not disturb our concentration.
See All 511 Episodes of "Dirshu Mishnah Berurah"