Do you know what quiet time is? It is fellowship with a holy God. The reason that some people do not have a quiet time is that they feel uncomfortable. They don't want to look God in the face because there is unconfessed, unrepented sin in their life.
What did Adam do after he sinned and God came walking in the Garden? Adam fled. Before that, Adam had quiet time alone with God, didn't he? They conversed, they communed. It was paradise. But when Adam sinned, he did not want to look God in the face.
When Moses went into God's presence, God said, "Draw not nigh hither: put off thy shoes from off thy feet, for the place whereon thou standest is holy ground. Moreover He said, I am the God of thy father, the God of Abraham, the God of Isaac, and the God of Jacob. And Moses hid his face; for he was afraid to look upon God." God cannot be in the presence of sin because He is holy.
If you find a reluctancy to go into the presence of God, there may be unconfessed, unrepented sin in your life. Part of your quiet time is to get your heart clean and pure. Each of us needs to take ourselves by the nap of our necks and confess and repent before we come into God's holy presence to fellowship. Psalm 66 verse 18 says, "If I regard iniquity in my heart, the Lord will not hear me.