I respect the idea that energy is how we exact time upon the world, however there also has to be the idea that time is moving of its own accord. If time is its own essence, and moves on its own whether or not we observe it. For example, if it continues to move even when we are asleep, then we must assume that perhaps it is of an energy source not of ourselves.