When this happens it is usually normal behavior. Empty VAST/DAAST ads are expected under certain circumstances, such as:
Frequency caps prevent ads from being delivered.
No programmatic ads are available to the listener because the targeting parameters (age, gender, location, etc.) were not matched.
The main thing to consider is that your implementation of the ad requests must support the use case in which an empty VAST/DAAST is returned.