Background In assisted reproductive technology, evaluating the quality of the embryo is crucial when selecting the most viable embryo for transferring to a woman. Assessment also plays an important role in determining the optimal transfer time, either in the cleavage stage or in the blastocyst stage. Several AI-based tools exist to automate the assessment process. However, none of the existing tools predicts upcoming video frames to assist embryologists in the early assessment of embryos. In this paper, we propose an AI system to forecast the dynamics of embryo morphology over a time period in the future. Methods The AI system is designed to analyze embryo development in the past two hours and predict the morphological changes of the embryo for the next two hours. It utilizes a predictive model incorporating Convolutional LSTM layers, to predict the future video frame by analyzing prior morphological changes within the embryos video sequence. The system uses the predictions recursively and forecasts up to 23 hours of embryo development. Results The results demonstrated that the AI system could accurately forecast embryo development at the cleavage stage on day 2 and the blastocyst stage on day 4. The system provided valuable information on the cell division processes on day 2 and the start of the blastocyst stage on day 4. The system focused on specific developmental features effective across both the categories of embryos. The embryos that were transferred to the female, and the embryos that were discarded. However, in the transfer category, the forecast had a clearer cell membrane and less distortion as compared to the avoid category. Conclusion This study assists in the embryo evaluation process by providing early insights into the quality of the embryo for both the transfer and avoid categories of videos. The embryologists recognize the ability of the forecast to depict the morphological changes of the embryo. Additionally, enhancement in image quality has the potential to make this approach relevant in clinical settings.