Non-Markovian processes are widespread in natural and human-made systems, yet explicit modeling and analysis of such systems is underdeveloped. We consider a non-Markovian dynamic network with random link activation and deletion (RLAD) and heavy-tailed Mittag-Leffler distribution for the interevent times. We derive an analytically and computationally tractable system of Kolmogorov-like forward equations utilizing the Caputo derivative for the probability of having a given number of active links in the network and solve them. Simulations for the RLAD are also studied for power-law interevent times and we show excellent agreement with the Mittag-Leffler model. This agreement holds even when the RLAD network dynamics is coupled with the susceptible-infected-susceptible spreading dynamics. Thus, the analytically solvable Mittag-Leffler model provides an excellent approximation to the case when the network dynamics is characterized by power-law-distributed interevent times. We further discuss possible generalizations of our result.