Watch THE CROWN on Netflix and you will see what The King subjected these 2 boys too whilst growing up.
What he did and put their mother through. He only had eyes for Camilla even put her before his own boys.
So really he doesnt deserve to see his grandkids he just needs to keep his eyes on...