Down CNN's Memory Hole