Facebook Researchers Teach AI To Disregard Irrelevant Information To Improve Overall Performance

Typically, AI is rather good at various tasks, but when it comes to searching long-term memories, performance drops, and the cost of storage grows exponentially. This can be quite the problem as time goes on since we constantly take in new information that would need to be processed. Thus, mimicking humans' long-term memory could be the solution, as we do not remember everything, just the important and vivid memories.
The Facebook researcher's paper, titled Not All Memories are Created Equal: Learning to Forget by Expiring, proposes an "Expire-Span" method. They claim that this would be a "straightforward extension to attention mechanisms that learns when to expire unneeded memories." This would allow AI systems to process data more efficiently at larger scales with up to "tens of thousands of timesteps into the past."
Expire-Span works by predicting what information is most important and therefore worthwhile to keep based on context. Then, it assigns a time for that information to be removed from memory in the "selective forgetting" process. In the example 'A' above, Expire-Span sees Egypt and Alexander as important. If Egypt is replaced with somewhere, which is a rather generic word, it is quickly expired. If somewhere is then replaced with Humpty Dumpty, it is saved indefinitely as it does not often appear, thus making it more likely to be important.

Though this may be complex, it is interesting that we can improve AI by teaching it to forget things selectively, but it does make sense. If you remembered everything from school, it might have made school easier, but recalling the important things may have become harder. Either way, it is certainly interesting research, so let us know what you think of it in the comments below.