Attention as Implicit Structural Inference has been accepted as a poster presentation for this year’s NeurIPS conference in New Orleans.

This work presents a new perspective on attention mechanisms in machine learning, viewing them as inference over potential adjacency structures in a graphical model, thereby unifying various architectures and suggesting modifications. It explores two new mechanisms, extends the application of attention, and links this approach to precision-regulation in Predictive Coding Networks, offering insights for bridging machine learning and neuroscience concepts of attention.

You can find the pre-print of this paper here