After being the parish priest at Notre-Dame-des-Douleurs in Edmundston for some years, Father Dionne was named Auxiliary Bishop to Archbishop Carter at Sault-Sainte-Marie, Ontario, in 1975. In 1984, he became Bishop of Edmundston and organized a diocesan synod. In 1994, he retired at Hôtel-Dieu in Saint-Basile and the following year, he published his autobiography under the title Petite Histoire.