Educated at the Séminaire de Chicoutimi, Laval (MA in social sciences, 1942) and Harvard (MEd, 1945), he joined Laval's École de Pédagogie et d'Orientation as assistant-director in 1945. Throughout the 1950s, he pursued the reform of Québec's Catholic education system at all levels. Indeed, he was the architect of most of the reforms that would later be implemented.
When the Quiet Revolution was ushered in with the victory of Jean Lesage's Liberal party in June 1960, Tremblay became executive assistant to the minister of youth, Paul Gerin-Lajoie, and was responsible for preparing recommendations for the reforms of agricultural, technical and professional education. He was also a member of the Commission royale d'enquête sur l'enseignement 1961–64, which recommended sweeping reforms beginning with the re-establishment of the Ministry of Education that had been abolished in 1875.
The Lesage government created the ministry in 1964 and named Tremblay its first deputy minister. Before the decade was out, the ministry had restructured all the school boards and created a comprehensive public secondary system topped off by the College d'enseignement général et professionnel (CEGEP) system, while secularizing and expanding the francophone Catholic universities. In 1971 he succeeded Claude Morin as deputy minister of Intergovernmental Affairs. He was named to the Senate in September 1979, and in 1984 Prime Minister Brian Mulroney made him special adviser on constitutional affairs and federal-provincial relations. Tremblay retired from the Senate in 1992. He was named an Officer of the Order of Canada in 1976 and an Officer of the Ordre National du Québec in 1991.