Nick Donofrio, 63, head of technology strategy, will step down in October after 44 years with the company he joined while still in college.
William Zeitler, 61, head of IBM's hardware group, will retire on 1 August after 39 years at IBM.
Zeitler's replacement has already been named as current supply chain manager Robert Moffat, but there is no news on Donofrio's position.
This has led some to think that his post may be abolished altogether as part of a wider company reshuffle.
IBM, once the ultimate blue-chip computer company, has undergone major changes in the past five years.
After selling off the PC and laptop division to Lenovo the company has been reconfiguring itself as a services provider with a particular emphasis on open source development.
Zeitler and Donofrio have stayed at the company longer than is usual, as IBM prefers its executives to retire at 60.
IBM is hoping that a new generation of managers will rise to the top of the company.
IBM reshuffle as top execs retire
By Iain Thomson on May 7, 2008 7:40AM