A management board hand-picked by Icahn would also have taken control of the remaining Yahoo business.
"It is ludicrous to think the our board could accept such a proposal," said Yahoo's chairman, Roy Bostock, in a statement.
"While this type of erratic and unpredictable behaviour is consistent with what we have come to expect from Microsoft, we will not be bludgeoned into a transaction that is not in the best interests of our stockholders."
Earlier last week, Microsoft had issued a statement confirming that it would revive its interest in acquiring Yahoo or some part of it if its current management was replaced.
Previously, Microsoft chief executive Steve Ballmer had insisted he had moved on from his attempts to buy Yahoo or its search business.
Yahoo's current board met on Saturday, before rejecting the latest offer.
It now faces a proxy battle at its forthcoming stockholders meeting, where Icahn is campaigning for the board to be sacked and replaced with a slate of his nominees.